-- Logs begin at Thu 2022-05-19 18:37:42 UTC, end at Thu 2022-05-19 21:11:49 UTC. -- May 19 19:45:11.408507 dv-cinder-841302-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 19:45:11.418870 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 19:45:11.421140 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 19:45:11 2022] *** May 19 19:45:11.421140 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 19:45:11.421140 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 19:45:11.421140 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: nodename: dv-cinder-841302-4-iscsi May 19 19:45:11.421140 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: machine: x86_64 May 19 19:45:11.421140 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: clock source: unix May 19 19:45:11.421140 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: pcre jit disabled May 19 19:45:11.421140 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: detected number of CPU cores: 4 May 19 19:45:11.421140 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: current working directory: / May 19 19:45:11.421140 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: detected binary path: /usr/bin/uwsgi-core May 19 19:45:11.421819 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: your processes number limit is 47680 May 19 19:45:11.421819 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: your memory page size is 4096 bytes May 19 19:45:11.421819 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: detected max file descriptor number: 2048 May 19 19:45:11.421819 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: lock engine: pthread robust mutexes May 19 19:45:11.421819 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: thunder lock: enabled May 19 19:45:11.421819 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 19:45:11.421819 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 19:45:11.467832 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: Python main interpreter initialized at 0x560ca89619c0 May 19 19:45:11.467832 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: python threads support enabled May 19 19:45:11.467832 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: your server socket listen backlog is limited to 100 connections May 19 19:45:11.467832 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: your mercy for graceful operations on workers is 90 seconds May 19 19:45:11.468211 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: mapped 403077 bytes (393 KB) for 2 cores May 19 19:45:11.468285 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: *** Operational MODE: preforking *** May 19 19:45:11.468558 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: *** uWSGI is running in multiple interpreter mode *** May 19 19:45:11.468558 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: spawned uWSGI master process (pid: 80441) May 19 19:45:11.468784 dv-cinder-841302-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 19:45:11.470301 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: spawned uWSGI worker 1 (pid: 80452, cores: 1) May 19 19:45:11.470301 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: spawned uWSGI worker 2 (pid: 80453, cores: 1) May 19 19:45:11.470301 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 19:45:17.257042 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:45:17.257042 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: from cryptography.utils import int_from_bytes May 19 19:45:17.257880 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:45:17.257880 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: from cryptography.utils import int_from_bytes May 19 19:45:17.269867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:45:17.269867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: from cryptography.utils import int_from_bytes May 19 19:45:17.270468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 19:45:17.270468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: from cryptography.utils import int_from_bytes May 19 19:45:17.309367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 19:45:17.309569 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 19:45:17.309724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] command line args: [] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 19:45:17.310001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 19:45:17.310175 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ================================================================================ {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 19:45:17.310423 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] admin_token = **** {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.310662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] config_dir = [] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.310844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.311022 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] config_source = [] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.311211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] debug = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.311453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.311953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.312039 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.312235 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.312508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.312735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] list_limit = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.312909 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] log_config_append = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.313082 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.313254 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] log_dir = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.313427 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] log_file = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.313670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.313900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.314141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.314381 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.314554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.314726 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.314965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.315204 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.315430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.315656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.315882 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.316108 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.316368 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.316632 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.316933 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.317477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.317627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] publish_errors = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.317859 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.318079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.318251 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.318492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.318720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.318949 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] standard_threads = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.319042 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 19:45:17.319173 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.319251 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 19:45:17.319411 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] command line args: [] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 19:45:17.319492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.319583 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.319693 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 19:45:17.319771 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] use_journal = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.319874 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ================================================================================ {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 19:45:17.319950 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] use_json = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.320068 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] use_stderr = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.320150 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] admin_token = **** {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.320277 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] use_syslog = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.320394 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] config_dir = [] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.320469 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.320579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.320729 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.320811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] config_source = [] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.320931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] debug = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.321007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.321138 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.321244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.321380 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.321465 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.321563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.321665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.321827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.321943 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.322093 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.322180 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.322313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] list_limit = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.322391 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.322495 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] log_config_append = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.322571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.322674 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.322804 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.322886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] log_dir = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.323015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] log_file = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.323106 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.323250 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.323323 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.323495 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.323639 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.323736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.323803 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.323947 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.324016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.324144 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.324233 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.324321 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.324430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.324678 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.324844 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.324912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.325079 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.325162 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.325314 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.325387 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.325550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.325623 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.325785 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.325864 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.326032 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.326124 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.326280 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.326354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.326533 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.326656 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.326765 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.326995 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.327080 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.327218 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] publish_errors = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.327305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.327451 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.327528 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.327683 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.327765 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.327848 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.328094 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.328183 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.328330 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.328425 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.328577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] standard_threads = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.328685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.328811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.328925 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.329060 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.329172 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.329259 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.329404 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] use_journal = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.329480 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.329582 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] use_json = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.329666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.329746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] use_stderr = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.329852 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.329935 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] use_syslog = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.330078 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.330161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 19:45:17.330304 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.330392 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.330530 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.330613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.330756 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.330872 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.330989 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.331117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.331233 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.331369 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.331452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.331610 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.331686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.331860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.331938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.332100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.332184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.332328 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.332437 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.332591 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.332670 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.332845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.332921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.333083 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.333213 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.333333 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.333431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.333526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.333657 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.333791 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.333889 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.333997 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.334130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.334241 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.334338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.334469 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.334598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.334683 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.334745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.334919 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.335001 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.335146 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.335236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.335402 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.335481 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.335635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.335731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.335867 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.335995 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.336083 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.336192 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.336319 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.336438 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.336684 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.336923 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.337100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.337204 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.337407 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.337663 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.337779 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.337901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.338001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.338132 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.338223 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.338391 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.338475 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.338634 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.338715 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.338866 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.338958 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.339121 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.339201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.339347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.339461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.339563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.339828 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.339927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.340125 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.340223 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.340377 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.340490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.340615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.340693 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.340854 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.340941 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.341146 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.341309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.341375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.341475 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.341631 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.341710 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.341848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.341937 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.342066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.342179 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.342246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.342305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.342478 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.342572 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.342705 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.342785 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.342939 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.343030 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.343160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.343249 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.343390 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.343506 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.343648 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.343729 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.343854 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.343946 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.344089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.344168 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.344293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.344397 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.344959 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345106 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345165 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345281 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345281 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345611 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345688 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345750 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345823 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345891 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.345986 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.346164 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] auth.external = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.346252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.346328 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.346446 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.346596 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.346676 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.346832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.346947 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.347062 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] auth.password = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.347190 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.347275 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] auth.token = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.347443 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.347534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.347693 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.347771 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.347933 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.348011 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.348170 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.348257 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.348976 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.349061 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.349224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.349308 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.349454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.349540 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.349717 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] credential.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.349806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.349932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.350009 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.350173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.350260 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.350447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.350541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.350644 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.350721 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.350881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.350969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.351111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.351187 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.351370 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.351448 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.351610 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.351694 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.351872 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.351987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.352073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.352147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.352324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.352456 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.352588 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.352706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.352827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.352967 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.353067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.353207 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.353311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.353459 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.353539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.353691 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] federation.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.353768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.353945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.354023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.354209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.354288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.354453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.354531 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.354683 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.354766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.354919 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.354995 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.355153 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.355232 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.355400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.355536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.355626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.355806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.355885 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.356056 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.356136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.356300 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] auth.external = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.356493 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.356570 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.356639 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.356794 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.356873 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.357076 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.357163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.357313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] auth.password = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.357405 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.357543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.357621 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] auth.token = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.357783 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.357862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.358012 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.358113 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.358251 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.358356 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.358483 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.358562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.358733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.358812 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.358962 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.359039 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.359190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.359282 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.359418 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.359497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] credential.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.359656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.359735 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.359900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.359979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.360141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.360220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.360403 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.360486 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.360650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.360736 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.360885 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.360963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.361143 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.361220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.361378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.361459 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.361606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.361686 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.361838 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.361917 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.362065 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.362146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.362296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.362375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.362534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.362635 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.362825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.362905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.363083 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.363167 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.363311 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.363390 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] federation.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.363555 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.363636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.363796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.363875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.364075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.364714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.364865 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.364947 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.365119 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.365199 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.365347 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.365435 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.365581 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.365662 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.365810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.365888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.366044 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.366123 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.366278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.366358 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.366516 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.366595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.366749 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.366828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.366983 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.367063 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.367231 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.367309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.367471 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.367548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.367698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.367777 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.367925 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.368004 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.368165 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.368242 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.368432 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.368520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.368682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.368772 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.368914 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.368992 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.369146 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.369227 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.369373 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.369452 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.369603 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.369680 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.369851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.369933 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.370087 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.370164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.370330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.370420 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.370571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.370659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.370811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.370897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.371041 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.371128 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.371266 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.371346 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.371496 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.371574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.371726 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.371817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.371965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.372044 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.372192 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.372270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.372453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.372548 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.372696 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.372776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.372938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.373017 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.373181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.373267 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.373412 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.373490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.373636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.373713 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.373878 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.373963 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.374111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.374180 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.374367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.374444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.374575 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.374653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.374806 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.374874 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.375050 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.375126 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.375282 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.375360 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.375512 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.375632 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.375748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.375816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.375975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.376051 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.376215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.376485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.376548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.376820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.376917 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.376986 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.377046 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.377154 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.377222 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.377391 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.377468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.377630 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.377708 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.377893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.377970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.378085 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] resource.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.378232 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.378332 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.378410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.378570 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.378649 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.378800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.378886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.379040 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.379118 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.379283 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.379360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.379513 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.379622 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.379797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.379877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.380030 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.380108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.380266 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.380386 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.380520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] role.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.380600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.380764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.380841 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] role.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.380991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.381081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.381228 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.381306 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.381466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.381545 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.381698 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.381786 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.381936 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.382014 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.382178 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.382257 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.382419 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.382500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.382661 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.382739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.382903 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.382981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.383131 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.383211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.383372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.383451 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.383608 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.383686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.383846 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.383927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.384084 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.384163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.384321 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.384544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.384636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.384777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.384856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.385022 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.385100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.385268 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.385354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.385514 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.385593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.385758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.385836 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.385989 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.386085 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.386228 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.386308 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.386467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.386547 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.386699 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.386785 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.386928 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.387008 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.387162 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.387241 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.387389 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.387479 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.387630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] resource.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.387707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.387876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.387953 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.388119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.388206 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.388376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.388460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.388643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.388722 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.388888 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.388976 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.389120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] token.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.389199 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.389351 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.389430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.389591 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.389677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.389821 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.389899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.390058 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] role.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.390142 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.390301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.390387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] role.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.390532 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.390610 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.390768 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.390850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.391006 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.391093 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.391255 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.391334 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.391505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.391583 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.391737 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.391816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.391968 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.392046 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.392210 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.392288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.392477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.392558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.392704 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.392782 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.392942 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.393020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.393188 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.393273 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.393437 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.393516 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.connection = **** {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.393686 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.393774 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.393931 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.394113 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.394202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.394355 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.394434 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.394606 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.394685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.394836 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.394922 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.395067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.395145 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.395321 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.395403 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.395562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.395654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.395792 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.395871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.396030 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.396109 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.396261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.396372 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.396527 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.396608 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.396759 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.396838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.396999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.397079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.397253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.397332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.397512 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.397740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.397820 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.397892 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80453) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 19:45:17.398007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.398249 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.398485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.398724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] token.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.398964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.399216 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.399455 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.399713 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.399962 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.400204 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.400473 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.400722 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.400976 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.401211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.401451 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.401689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.401937 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.402187 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.402420 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.402664 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.402981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.403221 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.connection = **** {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.403468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.403716 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.403956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.404199 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.404459 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.404702 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.404940 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.405174 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.405419 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.405657 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.405902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.406144 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.406392 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.406630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.406876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.407117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.407360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.407598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 19:45:17.407773 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80452) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 19:45:17.430594 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.430811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.431030 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.434396 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.434630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.434859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.439890 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.445164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.468756 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.469023 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.471810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.472041 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.523341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.523606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.577020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.577332 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.577762 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.577993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.644071 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.644361 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.644532 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.644532 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.644807 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.644943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.645075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.645202 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.645342 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.645468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.645595 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.645595 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.668814 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.669073 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.669215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.669345 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.669471 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.669599 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.669735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.669863 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.669989 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.670115 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.670241 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.670417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.670551 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.670551 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.682629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.682812 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.696097 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.696405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.696653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.696836 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.697053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.697271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.697481 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80452) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.698120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.698344 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.698617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.698845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.699057 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.699266 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.699474 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80453) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 19:45:17.714885 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.715093 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:45:17.715353 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.715507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.715589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:45:17.715663 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:45:17.715868 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.715950 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.716106 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 19:45:17.716188 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.716391 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.716476 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.716652 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.716748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.716903 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.717175 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.728380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.728666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.728934 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.729210 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.729624 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.729901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.730174 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.730455 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.737409 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.738609 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.741278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.741550 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.741827 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.742213 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.742493 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.742576 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.742752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.742839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.743016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.743108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.743285 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.743439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.743552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.743722 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.743812 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.743987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.744068 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.744258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.744558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.744830 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.745103 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.745372 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.771140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.772766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.772958 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.773062 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.774619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.774918 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.778294 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.778569 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.778842 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.780432 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.780713 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.780994 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.786153 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.786430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.788495 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.788797 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.791218 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.791473 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.791755 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.792028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.792295 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.792588 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.792855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.793116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.793573 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.793876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.794162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.794443 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.794717 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.794993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.795256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.795524 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.813092 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.813386 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.815676 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.815973 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.818617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.821354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.822458 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.822763 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.825129 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.825445 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.827911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.830669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.831566 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.834507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.835525 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.835832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.836112 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.836499 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.836712 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.836995 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.838398 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.838703 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.838994 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.839284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.839564 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.839852 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.856379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.856780 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.857050 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.857318 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.859585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.859990 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.860266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.860567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.865772 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.867473 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.867746 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.869008 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.870725 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.870923 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.871056 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.871188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.871425 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.871705 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.871978 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.874386 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.874669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.874972 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.875249 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.886062 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.886347 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.886647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.886904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.887178 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.887452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.887730 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.889484 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.889774 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.890077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.890359 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.890637 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.890925 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.891183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.905767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.909000 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.909398 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.912880 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.913254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.914459 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.916078 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.916426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.916596 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.916725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.918180 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.919825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.920111 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.920418 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.924118 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.925752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.928464 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.929863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.930149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.930342 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.930426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.930686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.933648 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.933938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.934223 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.934500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.939927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.940243 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.940549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.943826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.944110 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.944416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.948450 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80453) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.948726 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.949005 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.949280 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.949542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.949819 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.950084 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.950348 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.950608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.950874 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.951178 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.951435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.951697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.951954 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80453) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.952439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80452) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 19:45:17.952721 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.952999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.953271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.953555 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.953837 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.954117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.954385 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.954658 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.954926 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.955201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.955468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.955737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.956006 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80452) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 19:45:17.991775 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560ca89619c0 pid: 80453 (default app) May 19 19:45:17.995244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a2198-c4a1-4f1d-8116-990e5a18746b None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:17.995534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a2198-c4a1-4f1d-8116-990e5a18746b None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:17.995809 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a2198-c4a1-4f1d-8116-990e5a18746b None None] PATH_INFO: `/v3` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:17.996310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560ca89619c0 pid: 80452 (default app) May 19 19:45:18.000021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 1/1] 10.222.0.88 () {50 vars in 791 bytes} [Thu May 19 19:45:17 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:27.398272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbda5b-52a3-4eb4-8c95-f68d36958606 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:27.398910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbda5b-52a3-4eb4-8c95-f68d36958606 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:27.398910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbda5b-52a3-4eb4-8c95-f68d36958606 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:27.403193 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 1/2] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:45:27.408656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4727a65-5c16-45ea-bdc3-3e9c953289e7 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:27.409150 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4727a65-5c16-45ea-bdc3-3e9c953289e7 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:27.409251 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4727a65-5c16-45ea-bdc3-3e9c953289e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:27.478912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: INFO dbcounter [None req-c4727a65-5c16-45ea-bdc3-3e9c953289e7 None None] Registered counter for database keystone May 19 19:45:27.502163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG oslo_db.sqlalchemy.engines [None req-c4727a65-5c16-45ea-bdc3-3e9c953289e7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80453) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 19:45:27.509404 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writer thread running {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 19:45:27.703960 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG passlib.handlers.bcrypt [None req-c4727a65-5c16-45ea-bdc3-3e9c953289e7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80453) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 19:45:27.704369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG passlib.handlers.bcrypt [None req-c4727a65-5c16-45ea-bdc3-3e9c953289e7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80453) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 19:45:27.761041 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-c4727a65-5c16-45ea-bdc3-3e9c953289e7 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:27.762739 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-c4727a65-5c16-45ea-bdc3-3e9c953289e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:27.890968 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 2/3] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 19 19:45:28.050544 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbda5b-52a3-4eb4-8c95-f68d36958606 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:28.119745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: INFO dbcounter [None req-89fbda5b-52a3-4eb4-8c95-f68d36958606 None None] Registered counter for database keystone May 19 19:45:28.139767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG oslo_db.sqlalchemy.engines [None req-89fbda5b-52a3-4eb4-8c95-f68d36958606 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80452) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 19:45:28.146617 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writer thread running {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 19:45:28.265530 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbda5b-52a3-4eb4-8c95-f68d36958606 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:28.270364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261f11a-293a-453a-ad2a-80f1d6fd280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c59a6214d8424d94bb73ea602efdca3f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:28.271031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261f11a-293a-453a-ad2a-80f1d6fd280c admin admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:28.271312 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261f11a-293a-453a-ad2a-80f1d6fd280c admin admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:28.271581 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261f11a-293a-453a-ad2a-80f1d6fd280c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:28.279866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261f11a-293a-453a-ad2a-80f1d6fd280c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:28.331830 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a261f11a-293a-453a-ad2a-80f1d6fd280c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 19:45:28.332631 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261f11a-293a-453a-ad2a-80f1d6fd280c admin admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:28.343289 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 2/4] 10.222.0.88 () {58 vars in 1324 bytes} [Thu May 19 19:45:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 19:45:28.349925 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4727a65-5c16-45ea-bdc3-3e9c953289e7 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:28.364603 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4727a65-5c16-45ea-bdc3-3e9c953289e7 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:28.369020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5b998-9620-4620-99f3-e3e88244b75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c59a6214d8424d94bb73ea602efdca3f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:28.370146 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5b998-9620-4620-99f3-e3e88244b75c admin admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:28.370430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5b998-9620-4620-99f3-e3e88244b75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:28.370706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5b998-9620-4620-99f3-e3e88244b75c admin admin] PATH_INFO: `/v3/projects/c59a6214d8424d94bb73ea602efdca3f` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:28.372211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5b998-9620-4620-99f3-e3e88244b75c admin admin] RBAC: Authorizing `identity:get_project(project_id=c59a6214d8424d94bb73ea602efdca3f)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:28.532385 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: WARNING keystone.common.rbac_enforcer.enforcer [None req-81c5b998-9620-4620-99f3-e3e88244b75c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 19:45:28.533049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5b998-9620-4620-99f3-e3e88244b75c admin admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:28.537469 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 3/5] 10.222.0.88 () {56 vars in 1207 bytes} [Thu May 19 19:45:28 2022] GET /identity/v3/projects/c59a6214d8424d94bb73ea602efdca3f => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:30.779201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aad7cb-1e78-4487-9058-9a475800198f None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:30.779786 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aad7cb-1e78-4487-9058-9a475800198f None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:30.779786 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aad7cb-1e78-4487-9058-9a475800198f None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:30.784317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 3/6] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:45:30.791239 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b074-b7ce-4953-8f44-19533273590e None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:30.791749 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b074-b7ce-4953-8f44-19533273590e None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:30.791847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b074-b7ce-4953-8f44-19533273590e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:30.812372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-7a51b074-b7ce-4953-8f44-19533273590e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:30.813960 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-7a51b074-b7ce-4953-8f44-19533273590e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:30.822252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 4/7] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 19:45:30.845173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aad7cb-1e78-4487-9058-9a475800198f None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:30.858331 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aad7cb-1e78-4487-9058-9a475800198f None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:30.862705 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8257df7-636b-4dd9-88e1-f5b8262746fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c59a6214d8424d94bb73ea602efdca3f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:30.863968 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8257df7-636b-4dd9-88e1-f5b8262746fc admin admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:30.864245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8257df7-636b-4dd9-88e1-f5b8262746fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:30.864545 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8257df7-636b-4dd9-88e1-f5b8262746fc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:30.873763 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8257df7-636b-4dd9-88e1-f5b8262746fc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:30.875557 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8257df7-636b-4dd9-88e1-f5b8262746fc admin admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:30.885446 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 4/8] 10.222.0.88 () {58 vars in 1324 bytes} [Thu May 19 19:45:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 19:45:30.891358 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51b074-b7ce-4953-8f44-19533273590e None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:30.900905 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51b074-b7ce-4953-8f44-19533273590e None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:30.905347 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b64560-a7d6-489e-b592-c8e6d9fd5553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c59a6214d8424d94bb73ea602efdca3f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:30.906230 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b64560-a7d6-489e-b592-c8e6d9fd5553 admin admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:30.906511 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b64560-a7d6-489e-b592-c8e6d9fd5553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:30.906781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b64560-a7d6-489e-b592-c8e6d9fd5553 admin admin] PATH_INFO: `/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:30.908203 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b64560-a7d6-489e-b592-c8e6d9fd5553 admin admin] RBAC: Authorizing `identity:get_user(user_id=3f3a6a2da4254702bc7bd15361b6c9ae)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:30.910679 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b64560-a7d6-489e-b592-c8e6d9fd5553 admin admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:30.912460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 5/9] 10.222.0.88 () {56 vars in 1198 bytes} [Thu May 19 19:45:30 2022] GET /identity/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:33.203767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9cc4-aa65-40b0-8c2a-537240c97099 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:33.204357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9cc4-aa65-40b0-8c2a-537240c97099 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:33.204357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd9cc4-aa65-40b0-8c2a-537240c97099 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:33.205746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 5/10] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:33.212387 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab2ba9-b294-4c0b-997a-87e686138db3 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:33.212947 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab2ba9-b294-4c0b-997a-87e686138db3 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:33.213140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab2ba9-b294-4c0b-997a-87e686138db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:33.230962 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-7bab2ba9-b294-4c0b-997a-87e686138db3 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:33.232197 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-7bab2ba9-b294-4c0b-997a-87e686138db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:33.249055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092da9d0-8a9b-4990-95ed-e2d951f968f2 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:33.249346 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092da9d0-8a9b-4990-95ed-e2d951f968f2 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:33.249629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092da9d0-8a9b-4990-95ed-e2d951f968f2 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:33.251004 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 6/11] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:45:33.257501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ec8e8-e6ff-4140-9f5b-c973d22b3a2f None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:33.257795 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ec8e8-e6ff-4140-9f5b-c973d22b3a2f None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:33.258140 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ec8e8-e6ff-4140-9f5b-c973d22b3a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:33.301785 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG passlib.handlers.bcrypt [None req-702ec8e8-e6ff-4140-9f5b-c973d22b3a2f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80452) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 19:45:33.302190 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG passlib.handlers.bcrypt [None req-702ec8e8-e6ff-4140-9f5b-c973d22b3a2f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80452) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 19:45:33.326650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 6/12] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:33.349783 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-702ec8e8-e6ff-4140-9f5b-c973d22b3a2f None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:33.350166 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab2ba9-b294-4c0b-997a-87e686138db3 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:33.351413 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-702ec8e8-e6ff-4140-9f5b-c973d22b3a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:33.376479 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:33.460315 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 7/13] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:45:33.466782 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab2ba9-b294-4c0b-997a-87e686138db3 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:33.482508 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ec8e8-e6ff-4140-9f5b-c973d22b3a2f None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:33.512934 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e1e48-d636-4598-9e1c-38017cd2ce22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:33.513752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e1e48-d636-4598-9e1c-38017cd2ce22 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:33.514040 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e1e48-d636-4598-9e1c-38017cd2ce22 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:33.514318 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e1e48-d636-4598-9e1c-38017cd2ce22 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:33.528076 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e1e48-d636-4598-9e1c-38017cd2ce22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-be1e1e48-d636-4598-9e1c-38017cd2ce22 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:33.539725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:33.540863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:45:33.541761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 7/14] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:45:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:45:33.547424 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e1e48-d636-4598-9e1c-38017cd2ce22 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:33.597848 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ec8e8-e6ff-4140-9f5b-c973d22b3a2f None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:33.600403 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:33.637783 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566c7b4-f64a-48cf-a018-56ab476eaeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:33.638061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:33.638928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566c7b4-f64a-48cf-a018-56ab476eaeeb None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:33.639293 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566c7b4-f64a-48cf-a018-56ab476eaeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:33.639577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566c7b4-f64a-48cf-a018-56ab476eaeeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:33.658811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e1e48-d636-4598-9e1c-38017cd2ce22 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:33.703359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5a9ab-b6ff-4993-9d77-0282db99fe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:33.704121 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5a9ab-b6ff-4993-9d77-0282db99fe16 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:33.704498 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5a9ab-b6ff-4993-9d77-0282db99fe16 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:33.704852 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5a9ab-b6ff-4993-9d77-0282db99fe16 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:33.705799 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5a9ab-b6ff-4993-9d77-0282db99fe16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:33.706241 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5a9ab-b6ff-4993-9d77-0282db99fe16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:33.707811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae5a9ab-b6ff-4993-9d77-0282db99fe16 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:33.717180 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566c7b4-f64a-48cf-a018-56ab476eaeeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:33.718762 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566c7b4-f64a-48cf-a018-56ab476eaeeb None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:33.721140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 8/15] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:45:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:33.727607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5a9ab-b6ff-4993-9d77-0282db99fe16 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:33.833955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 8/16] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:45:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:45:33.834777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5a9ab-b6ff-4993-9d77-0282db99fe16 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:33.839657 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566c7b4-f64a-48cf-a018-56ab476eaeeb None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:33.842658 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:33.872800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56e365-6760-4dd8-905d-9ee194f9ec30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:33.873677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56e365-6760-4dd8-905d-9ee194f9ec30 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:33.873966 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56e365-6760-4dd8-905d-9ee194f9ec30 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:33.874244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56e365-6760-4dd8-905d-9ee194f9ec30 None admin] PATH_INFO: `/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:33.875491 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56e365-6760-4dd8-905d-9ee194f9ec30 None admin] RBAC: Authorizing `identity:get_user(user_id=3f3a6a2da4254702bc7bd15361b6c9ae)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:33.877072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56e365-6760-4dd8-905d-9ee194f9ec30 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:33.878795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 9/17] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:33 2022] GET /identity/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:33.884196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56e365-6760-4dd8-905d-9ee194f9ec30 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:33.899057 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:33.940779 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566c7b4-f64a-48cf-a018-56ab476eaeeb None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:33.979252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efe0ad-0a27-4b6b-b09c-9f7af24c2ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:33.979968 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe0ad-0a27-4b6b-b09c-9f7af24c2ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:33.980252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe0ad-0a27-4b6b-b09c-9f7af24c2ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:33.980559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6efe0ad-0a27-4b6b-b09c-9f7af24c2ed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:33.989841 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56e365-6760-4dd8-905d-9ee194f9ec30 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:33.999923 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efe0ad-0a27-4b6b-b09c-9f7af24c2ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:34.001428 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6efe0ad-0a27-4b6b-b09c-9f7af24c2ed5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-c6efe0ad-0a27-4b6b-b09c-9f7af24c2ed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:45:34.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:34.024173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:34.025123 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:45:34.025123 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:45:34.025123 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:34.025123 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:45:34.025123 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 9/18] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:45:34.026205 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865aa69-3871-4ec0-8e32-5c470f8ee462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:34.026999 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865aa69-3871-4ec0-8e32-5c470f8ee462 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:34.027280 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865aa69-3871-4ec0-8e32-5c470f8ee462 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:34.027550 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6865aa69-3871-4ec0-8e32-5c470f8ee462 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:34.028777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865aa69-3871-4ec0-8e32-5c470f8ee462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:34.030151 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efe0ad-0a27-4b6b-b09c-9f7af24c2ed5 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:34.030259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6865aa69-3871-4ec0-8e32-5c470f8ee462 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:34.031833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 10/19] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:45:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:34.037974 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865aa69-3871-4ec0-8e32-5c470f8ee462 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:34.089864 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:34.120945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:34.135531 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6efe0ad-0a27-4b6b-b09c-9f7af24c2ed5 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:34.140727 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6865aa69-3871-4ec0-8e32-5c470f8ee462 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:34.170975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe799df-b0aa-44fc-aa9c-a9f517ae6256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:34.171638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe799df-b0aa-44fc-aa9c-a9f517ae6256 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:34.171924 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe799df-b0aa-44fc-aa9c-a9f517ae6256 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:34.172201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe799df-b0aa-44fc-aa9c-a9f517ae6256 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:34.172982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe799df-b0aa-44fc-aa9c-a9f517ae6256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:34.173286 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe799df-b0aa-44fc-aa9c-a9f517ae6256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:34.174679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe799df-b0aa-44fc-aa9c-a9f517ae6256 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:34.177631 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042f786-8bd9-43ad-a166-95768230cbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:34.178301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042f786-8bd9-43ad-a166-95768230cbac None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:34.178606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042f786-8bd9-43ad-a166-95768230cbac None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:34.179149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042f786-8bd9-43ad-a166-95768230cbac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:34.179997 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042f786-8bd9-43ad-a166-95768230cbac None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.domain.id=default, user.id=3f3a6a2da4254702bc7bd15361b6c9ae) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:34.180306 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042f786-8bd9-43ad-a166-95768230cbac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:34.181810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042f786-8bd9-43ad-a166-95768230cbac None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:34.189073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 11/20] 10.222.0.88 () {56 vars in 1322 bytes} [Thu May 19 19:45:34 2022] GET /identity/v3/role_assignments?user.id=3f3a6a2da4254702bc7bd15361b6c9ae&scope.domain.id=default&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:34.194464 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 10/21] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:36.749683 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743306db-9eae-4180-bdc8-05852c1b7eb1 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:36.750175 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743306db-9eae-4180-bdc8-05852c1b7eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:36.750175 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743306db-9eae-4180-bdc8-05852c1b7eb1 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:36.751674 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 12/22] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:36.758410 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce003b-e017-4dde-b3b7-4465c3dd57db None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:36.759606 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce003b-e017-4dde-b3b7-4465c3dd57db None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:36.760098 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce003b-e017-4dde-b3b7-4465c3dd57db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:36.920194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-e6ce003b-e017-4dde-b3b7-4465c3dd57db None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:36.921873 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-e6ce003b-e017-4dde-b3b7-4465c3dd57db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:37.017256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 11/23] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:37.060920 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743306db-9eae-4180-bdc8-05852c1b7eb1 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:37.085111 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec636da3-e510-4a9a-899a-e08e4608110d None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:37.085714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec636da3-e510-4a9a-899a-e08e4608110d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:37.086201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec636da3-e510-4a9a-899a-e08e4608110d None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:37.088288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 12/24] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:37.098741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa466ca-26a4-49be-9103-eb9dae126e9b None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:37.099345 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa466ca-26a4-49be-9103-eb9dae126e9b None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:37.099816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa466ca-26a4-49be-9103-eb9dae126e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:37.135474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-5aa466ca-26a4-49be-9103-eb9dae126e9b None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:37.137044 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-5aa466ca-26a4-49be-9103-eb9dae126e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:37.242968 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:37.300926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743306db-9eae-4180-bdc8-05852c1b7eb1 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:37.321679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 13/25] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:45:37.377278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74752946-e5c1-4339-bce1-f5cbb3f02c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:37.379000 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752946-e5c1-4339-bce1-f5cbb3f02c3e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:37.379519 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752946-e5c1-4339-bce1-f5cbb3f02c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:37.380053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74752946-e5c1-4339-bce1-f5cbb3f02c3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:37.390169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa466ca-26a4-49be-9103-eb9dae126e9b None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:37.402525 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74752946-e5c1-4339-bce1-f5cbb3f02c3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-74752946-e5c1-4339-bce1-f5cbb3f02c3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:37.411963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:37.412904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:45:37.413659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:45:37.416089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 13/26] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:45:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:37.422333 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74752946-e5c1-4339-bce1-f5cbb3f02c3e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:37.432520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:37.608108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa466ca-26a4-49be-9103-eb9dae126e9b None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:37.658420 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f21dbb-2df4-4bd1-ace2-1def32540cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:37.660055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21dbb-2df4-4bd1-ace2-1def32540cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:37.660370 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21dbb-2df4-4bd1-ace2-1def32540cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:37.660648 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21dbb-2df4-4bd1-ace2-1def32540cf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:37.668372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74752946-e5c1-4339-bce1-f5cbb3f02c3e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:37.675753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f21dbb-2df4-4bd1-ace2-1def32540cf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:37.677889 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f21dbb-2df4-4bd1-ace2-1def32540cf4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-a5f21dbb-2df4-4bd1-ace2-1def32540cf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:45:37.699778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:37.700697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:37.701485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:45:37.701485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:45:37.701485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:45:37.701485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:45:37.701485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 14/27] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:45:37.703439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58e750-3244-4933-8884-f00d5d950862 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:37.703439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58e750-3244-4933-8884-f00d5d950862 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:37.703439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58e750-3244-4933-8884-f00d5d950862 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:37.709421 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 15/28] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:37.710617 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f81c2-04a5-4e14-ade5-1482e1977b5d None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:37.710966 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f81c2-04a5-4e14-ade5-1482e1977b5d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:37.711243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f81c2-04a5-4e14-ade5-1482e1977b5d None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:37.712598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 16/29] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:37.713329 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f81c2-04a5-4e14-ade5-1482e1977b5d None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:37.713506 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379ecb3-d6ad-4edc-822e-6f344df91b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:37.715039 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379ecb3-d6ad-4edc-822e-6f344df91b9e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:37.715331 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379ecb3-d6ad-4edc-822e-6f344df91b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:37.715614 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379ecb3-d6ad-4edc-822e-6f344df91b9e None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:37.716757 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379ecb3-d6ad-4edc-822e-6f344df91b9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:37.717028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379ecb3-d6ad-4edc-822e-6f344df91b9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:37.718471 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379ecb3-d6ad-4edc-822e-6f344df91b9e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:37.728857 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 14/30] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:45:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:37.730135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a910fe-4aeb-46f1-81e0-61dd7966adeb None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:37.730390 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a910fe-4aeb-46f1-81e0-61dd7966adeb None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:37.730666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a910fe-4aeb-46f1-81e0-61dd7966adeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:37.749570 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-96a910fe-4aeb-46f1-81e0-61dd7966adeb None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:37.750801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-96a910fe-4aeb-46f1-81e0-61dd7966adeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:37.751399 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:37.818485 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:37.826893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 15/31] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:37.828580 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a500e-87af-4e83-94da-b79d52a4e44c None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:37.829019 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a500e-87af-4e83-94da-b79d52a4e44c None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:37.829438 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a500e-87af-4e83-94da-b79d52a4e44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:37.856888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f81c2-04a5-4e14-ade5-1482e1977b5d None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:37.865439 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-7b5a500e-87af-4e83-94da-b79d52a4e44c None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:37.866924 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-7b5a500e-87af-4e83-94da-b79d52a4e44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:37.905116 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f9fa8-960d-426b-8b3d-d2048a3782be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:37.905964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f9fa8-960d-426b-8b3d-d2048a3782be None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:37.906534 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f9fa8-960d-426b-8b3d-d2048a3782be None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:37.906673 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83f9fa8-960d-426b-8b3d-d2048a3782be None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:37.907744 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f9fa8-960d-426b-8b3d-d2048a3782be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:37.908055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f9fa8-960d-426b-8b3d-d2048a3782be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:37.911417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83f9fa8-960d-426b-8b3d-d2048a3782be None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:37.928465 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 17/32] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:37.930173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f9fa8-960d-426b-8b3d-d2048a3782be None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:37.955131 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 16/33] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:37.956249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a500e-87af-4e83-94da-b79d52a4e44c None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:38.064746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:38.071899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a500e-87af-4e83-94da-b79d52a4e44c None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:38.076079 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83f9fa8-960d-426b-8b3d-d2048a3782be None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:38.101427 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:38.110728 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101f400-de6f-413b-819b-35d44a43d117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:38.111409 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101f400-de6f-413b-819b-35d44a43d117 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:38.111697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101f400-de6f-413b-819b-35d44a43d117 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:38.111974 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101f400-de6f-413b-819b-35d44a43d117 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:38.112987 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101f400-de6f-413b-819b-35d44a43d117 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:38.114429 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101f400-de6f-413b-819b-35d44a43d117 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:38.120115 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4cf17-43cf-4028-bdfb-0f1176409e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:38.120953 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4cf17-43cf-4028-bdfb-0f1176409e74 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:38.121239 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4cf17-43cf-4028-bdfb-0f1176409e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:38.121514 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4cf17-43cf-4028-bdfb-0f1176409e74 None admin] PATH_INFO: `/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:38.123182 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4cf17-43cf-4028-bdfb-0f1176409e74 None admin] RBAC: Authorizing `identity:get_user(user_id=3f3a6a2da4254702bc7bd15361b6c9ae)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:38.125236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4cf17-43cf-4028-bdfb-0f1176409e74 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:38.126711 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 18/34] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:37 2022] GET /identity/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:38.127131 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4cf17-43cf-4028-bdfb-0f1176409e74 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:38.275950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4cf17-43cf-4028-bdfb-0f1176409e74 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:38.317921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743197d-ae03-4d45-b95f-56e35ee86974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:38.318783 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743197d-ae03-4d45-b95f-56e35ee86974 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:38.319071 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743197d-ae03-4d45-b95f-56e35ee86974 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:38.319432 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743197d-ae03-4d45-b95f-56e35ee86974 None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:38.320169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743197d-ae03-4d45-b95f-56e35ee86974 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:38.321743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743197d-ae03-4d45-b95f-56e35ee86974 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:38.356622 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 17/35] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 19:45:37 2022] POST /identity/v3/roles => generated 225 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:45:38.356622 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101f400-de6f-413b-819b-35d44a43d117 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:38.506457 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101f400-de6f-413b-819b-35d44a43d117 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:38.561902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29e21c-7fa1-4026-9bc1-423325467adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:38.562589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29e21c-7fa1-4026-9bc1-423325467adc None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:38.562877 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29e21c-7fa1-4026-9bc1-423325467adc None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:38.563158 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29e21c-7fa1-4026-9bc1-423325467adc None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:38.563923 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29e21c-7fa1-4026-9bc1-423325467adc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:38.565506 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29e21c-7fa1-4026-9bc1-423325467adc None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:38.616626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 19/36] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:45:38 2022] POST /identity/v3/projects => generated 301 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:45:38.617664 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743197d-ae03-4d45-b95f-56e35ee86974 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:38.635126 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:38.654142 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 18/37] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 19:45:38 2022] POST /identity/v3/roles => generated 219 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:45:38.656956 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e867f-f981-48ea-b4c9-912b777ea3e5 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:38.657530 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e867f-f981-48ea-b4c9-912b777ea3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:38.658031 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e867f-f981-48ea-b4c9-912b777ea3e5 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:38.659907 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 19/38] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:38.669219 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbebf5-54c1-42ed-8f85-516b3b295b2b None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:38.669791 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbebf5-54c1-42ed-8f85-516b3b295b2b None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:38.670246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbebf5-54c1-42ed-8f85-516b3b295b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:38.710243 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-6ffbebf5-54c1-42ed-8f85-516b3b295b2b None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:38.713796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-6ffbebf5-54c1-42ed-8f85-516b3b295b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:38.801549 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743197d-ae03-4d45-b95f-56e35ee86974 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:38.812831 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:38.844832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 20/39] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:38.846688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c507b9-b427-4f23-8d5f-1c3f56aafbdb None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:38.847143 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c507b9-b427-4f23-8d5f-1c3f56aafbdb None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:38.847543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c507b9-b427-4f23-8d5f-1c3f56aafbdb None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:38.849139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 21/40] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:38.850533 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea78152-3cf3-4a08-825e-dbf031dcc34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:38.851724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea78152-3cf3-4a08-825e-dbf031dcc34a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:38.852141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea78152-3cf3-4a08-825e-dbf031dcc34a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:38.852657 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea78152-3cf3-4a08-825e-dbf031dcc34a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:38.854833 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea78152-3cf3-4a08-825e-dbf031dcc34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:38.856620 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea78152-3cf3-4a08-825e-dbf031dcc34a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:38.857844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1abad4-644a-428a-abcd-04d33c909dd2 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:38.858191 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1abad4-644a-428a-abcd-04d33c909dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:38.858944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1abad4-644a-428a-abcd-04d33c909dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:38.859390 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 20/41] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:45:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:38.866087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea78152-3cf3-4a08-825e-dbf031dcc34a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:38.886195 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-dc1abad4-644a-428a-abcd-04d33c909dd2 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:38.887670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-dc1abad4-644a-428a-abcd-04d33c909dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:38.981892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:38.983028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 22/42] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:38.983420 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1abad4-644a-428a-abcd-04d33c909dd2 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:39.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea78152-3cf3-4a08-825e-dbf031dcc34a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:39.050886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd81ee-e5b5-413b-a012-98476158d223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:39.051831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd81ee-e5b5-413b-a012-98476158d223 None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:39.052472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd81ee-e5b5-413b-a012-98476158d223 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:39.052472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd81ee-e5b5-413b-a012-98476158d223 None admin] PATH_INFO: `/v3/domains/default/users/3f3a6a2da4254702bc7bd15361b6c9ae/roles/815bd48c3df0405babfd5b53430fd3ea` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:39.054850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd81ee-e5b5-413b-a012-98476158d223 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3f3a6a2da4254702bc7bd15361b6c9ae, role_id=815bd48c3df0405babfd5b53430fd3ea)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:39.056305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd81ee-e5b5-413b-a012-98476158d223 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:39.098048 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1abad4-644a-428a-abcd-04d33c909dd2 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:39.105711 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:39.136355 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01acf9-bb68-4152-975d-a5adbce72d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:39.137045 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01acf9-bb68-4152-975d-a5adbce72d1b None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:39.137376 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01acf9-bb68-4152-975d-a5adbce72d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:39.137652 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01acf9-bb68-4152-975d-a5adbce72d1b None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:39.138606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01acf9-bb68-4152-975d-a5adbce72d1b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:39.139793 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01acf9-bb68-4152-975d-a5adbce72d1b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:39.168920 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 21/43] 10.222.0.88 () {58 vars in 1361 bytes} [Thu May 19 19:45:38 2022] PUT /identity/v3/domains/default/users/3f3a6a2da4254702bc7bd15361b6c9ae/roles/815bd48c3df0405babfd5b53430fd3ea => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:45:39.169870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd81ee-e5b5-413b-a012-98476158d223 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:39.257919 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 23/44] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 19:45:38 2022] POST /identity/v3/roles => generated 223 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:45:39.304673 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd81ee-e5b5-413b-a012-98476158d223 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:39.367884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e029a08-4aeb-41a2-a221-893f7ffc0b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:39.369625 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e029a08-4aeb-41a2-a221-893f7ffc0b51 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:39.369625 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e029a08-4aeb-41a2-a221-893f7ffc0b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:39.369625 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e029a08-4aeb-41a2-a221-893f7ffc0b51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:39.371158 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e029a08-4aeb-41a2-a221-893f7ffc0b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:39.372747 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e029a08-4aeb-41a2-a221-893f7ffc0b51 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:39.374849 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 22/45] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:45:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:39.380956 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01acf9-bb68-4152-975d-a5adbce72d1b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:39.519700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01acf9-bb68-4152-975d-a5adbce72d1b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:39.565707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5df4b5-a862-4b14-a692-7123be6e42b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:39.566843 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5df4b5-a862-4b14-a692-7123be6e42b8 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:39.567132 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5df4b5-a862-4b14-a692-7123be6e42b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:39.567407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5df4b5-a862-4b14-a692-7123be6e42b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:39.568134 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5df4b5-a862-4b14-a692-7123be6e42b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:39.569628 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5df4b5-a862-4b14-a692-7123be6e42b8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:39.662264 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 24/46] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:45:39 2022] POST /identity/v3/projects => generated 312 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:45:41.461526 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb7444-e15a-4a40-8e72-f3daa2c5425a None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:41.473641 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb7444-e15a-4a40-8e72-f3daa2c5425a None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:41.473641 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb7444-e15a-4a40-8e72-f3daa2c5425a None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:41.473641 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 23/47] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:41.473912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1445f-8668-4213-8cb8-274abde34ca1 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:41.473912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1445f-8668-4213-8cb8-274abde34ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:41.473912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1445f-8668-4213-8cb8-274abde34ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:41.487018 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-a4a1445f-8668-4213-8cb8-274abde34ca1 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:41.488230 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-a4a1445f-8668-4213-8cb8-274abde34ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:41.503636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:41.564689 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 25/48] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:41.588083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb7444-e15a-4a40-8e72-f3daa2c5425a None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:41.591963 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:41.696443 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb7444-e15a-4a40-8e72-f3daa2c5425a None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:41.731739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3741df-960e-4404-8189-973fa5e09e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:41.732604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3741df-960e-4404-8189-973fa5e09e19 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:41.732901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3741df-960e-4404-8189-973fa5e09e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:41.733191 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3741df-960e-4404-8189-973fa5e09e19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:41.747891 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3741df-960e-4404-8189-973fa5e09e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-6a3741df-960e-4404-8189-973fa5e09e19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:41.757689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:41.758643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:45:41.759563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 24/49] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:45:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:41.764792 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1445f-8668-4213-8cb8-274abde34ca1 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:41.868026 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1445f-8668-4213-8cb8-274abde34ca1 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:41.903799 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a9b95-cf71-4154-a532-461191e65e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:41.904530 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a9b95-cf71-4154-a532-461191e65e4b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:41.904821 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a9b95-cf71-4154-a532-461191e65e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:41.905103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a9b95-cf71-4154-a532-461191e65e4b None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:41.905867 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a9b95-cf71-4154-a532-461191e65e4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:41.906175 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a9b95-cf71-4154-a532-461191e65e4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:41.907730 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a9b95-cf71-4154-a532-461191e65e4b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:41.908645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a14e3b-462b-4043-920b-70c76ffdc58a None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:41.908940 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a14e3b-462b-4043-920b-70c76ffdc58a None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:41.909216 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a14e3b-462b-4043-920b-70c76ffdc58a None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:41.911131 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 25/50] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:41.916126 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c9e31-375a-4d2d-845c-92583c1e66dd None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:41.916441 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c9e31-375a-4d2d-845c-92583c1e66dd None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:41.916722 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c9e31-375a-4d2d-845c-92583c1e66dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:41.917584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 26/51] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:45:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:41.923299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a9b95-cf71-4154-a532-461191e65e4b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:41.935523 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-258c9e31-375a-4d2d-845c-92583c1e66dd None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:41.936506 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:41.937090 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-258c9e31-375a-4d2d-845c-92583c1e66dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:41.998655 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:42.006885 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 26/52] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:45:42.027348 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a9b95-cf71-4154-a532-461191e65e4b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:42.031125 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c9e31-375a-4d2d-845c-92583c1e66dd None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:42.063114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0b523-ad71-4713-92fc-e1ec8ca633b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:42.063990 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0b523-ad71-4713-92fc-e1ec8ca633b9 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:42.064271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0b523-ad71-4713-92fc-e1ec8ca633b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:42.064573 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0b523-ad71-4713-92fc-e1ec8ca633b9 None admin] PATH_INFO: `/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:42.065794 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0b523-ad71-4713-92fc-e1ec8ca633b9 None admin] RBAC: Authorizing `identity:get_user(user_id=3f3a6a2da4254702bc7bd15361b6c9ae)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:42.067265 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0b523-ad71-4713-92fc-e1ec8ca633b9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:42.068927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 27/53] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:41 2022] GET /identity/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:42.074306 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0b523-ad71-4713-92fc-e1ec8ca633b9 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:42.134912 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c9e31-375a-4d2d-845c-92583c1e66dd None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:42.157025 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:42.172795 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764f53-56cc-4d23-8128-c5412032e621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:42.173837 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764f53-56cc-4d23-8128-c5412032e621 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:42.174126 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764f53-56cc-4d23-8128-c5412032e621 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:42.174405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3764f53-56cc-4d23-8128-c5412032e621 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:42.176258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3764f53-56cc-4d23-8128-c5412032e621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:42.177767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3764f53-56cc-4d23-8128-c5412032e621 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:42.179733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 27/54] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:45:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:42.181375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0b523-ad71-4713-92fc-e1ec8ca633b9 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:42.185415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764f53-56cc-4d23-8128-c5412032e621 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:42.223390 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355f99f-b489-44f9-a1da-4a1c293f8966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:42.224184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355f99f-b489-44f9-a1da-4a1c293f8966 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:42.224508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355f99f-b489-44f9-a1da-4a1c293f8966 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:42.224788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355f99f-b489-44f9-a1da-4a1c293f8966 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:42.226005 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355f99f-b489-44f9-a1da-4a1c293f8966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:42.227500 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355f99f-b489-44f9-a1da-4a1c293f8966 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:42.229176 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 28/55] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:45:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:42.234846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355f99f-b489-44f9-a1da-4a1c293f8966 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:42.248211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:42.288717 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3764f53-56cc-4d23-8128-c5412032e621 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:42.323710 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2548d4-ffe6-4272-8529-9b22d0dcf09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:42.324435 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2548d4-ffe6-4272-8529-9b22d0dcf09f None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:42.324722 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2548d4-ffe6-4272-8529-9b22d0dcf09f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:42.325004 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2548d4-ffe6-4272-8529-9b22d0dcf09f None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:42.325727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2548d4-ffe6-4272-8529-9b22d0dcf09f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:42.327196 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2548d4-ffe6-4272-8529-9b22d0dcf09f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:42.341748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355f99f-b489-44f9-a1da-4a1c293f8966 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:42.376079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b3a7b-7a75-4103-b613-aaa905eaf4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:42.376762 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b3a7b-7a75-4103-b613-aaa905eaf4bc None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:42.377047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b3a7b-7a75-4103-b613-aaa905eaf4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:42.377327 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b3a7b-7a75-4103-b613-aaa905eaf4bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:42.378164 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b3a7b-7a75-4103-b613-aaa905eaf4bc None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.domain.id=default, user.id=3f3a6a2da4254702bc7bd15361b6c9ae) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:42.378506 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b3a7b-7a75-4103-b613-aaa905eaf4bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:42.379973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b3a7b-7a75-4103-b613-aaa905eaf4bc None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:42.385764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 29/56] 10.222.0.88 () {56 vars in 1322 bytes} [Thu May 19 19:45:42 2022] GET /identity/v3/role_assignments?user.id=3f3a6a2da4254702bc7bd15361b6c9ae&scope.domain.id=default&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:42.420941 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 28/57] 10.222.0.88 () {60 vars in 1140 bytes} [Thu May 19 19:45:42 2022] POST /identity/v3/projects => generated 298 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:45:44.671530 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82462ec7-4f88-4635-979b-43278babad47 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:44.672037 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82462ec7-4f88-4635-979b-43278babad47 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:44.672037 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82462ec7-4f88-4635-979b-43278babad47 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:44.673437 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 30/58] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:44.678842 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74170d-86f1-452a-97bd-e22cd5882d44 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:44.679226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74170d-86f1-452a-97bd-e22cd5882d44 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:44.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74170d-86f1-452a-97bd-e22cd5882d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:44.695979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-8e74170d-86f1-452a-97bd-e22cd5882d44 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:44.697215 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-8e74170d-86f1-452a-97bd-e22cd5882d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:44.763202 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 29/59] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:44.788835 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82462ec7-4f88-4635-979b-43278babad47 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:44.802163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:44.895382 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82462ec7-4f88-4635-979b-43278babad47 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:44.928376 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa772aa0-8417-44bc-b793-590b20efd9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:44.929427 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa772aa0-8417-44bc-b793-590b20efd9da None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:44.929707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa772aa0-8417-44bc-b793-590b20efd9da None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:44.929984 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa772aa0-8417-44bc-b793-590b20efd9da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:44.931150 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa772aa0-8417-44bc-b793-590b20efd9da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:44.932704 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa772aa0-8417-44bc-b793-590b20efd9da None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:44.934365 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 31/60] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:45:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:44.939889 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74170d-86f1-452a-97bd-e22cd5882d44 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:44.983988 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:45.036943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74170d-86f1-452a-97bd-e22cd5882d44 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:45.069643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331d5a7-9f16-4728-b35c-50d40e47cc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:45.070334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331d5a7-9f16-4728-b35c-50d40e47cc6f None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:45.070614 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331d5a7-9f16-4728-b35c-50d40e47cc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:45.070887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331d5a7-9f16-4728-b35c-50d40e47cc6f None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:45.071591 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331d5a7-9f16-4728-b35c-50d40e47cc6f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:45.073044 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331d5a7-9f16-4728-b35c-50d40e47cc6f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:45.209656 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 30/61] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:45:44 2022] POST /identity/v3/users => generated 275 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:45:48.718582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e957d-79dd-4f11-a182-ea3b67fca1b1 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:48.719645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e957d-79dd-4f11-a182-ea3b67fca1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:48.720224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e957d-79dd-4f11-a182-ea3b67fca1b1 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:48.722050 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 32/62] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:48.728688 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af799b-a20c-4545-833c-68559e6debf7 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:48.729677 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af799b-a20c-4545-833c-68559e6debf7 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:48.730156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af799b-a20c-4545-833c-68559e6debf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:48.748532 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-97af799b-a20c-4545-833c-68559e6debf7 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:48.750078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-97af799b-a20c-4545-833c-68559e6debf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:48.846975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 31/63] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:45:48.882549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e957d-79dd-4f11-a182-ea3b67fca1b1 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:48.995937 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb8e6-c133-4678-9fd7-0ad7928144fd None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:48.996584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb8e6-c133-4678-9fd7-0ad7928144fd None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:48.997078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4bb8e6-c133-4678-9fd7-0ad7928144fd None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:48.998657 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 32/64] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:45:49.004617 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24230af2-5ab2-478a-ae93-faa1da438a21 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.005186 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24230af2-5ab2-478a-ae93-faa1da438a21 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.005659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24230af2-5ab2-478a-ae93-faa1da438a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.012788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:49.029484 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-24230af2-5ab2-478a-ae93-faa1da438a21 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:49.030779 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-24230af2-5ab2-478a-ae93-faa1da438a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:49.045746 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e957d-79dd-4f11-a182-ea3b67fca1b1 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:49.080764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:49.098446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5fe91-acf0-4680-a098-f143b4b3ddbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:49.099524 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5fe91-acf0-4680-a098-f143b4b3ddbf None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.099934 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5fe91-acf0-4680-a098-f143b4b3ddbf None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.100368 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5fe91-acf0-4680-a098-f143b4b3ddbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.113094 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5fe91-acf0-4680-a098-f143b4b3ddbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-7dc5fe91-acf0-4680-a098-f143b4b3ddbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:49.121735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:49.122847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:45:49.123817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:45:49.127089 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 33/66] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:45:49.127343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 33/66] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:45:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:49.136407 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e5f54-b0c3-4c1f-a4ae-6e9c04e504a2 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.136407 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e5f54-b0c3-4c1f-a4ae-6e9c04e504a2 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.136407 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e5f54-b0c3-4c1f-a4ae-6e9c04e504a2 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.136407 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 34/67] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:49.138061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5fe91-acf0-4680-a098-f143b4b3ddbf None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:49.139761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e5f5f-1029-49d2-937e-7c4e04a12ff1 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.140816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e5f5f-1029-49d2-937e-7c4e04a12ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.141689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e5f5f-1029-49d2-937e-7c4e04a12ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.170252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-356e5f5f-1029-49d2-937e-7c4e04a12ff1 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:49.171568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-356e5f5f-1029-49d2-937e-7c4e04a12ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:49.268085 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 35/68] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:49.273354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e5f5f-1029-49d2-937e-7c4e04a12ff1 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:49.297048 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5fe91-acf0-4680-a098-f143b4b3ddbf None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:49.349665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aceefd3-ac85-4a0d-9ba3-868db20cc4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:49.350341 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceefd3-ac85-4a0d-9ba3-868db20cc4fd None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.350619 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceefd3-ac85-4a0d-9ba3-868db20cc4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.350891 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aceefd3-ac85-4a0d-9ba3-868db20cc4fd None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.351646 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aceefd3-ac85-4a0d-9ba3-868db20cc4fd None admin] RBAC: Adding query filter params (name=member) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:49.351948 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aceefd3-ac85-4a0d-9ba3-868db20cc4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:49.353356 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aceefd3-ac85-4a0d-9ba3-868db20cc4fd None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:49.358650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:49.363194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 34/69] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:45:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:49.364028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aceefd3-ac85-4a0d-9ba3-868db20cc4fd None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:49.389407 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:49.398371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e5f5f-1029-49d2-937e-7c4e04a12ff1 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:49.433379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc0ed9d-7521-4c51-92ac-7c488c3a0c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:49.434190 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0ed9d-7521-4c51-92ac-7c488c3a0c3b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.434484 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0ed9d-7521-4c51-92ac-7c488c3a0c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.434770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc0ed9d-7521-4c51-92ac-7c488c3a0c3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.445829 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc0ed9d-7521-4c51-92ac-7c488c3a0c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-efc0ed9d-7521-4c51-92ac-7c488c3a0c3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:49.453660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:49.454661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:45:49.455405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:45:49.456658 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 36/70] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:45:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:49.458115 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1f51b-22ad-4408-ad35-dae98a807613 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.458409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1f51b-22ad-4408-ad35-dae98a807613 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.458682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1f51b-22ad-4408-ad35-dae98a807613 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.459979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 37/71] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:49.465203 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a366666-cf77-4ccb-908c-87afb6d7c2de None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.465338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aceefd3-ac85-4a0d-9ba3-868db20cc4fd None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:49.465450 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a366666-cf77-4ccb-908c-87afb6d7c2de None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.465687 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a366666-cf77-4ccb-908c-87afb6d7c2de None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.466968 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 38/72] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:49.467678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a366666-cf77-4ccb-908c-87afb6d7c2de None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:49.504670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016aa171-004e-4210-a03a-a2c67893c9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:49.505513 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016aa171-004e-4210-a03a-a2c67893c9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.505767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016aa171-004e-4210-a03a-a2c67893c9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.506043 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016aa171-004e-4210-a03a-a2c67893c9d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.516718 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016aa171-004e-4210-a03a-a2c67893c9d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-016aa171-004e-4210-a03a-a2c67893c9d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:49.525120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:49.525982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:45:49.526733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 35/73] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:45:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:49.527514 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016aa171-004e-4210-a03a-a2c67893c9d7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:49.573334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a366666-cf77-4ccb-908c-87afb6d7c2de None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:49.589482 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:49.609258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30978d1b-ed84-4a2d-b189-156481aa56ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:49.610133 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30978d1b-ed84-4a2d-b189-156481aa56ff None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.610424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30978d1b-ed84-4a2d-b189-156481aa56ff None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.610703 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30978d1b-ed84-4a2d-b189-156481aa56ff None admin] PATH_INFO: `/v3/users/c0ff018c6850486aa01127bb8c1bd8b2` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.626524 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30978d1b-ed84-4a2d-b189-156481aa56ff None admin] RBAC: Authorizing `identity:get_user(user_id=c0ff018c6850486aa01127bb8c1bd8b2)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:49.628019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30978d1b-ed84-4a2d-b189-156481aa56ff None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:49.629951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 39/74] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:49 2022] GET /identity/v3/users/c0ff018c6850486aa01127bb8c1bd8b2 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:49.631106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fda75-dc89-4ea6-99f2-8ea5856f0249 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.631401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fda75-dc89-4ea6-99f2-8ea5856f0249 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.631683 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fda75-dc89-4ea6-99f2-8ea5856f0249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.632293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016aa171-004e-4210-a03a-a2c67893c9d7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:49.647760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-834fda75-dc89-4ea6-99f2-8ea5856f0249 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:49.648999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-834fda75-dc89-4ea6-99f2-8ea5856f0249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:49.651092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:49.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21885c-bdd0-40ef-8ea6-ada63b4098bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:49.678779 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21885c-bdd0-40ef-8ea6-ada63b4098bc None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.679062 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21885c-bdd0-40ef-8ea6-ada63b4098bc None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.679337 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21885c-bdd0-40ef-8ea6-ada63b4098bc None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.680097 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21885c-bdd0-40ef-8ea6-ada63b4098bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:49.680434 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21885c-bdd0-40ef-8ea6-ada63b4098bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:49.682072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21885c-bdd0-40ef-8ea6-ada63b4098bc None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:49.690909 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 36/75] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:45:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:49.693753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618f280-4b47-4a63-86f2-616e98e0c00e None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.694406 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618f280-4b47-4a63-86f2-616e98e0c00e None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.694692 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618f280-4b47-4a63-86f2-616e98e0c00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.711320 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-c618f280-4b47-4a63-86f2-616e98e0c00e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:49.712567 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-c618f280-4b47-4a63-86f2-616e98e0c00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:49.717888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 40/76] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:49.718740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fda75-dc89-4ea6-99f2-8ea5856f0249 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:49.788572 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 37/77] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:49.792982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618f280-4b47-4a63-86f2-616e98e0c00e None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:49.818681 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fda75-dc89-4ea6-99f2-8ea5856f0249 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:49.852371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:49.853982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa423f1-c8f1-42fd-88c1-7c446bedee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:49.854657 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa423f1-c8f1-42fd-88c1-7c446bedee21 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.854940 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa423f1-c8f1-42fd-88c1-7c446bedee21 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.855222 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa423f1-c8f1-42fd-88c1-7c446bedee21 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.855974 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa423f1-c8f1-42fd-88c1-7c446bedee21 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:49.856283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa423f1-c8f1-42fd-88c1-7c446bedee21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:49.857795 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa423f1-c8f1-42fd-88c1-7c446bedee21 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:49.866839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:49.870838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 41/78] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:45:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:49.871436 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa423f1-c8f1-42fd-88c1-7c446bedee21 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:49.902922 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618f280-4b47-4a63-86f2-616e98e0c00e None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:49.939172 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca9019-9cab-4b80-b607-179bc8973671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:49.939826 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9019-9cab-4b80-b607-179bc8973671 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:49.940140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9019-9cab-4b80-b607-179bc8973671 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:49.940453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ca9019-9cab-4b80-b607-179bc8973671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:49.972969 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa423f1-c8f1-42fd-88c1-7c446bedee21 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:50.008051 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b7607-5731-421f-9c66-026d8bf75f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:50.008397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca9019-9cab-4b80-b607-179bc8973671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:50.008957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b7607-5731-421f-9c66-026d8bf75f6a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:50.009238 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b7607-5731-421f-9c66-026d8bf75f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:50.009509 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b7607-5731-421f-9c66-026d8bf75f6a None admin] PATH_INFO: `/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:50.009888 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ca9019-9cab-4b80-b607-179bc8973671 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:50.010788 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b7607-5731-421f-9c66-026d8bf75f6a None admin] RBAC: Authorizing `identity:get_user(user_id=3f3a6a2da4254702bc7bd15361b6c9ae)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:50.012263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b7607-5731-421f-9c66-026d8bf75f6a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:50.014062 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 42/79] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:49 2022] GET /identity/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:50.019483 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b7607-5731-421f-9c66-026d8bf75f6a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:50.069576 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:50.096301 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:50.120447 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 38/80] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:45:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:45:50.121376 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca9019-9cab-4b80-b607-179bc8973671 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:50.132969 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b7607-5731-421f-9c66-026d8bf75f6a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:50.168432 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e4fef-6c3b-496b-a612-a417fca00137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:50.169223 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e4fef-6c3b-496b-a612-a417fca00137 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:50.169504 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e4fef-6c3b-496b-a612-a417fca00137 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:50.169775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e4fef-6c3b-496b-a612-a417fca00137 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:50.170965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e4fef-6c3b-496b-a612-a417fca00137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:50.172461 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e4fef-6c3b-496b-a612-a417fca00137 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:50.174061 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 43/81] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:45:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:50.177190 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e4fef-6c3b-496b-a612-a417fca00137 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:50.232850 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ca9019-9cab-4b80-b607-179bc8973671 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:50.272053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e615b-8a46-47cf-80fa-a7be956f5d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:50.272904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e615b-8a46-47cf-80fa-a7be956f5d06 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:50.273194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e615b-8a46-47cf-80fa-a7be956f5d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:50.273474 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e615b-8a46-47cf-80fa-a7be956f5d06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:50.285750 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e615b-8a46-47cf-80fa-a7be956f5d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:50.288781 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e4fef-6c3b-496b-a612-a417fca00137 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-1f6e615b-8a46-47cf-80fa-a7be956f5d06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:50.293817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:50.295010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 39/82] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:45:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:50.296433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e615b-8a46-47cf-80fa-a7be956f5d06 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:50.312728 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:50.336369 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4612b0c-315e-40ec-aedd-a90aeeaba9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:50.336369 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4612b0c-315e-40ec-aedd-a90aeeaba9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:50.336369 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4612b0c-315e-40ec-aedd-a90aeeaba9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:50.336369 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4612b0c-315e-40ec-aedd-a90aeeaba9f1 None admin] PATH_INFO: `/v3/users/c0ff018c6850486aa01127bb8c1bd8b2` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:50.336807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4612b0c-315e-40ec-aedd-a90aeeaba9f1 None admin] RBAC: Authorizing `identity:get_user(user_id=c0ff018c6850486aa01127bb8c1bd8b2)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:50.337738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4612b0c-315e-40ec-aedd-a90aeeaba9f1 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:50.341539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 44/83] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:50 2022] GET /identity/v3/users/c0ff018c6850486aa01127bb8c1bd8b2 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:50.342845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4612b0c-315e-40ec-aedd-a90aeeaba9f1 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:50.347206 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:50.414259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e615b-8a46-47cf-80fa-a7be956f5d06 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:50.450771 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0f94c-43ba-4d07-8b24-6fde5f828fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:50.451229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4612b0c-315e-40ec-aedd-a90aeeaba9f1 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:50.451430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0f94c-43ba-4d07-8b24-6fde5f828fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:50.451713 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0f94c-43ba-4d07-8b24-6fde5f828fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:50.451991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0f94c-43ba-4d07-8b24-6fde5f828fd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:50.489350 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd0d94-8ad7-4334-abd3-aebbd52c5aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:50.490224 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd0d94-8ad7-4334-abd3-aebbd52c5aca None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:50.490517 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd0d94-8ad7-4334-abd3-aebbd52c5aca None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:50.490800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd0d94-8ad7-4334-abd3-aebbd52c5aca None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:50.492047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd0d94-8ad7-4334-abd3-aebbd52c5aca None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:50.493564 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd0d94-8ad7-4334-abd3-aebbd52c5aca None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:50.495567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 45/84] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:45:50 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:50.496119 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd0d94-8ad7-4334-abd3-aebbd52c5aca None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:50.531925 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0f94c-43ba-4d07-8b24-6fde5f828fd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:50.533513 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0f94c-43ba-4d07-8b24-6fde5f828fd2 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:50.549629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:50.577406 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:50.611885 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd0d94-8ad7-4334-abd3-aebbd52c5aca None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:50.643981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 40/85] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:45:50.644926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0f94c-43ba-4d07-8b24-6fde5f828fd2 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:50.649283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0b41f-50b3-4e6d-910f-0336fd55fc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:50.650041 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0b41f-50b3-4e6d-910f-0336fd55fc5a None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:50.650329 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0b41f-50b3-4e6d-910f-0336fd55fc5a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:50.650611 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0b41f-50b3-4e6d-910f-0336fd55fc5a None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:50.651338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0b41f-50b3-4e6d-910f-0336fd55fc5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:50.652869 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0b41f-50b3-4e6d-910f-0336fd55fc5a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:50.752647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0f94c-43ba-4d07-8b24-6fde5f828fd2 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:50.765897 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:50.774441 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 46/86] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:45:50 2022] POST /identity/v3/users => generated 289 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:45:50.775288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0b41f-50b3-4e6d-910f-0336fd55fc5a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:50.800760 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf67033-f52b-45eb-95b4-5040cc3fb7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:50.802274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf67033-f52b-45eb-95b4-5040cc3fb7fd None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:50.802274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf67033-f52b-45eb-95b4-5040cc3fb7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:50.802274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf67033-f52b-45eb-95b4-5040cc3fb7fd None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:50.802967 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf67033-f52b-45eb-95b4-5040cc3fb7fd None admin] RBAC: Adding query filter params (name=member) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:50.805024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf67033-f52b-45eb-95b4-5040cc3fb7fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:50.808250 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf67033-f52b-45eb-95b4-5040cc3fb7fd None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:50.821704 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 41/87] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:45:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:50.821704 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf67033-f52b-45eb-95b4-5040cc3fb7fd None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:50.919481 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0b41f-50b3-4e6d-910f-0336fd55fc5a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:50.949010 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:50.957625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf67033-f52b-45eb-95b4-5040cc3fb7fd None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:50.961049 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe7e69-e125-45ad-93bd-4f225edfbc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:50.982318 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe7e69-e125-45ad-93bd-4f225edfbc3e None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:50.982318 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe7e69-e125-45ad-93bd-4f225edfbc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:50.982318 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe7e69-e125-45ad-93bd-4f225edfbc3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:51.000712 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01406e1b-0329-45d4-9e38-626e0e04ac5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:51.001376 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01406e1b-0329-45d4-9e38-626e0e04ac5d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:51.001659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01406e1b-0329-45d4-9e38-626e0e04ac5d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:51.001935 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01406e1b-0329-45d4-9e38-626e0e04ac5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:51.003309 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01406e1b-0329-45d4-9e38-626e0e04ac5d None admin] RBAC: Adding query filter params (role.id=6a41401a07c443f0b5acd2e189145727, scope.project.id=f14d8cdc0ed5452ca04a981594be5974, user.id=c0ff018c6850486aa01127bb8c1bd8b2) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:51.003713 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01406e1b-0329-45d4-9e38-626e0e04ac5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:51.005279 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01406e1b-0329-45d4-9e38-626e0e04ac5d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:51.012812 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 42/88] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:45:50 2022] GET /identity/v3/role_assignments?user.id=c0ff018c6850486aa01127bb8c1bd8b2&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=6a41401a07c443f0b5acd2e189145727 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:51.013668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01406e1b-0329-45d4-9e38-626e0e04ac5d None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:51.045515 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe7e69-e125-45ad-93bd-4f225edfbc3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:51.047066 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe7e69-e125-45ad-93bd-4f225edfbc3e None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:51.061386 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:51.161351 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01406e1b-0329-45d4-9e38-626e0e04ac5d None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:51.191228 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 47/89] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:45:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:45:51.192434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe7e69-e125-45ad-93bd-4f225edfbc3e None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:51.211551 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96350fa5-c133-4e08-8298-11b2632acca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:51.212492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96350fa5-c133-4e08-8298-11b2632acca9 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:51.213237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96350fa5-c133-4e08-8298-11b2632acca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:51.213531 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96350fa5-c133-4e08-8298-11b2632acca9 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:51.215180 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96350fa5-c133-4e08-8298-11b2632acca9 None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:51.217274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96350fa5-c133-4e08-8298-11b2632acca9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:51.219105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 43/90] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:45:51 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:51.220734 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96350fa5-c133-4e08-8298-11b2632acca9 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:51.232665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:51.313291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe7e69-e125-45ad-93bd-4f225edfbc3e None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:51.358795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96350fa5-c133-4e08-8298-11b2632acca9 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:51.359649 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b06af-6932-487e-8054-e4add2adfd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:51.360937 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b06af-6932-487e-8054-e4add2adfd0a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:51.361252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b06af-6932-487e-8054-e4add2adfd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:51.361619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b06af-6932-487e-8054-e4add2adfd0a None admin] PATH_INFO: `/v3/users/c0ff018c6850486aa01127bb8c1bd8b2` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:51.363417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b06af-6932-487e-8054-e4add2adfd0a None admin] RBAC: Authorizing `identity:get_user(user_id=c0ff018c6850486aa01127bb8c1bd8b2)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:51.364908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b06af-6932-487e-8054-e4add2adfd0a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:51.366586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 48/91] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:51 2022] GET /identity/v3/users/c0ff018c6850486aa01127bb8c1bd8b2 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:51.367268 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:51.368654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b06af-6932-487e-8054-e4add2adfd0a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:51.404287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d2c1e-248f-4764-b49d-c30fd771282b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:51.405192 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d2c1e-248f-4764-b49d-c30fd771282b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:51.405765 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d2c1e-248f-4764-b49d-c30fd771282b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:51.406274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d2c1e-248f-4764-b49d-c30fd771282b None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:51.408679 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d2c1e-248f-4764-b49d-c30fd771282b None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:51.410585 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d2c1e-248f-4764-b49d-c30fd771282b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:51.412542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 44/92] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:45:51 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:51.414620 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d2c1e-248f-4764-b49d-c30fd771282b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:51.589853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d2c1e-248f-4764-b49d-c30fd771282b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:51.607764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:51.616992 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b06af-6932-487e-8054-e4add2adfd0a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:51.661056 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda3bcc-bfee-40c2-8bf2-2a4c3c79a5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:51.662053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda3bcc-bfee-40c2-8bf2-2a4c3c79a5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:51.662550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda3bcc-bfee-40c2-8bf2-2a4c3c79a5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:51.663025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda3bcc-bfee-40c2-8bf2-2a4c3c79a5f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:51.664066 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda3bcc-bfee-40c2-8bf2-2a4c3c79a5f2 None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=f14d8cdc0ed5452ca04a981594be5974, user.id=3f3a6a2da4254702bc7bd15361b6c9ae) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:51.664614 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda3bcc-bfee-40c2-8bf2-2a4c3c79a5f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:51.665257 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a5794-6dc0-4f16-883c-a3abdb472538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:51.665925 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a5794-6dc0-4f16-883c-a3abdb472538 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:51.666212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a5794-6dc0-4f16-883c-a3abdb472538 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:51.666486 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a5794-6dc0-4f16-883c-a3abdb472538 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:51.667204 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda3bcc-bfee-40c2-8bf2-2a4c3c79a5f2 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:51.676302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 49/93] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:45:51 2022] GET /identity/v3/role_assignments?user.id=3f3a6a2da4254702bc7bd15361b6c9ae&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:51.677447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda3bcc-bfee-40c2-8bf2-2a4c3c79a5f2 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:51.768916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:51.815430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a5794-6dc0-4f16-883c-a3abdb472538 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:51.817232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a5794-6dc0-4f16-883c-a3abdb472538 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:51.878595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda3bcc-bfee-40c2-8bf2-2a4c3c79a5f2 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:51.933107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 45/94] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:45:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:45:51.938648 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a5794-6dc0-4f16-883c-a3abdb472538 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:51.950806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708f9d6-303e-4cca-b7fd-f6510a6ab44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:51.952222 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708f9d6-303e-4cca-b7fd-f6510a6ab44a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:51.952856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708f9d6-303e-4cca-b7fd-f6510a6ab44a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:51.953327 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708f9d6-303e-4cca-b7fd-f6510a6ab44a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:51.954483 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708f9d6-303e-4cca-b7fd-f6510a6ab44a None admin] RBAC: Adding query filter params (role.id=cb116a5614cf4a48ad99840d95f96bb3, scope.project.id=f14d8cdc0ed5452ca04a981594be5974, user.id=c0ff018c6850486aa01127bb8c1bd8b2) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:51.955005 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708f9d6-303e-4cca-b7fd-f6510a6ab44a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:51.956847 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708f9d6-303e-4cca-b7fd-f6510a6ab44a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:51.968800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 50/95] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:45:51 2022] GET /identity/v3/role_assignments?user.id=c0ff018c6850486aa01127bb8c1bd8b2&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=cb116a5614cf4a48ad99840d95f96bb3 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:52.025744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:52.049380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a5794-6dc0-4f16-883c-a3abdb472538 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:52.088285 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871df1a6-fc2e-4f57-993c-4e98b65d00ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:52.089218 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871df1a6-fc2e-4f57-993c-4e98b65d00ca None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:52.089507 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871df1a6-fc2e-4f57-993c-4e98b65d00ca None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:52.089786 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871df1a6-fc2e-4f57-993c-4e98b65d00ca None admin] PATH_INFO: `/v3/projects/5beb6dc9dc3f45d5919111c3e6c396e7` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:52.091191 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871df1a6-fc2e-4f57-993c-4e98b65d00ca None admin] RBAC: Authorizing `identity:get_project(project_id=5beb6dc9dc3f45d5919111c3e6c396e7)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:52.092831 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871df1a6-fc2e-4f57-993c-4e98b65d00ca None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:52.094634 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 46/96] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:45:51 2022] GET /identity/v3/projects/5beb6dc9dc3f45d5919111c3e6c396e7 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:52.101921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708f9d6-303e-4cca-b7fd-f6510a6ab44a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:52.157310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:52.260308 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708f9d6-303e-4cca-b7fd-f6510a6ab44a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:52.312748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c7d8f-3614-40ab-916a-30092e43aaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:52.313477 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7d8f-3614-40ab-916a-30092e43aaad None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:52.313765 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7d8f-3614-40ab-916a-30092e43aaad None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:52.314048 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c7d8f-3614-40ab-916a-30092e43aaad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:52.315101 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c7d8f-3614-40ab-916a-30092e43aaad None admin] RBAC: Adding query filter params (role.id=6a41401a07c443f0b5acd2e189145727, scope.project.id=5beb6dc9dc3f45d5919111c3e6c396e7, user.id=c0ff018c6850486aa01127bb8c1bd8b2) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:52.315418 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c7d8f-3614-40ab-916a-30092e43aaad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:52.316909 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c7d8f-3614-40ab-916a-30092e43aaad None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:52.327466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 51/97] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:45:52 2022] GET /identity/v3/role_assignments?user.id=c0ff018c6850486aa01127bb8c1bd8b2&scope.project.id=5beb6dc9dc3f45d5919111c3e6c396e7&role.id=6a41401a07c443f0b5acd2e189145727 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:54.295902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae46cd-99a9-4e13-acc0-7a1f88101209 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.296711 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae46cd-99a9-4e13-acc0-7a1f88101209 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.296711 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae46cd-99a9-4e13-acc0-7a1f88101209 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.298138 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 47/98] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:45:54.304618 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7649b1c-1e20-4b71-96d0-9ff0a92d80cb None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.305345 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7649b1c-1e20-4b71-96d0-9ff0a92d80cb None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.305345 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7649b1c-1e20-4b71-96d0-9ff0a92d80cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.312458 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80b394-d00b-420b-ae2f-67468cb21635 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.312839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80b394-d00b-420b-ae2f-67468cb21635 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.313070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80b394-d00b-420b-ae2f-67468cb21635 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.315296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 48/99] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:54.321027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58578b7-d72a-4395-9534-f44318975f53 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.321418 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58578b7-d72a-4395-9534-f44318975f53 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.321663 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58578b7-d72a-4395-9534-f44318975f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.342759 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-c7649b1c-1e20-4b71-96d0-9ff0a92d80cb None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:54.344004 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-c7649b1c-1e20-4b71-96d0-9ff0a92d80cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:54.360280 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-b58578b7-d72a-4395-9534-f44318975f53 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:54.362146 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-b58578b7-d72a-4395-9534-f44318975f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:54.443190 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 52/100] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:45:54.466260 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7649b1c-1e20-4b71-96d0-9ff0a92d80cb None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:54.493082 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:54.578876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7649b1c-1e20-4b71-96d0-9ff0a92d80cb None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:54.592952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 49/101] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:45:54.630123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58578b7-d72a-4395-9534-f44318975f53 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:54.634135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d9eb7-8dbf-4ea7-aa2b-c4dfa8b7791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:54.634967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d9eb7-8dbf-4ea7-aa2b-c4dfa8b7791e None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.635254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d9eb7-8dbf-4ea7-aa2b-c4dfa8b7791e None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.635534 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d9eb7-8dbf-4ea7-aa2b-c4dfa8b7791e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.653555 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d9eb7-8dbf-4ea7-aa2b-c4dfa8b7791e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-c04d9eb7-8dbf-4ea7-aa2b-c4dfa8b7791e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:54.662078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:54.663468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:45:54.664522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 53/102] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:45:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:45:54.672478 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d9eb7-8dbf-4ea7-aa2b-c4dfa8b7791e None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:54.776705 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:54.785773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:54.803990 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d9eb7-8dbf-4ea7-aa2b-c4dfa8b7791e None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:54.849305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58578b7-d72a-4395-9534-f44318975f53 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:54.849692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e6d816-b397-4864-8f31-e2025f1c3319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:54.851297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6d816-b397-4864-8f31-e2025f1c3319 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.851976 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6d816-b397-4864-8f31-e2025f1c3319 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.853920 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e6d816-b397-4864-8f31-e2025f1c3319 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.855624 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6d816-b397-4864-8f31-e2025f1c3319 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:54.856444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6d816-b397-4864-8f31-e2025f1c3319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:54.860095 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e6d816-b397-4864-8f31-e2025f1c3319 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:54.884435 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 54/103] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:45:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:54.887356 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d204ab-e82c-423f-b217-06c631523fd9 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.888075 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d204ab-e82c-423f-b217-06c631523fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.888982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d204ab-e82c-423f-b217-06c631523fd9 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.893278 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 55/104] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:54.896439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96c0cf-ebfc-4147-a70b-ad9930ee7e47 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.897142 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b9774-acb9-4c90-9fc7-2d779dd5a1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:54.897723 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96c0cf-ebfc-4147-a70b-ad9930ee7e47 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.898021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b9774-acb9-4c90-9fc7-2d779dd5a1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.898507 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b9774-acb9-4c90-9fc7-2d779dd5a1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.898819 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96c0cf-ebfc-4147-a70b-ad9930ee7e47 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.898994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b9774-acb9-4c90-9fc7-2d779dd5a1e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.903001 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 56/105] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:54.904777 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96c0cf-ebfc-4147-a70b-ad9930ee7e47 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:54.914587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b9774-acb9-4c90-9fc7-2d779dd5a1e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-b25b9774-acb9-4c90-9fc7-2d779dd5a1e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:54.923111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:54.925629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:45:54.927359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:45:54.929244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 50/106] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:45:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:54.931167 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da184b86-b6f3-4b1b-b499-ce921ac99ae8 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:54.931167 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da184b86-b6f3-4b1b-b499-ce921ac99ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:54.931481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da184b86-b6f3-4b1b-b499-ce921ac99ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:54.951107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-da184b86-b6f3-4b1b-b499-ce921ac99ae8 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:54.952580 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-da184b86-b6f3-4b1b-b499-ce921ac99ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:55.069907 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 51/107] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:55.073374 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10456-f4d1-4b8c-8d16-4b22a76074d0 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:55.074715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10456-f4d1-4b8c-8d16-4b22a76074d0 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:55.075749 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb10456-f4d1-4b8c-8d16-4b22a76074d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:55.085837 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96c0cf-ebfc-4147-a70b-ad9930ee7e47 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:55.120009 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-dfb10456-f4d1-4b8c-8d16-4b22a76074d0 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:55.121329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-dfb10456-f4d1-4b8c-8d16-4b22a76074d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:55.165435 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aee0e9-76a4-4e88-adc7-3a1fd8b39291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:55.167314 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aee0e9-76a4-4e88-adc7-3a1fd8b39291 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:55.168044 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aee0e9-76a4-4e88-adc7-3a1fd8b39291 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:55.168788 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aee0e9-76a4-4e88-adc7-3a1fd8b39291 None admin] PATH_INFO: `/v3/users/61a6d10d178d4987839e337f1cab52cc` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:55.191545 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:55.197202 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:55.204795 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aee0e9-76a4-4e88-adc7-3a1fd8b39291 None admin] RBAC: Authorizing `identity:get_user(user_id=61a6d10d178d4987839e337f1cab52cc)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:55.212543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 52/108] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:55.212543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb10456-f4d1-4b8c-8d16-4b22a76074d0 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:55.212869 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aee0e9-76a4-4e88-adc7-3a1fd8b39291 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:55.214109 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 57/109] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:54 2022] GET /identity/v3/users/61a6d10d178d4987839e337f1cab52cc => generated 289 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:55.215830 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aee0e9-76a4-4e88-adc7-3a1fd8b39291 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:55.364265 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb10456-f4d1-4b8c-8d16-4b22a76074d0 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:55.416590 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aee0e9-76a4-4e88-adc7-3a1fd8b39291 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:55.425954 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac2524-1b94-404c-b860-d70cd23796a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:55.427984 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2524-1b94-404c-b860-d70cd23796a7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:55.427984 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2524-1b94-404c-b860-d70cd23796a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:55.428936 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac2524-1b94-404c-b860-d70cd23796a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:55.430898 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac2524-1b94-404c-b860-d70cd23796a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:55.431357 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac2524-1b94-404c-b860-d70cd23796a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:55.434248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac2524-1b94-404c-b860-d70cd23796a7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:55.452195 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 53/110] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:45:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:55.453844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac2524-1b94-404c-b860-d70cd23796a7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:55.463431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2746ea-f782-4e99-9eb7-aa5c90301039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:55.464505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2746ea-f782-4e99-9eb7-aa5c90301039 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:55.469251 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2746ea-f782-4e99-9eb7-aa5c90301039 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:55.469251 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2746ea-f782-4e99-9eb7-aa5c90301039 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:55.479182 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2746ea-f782-4e99-9eb7-aa5c90301039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-db2746ea-f782-4e99-9eb7-aa5c90301039 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:55.489841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:55.490999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:45:55.491951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:45:55.500104 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 58/111] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:45:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:55.503352 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2746ea-f782-4e99-9eb7-aa5c90301039 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:55.571924 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:55.619993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:55.636706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac2524-1b94-404c-b860-d70cd23796a7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:55.681269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621900cc-62e1-4c5e-aa5a-a4860df7b6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:55.682161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621900cc-62e1-4c5e-aa5a-a4860df7b6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:55.682603 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621900cc-62e1-4c5e-aa5a-a4860df7b6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:55.683031 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621900cc-62e1-4c5e-aa5a-a4860df7b6b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:55.704612 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2746ea-f782-4e99-9eb7-aa5c90301039 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:55.769777 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb693a9-5976-4290-b137-b157eeb34f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:55.771099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb693a9-5976-4290-b137-b157eeb34f9c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:55.771585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb693a9-5976-4290-b137-b157eeb34f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:55.772062 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb693a9-5976-4290-b137-b157eeb34f9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:55.773980 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb693a9-5976-4290-b137-b157eeb34f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:55.776580 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb693a9-5976-4290-b137-b157eeb34f9c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:55.778586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621900cc-62e1-4c5e-aa5a-a4860df7b6b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:55.779562 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 59/112] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:45:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:55.780950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb693a9-5976-4290-b137-b157eeb34f9c None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:55.782876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621900cc-62e1-4c5e-aa5a-a4860df7b6b8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:55.869810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:55.943321 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb693a9-5976-4290-b137-b157eeb34f9c None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:55.955839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 54/113] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:45:55.955839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621900cc-62e1-4c5e-aa5a-a4860df7b6b8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:55.979828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:55.990838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e77091-ee71-4317-9447-5fddeb2d63b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:55.991966 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e77091-ee71-4317-9447-5fddeb2d63b8 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:55.992373 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e77091-ee71-4317-9447-5fddeb2d63b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:55.992744 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e77091-ee71-4317-9447-5fddeb2d63b8 None admin] PATH_INFO: `/v3/users/c0ff018c6850486aa01127bb8c1bd8b2` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:55.994543 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e77091-ee71-4317-9447-5fddeb2d63b8 None admin] RBAC: Authorizing `identity:get_user(user_id=c0ff018c6850486aa01127bb8c1bd8b2)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:55.996125 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e77091-ee71-4317-9447-5fddeb2d63b8 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:55.997878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 60/114] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:55 2022] GET /identity/v3/users/c0ff018c6850486aa01127bb8c1bd8b2 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:55.999155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb50dc2-1757-4a71-b76f-fed5f1829c90 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:55.999441 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb50dc2-1757-4a71-b76f-fed5f1829c90 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:55.999719 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb50dc2-1757-4a71-b76f-fed5f1829c90 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:56.001321 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 61/115] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:56.004606 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb50dc2-1757-4a71-b76f-fed5f1829c90 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:56.091621 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621900cc-62e1-4c5e-aa5a-a4860df7b6b8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:56.145755 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5326d-2d30-4a6e-99b2-6287fa50c812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:56.146460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5326d-2d30-4a6e-99b2-6287fa50c812 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:56.146749 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5326d-2d30-4a6e-99b2-6287fa50c812 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:56.147029 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5326d-2d30-4a6e-99b2-6287fa50c812 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:56.147837 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5326d-2d30-4a6e-99b2-6287fa50c812 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:56.148234 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5326d-2d30-4a6e-99b2-6287fa50c812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:56.149949 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5326d-2d30-4a6e-99b2-6287fa50c812 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:56.151619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb50dc2-1757-4a71-b76f-fed5f1829c90 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:56.160517 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 55/116] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:45:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:56.161442 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5326d-2d30-4a6e-99b2-6287fa50c812 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:56.186899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:56.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1699b7-625c-457b-b369-cf36a74f8ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:56.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1699b7-625c-457b-b369-cf36a74f8ef9 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:56.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1699b7-625c-457b-b369-cf36a74f8ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:56.203336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1699b7-625c-457b-b369-cf36a74f8ef9 None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:56.203639 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1699b7-625c-457b-b369-cf36a74f8ef9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:56.205146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1699b7-625c-457b-b369-cf36a74f8ef9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:56.281795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5326d-2d30-4a6e-99b2-6287fa50c812 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:56.291012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 62/117] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:45:56 2022] POST /identity/v3/projects => generated 302 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:45:56.293285 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5af1db-2e25-4796-8d2f-1e39fbed0b4b None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:56.293853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5af1db-2e25-4796-8d2f-1e39fbed0b4b None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:56.294451 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5af1db-2e25-4796-8d2f-1e39fbed0b4b None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:56.300647 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 63/118] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:45:56.300647 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5af1db-2e25-4796-8d2f-1e39fbed0b4b None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:56.330436 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84452b-bb7e-4c59-b575-80357640c96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:56.331337 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84452b-bb7e-4c59-b575-80357640c96d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:56.331743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84452b-bb7e-4c59-b575-80357640c96d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:56.332161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84452b-bb7e-4c59-b575-80357640c96d None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:56.333729 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84452b-bb7e-4c59-b575-80357640c96d None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:56.335417 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84452b-bb7e-4c59-b575-80357640c96d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:56.337543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 56/119] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:45:56 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:56.338573 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c07880-9589-4ac8-bde6-6d1cb11e96fd None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:56.338841 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c07880-9589-4ac8-bde6-6d1cb11e96fd None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:56.339121 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c07880-9589-4ac8-bde6-6d1cb11e96fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:56.360632 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-98c07880-9589-4ac8-bde6-6d1cb11e96fd None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:56.362439 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-98c07880-9589-4ac8-bde6-6d1cb11e96fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:56.418156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:56.506766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 57/120] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:56.506766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c07880-9589-4ac8-bde6-6d1cb11e96fd None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:56.533410 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5af1db-2e25-4796-8d2f-1e39fbed0b4b None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:56.535825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:56.594790 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88288b59-b219-4264-917f-6588936ffac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:56.595788 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88288b59-b219-4264-917f-6588936ffac0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:56.596088 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88288b59-b219-4264-917f-6588936ffac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:56.596494 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88288b59-b219-4264-917f-6588936ffac0 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:56.597778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88288b59-b219-4264-917f-6588936ffac0 None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:56.599252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88288b59-b219-4264-917f-6588936ffac0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:56.604716 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 64/121] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:45:56 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:56.604716 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920ffdd-d1bf-4471-a7d9-714702b000d5 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:56.605318 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920ffdd-d1bf-4471-a7d9-714702b000d5 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:56.605806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920ffdd-d1bf-4471-a7d9-714702b000d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:56.630227 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-5920ffdd-d1bf-4471-a7d9-714702b000d5 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:56.631723 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-5920ffdd-d1bf-4471-a7d9-714702b000d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:56.664586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c07880-9589-4ac8-bde6-6d1cb11e96fd None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:56.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059ddb8-b949-4696-bbb8-f06ffb15edbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:56.706894 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059ddb8-b949-4696-bbb8-f06ffb15edbc None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:56.707183 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059ddb8-b949-4696-bbb8-f06ffb15edbc None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:56.707459 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059ddb8-b949-4696-bbb8-f06ffb15edbc None admin] PATH_INFO: `/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:56.708758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059ddb8-b949-4696-bbb8-f06ffb15edbc None admin] RBAC: Authorizing `identity:get_user(user_id=3f3a6a2da4254702bc7bd15361b6c9ae)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:56.710469 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059ddb8-b949-4696-bbb8-f06ffb15edbc None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:56.712185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 58/122] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:56 2022] GET /identity/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:56.716983 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059ddb8-b949-4696-bbb8-f06ffb15edbc None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:56.719888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 65/123] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:56.721111 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920ffdd-d1bf-4471-a7d9-714702b000d5 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:56.783391 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:56.813507 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:56.838206 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059ddb8-b949-4696-bbb8-f06ffb15edbc None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:56.871365 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920ffdd-d1bf-4471-a7d9-714702b000d5 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:56.894532 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28484090-02af-4d21-8752-d5d08ec57cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:56.895204 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28484090-02af-4d21-8752-d5d08ec57cec None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:56.895477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28484090-02af-4d21-8752-d5d08ec57cec None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:56.895928 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28484090-02af-4d21-8752-d5d08ec57cec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:56.896807 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28484090-02af-4d21-8752-d5d08ec57cec None admin] RBAC: Adding query filter params (role.id=bd9a605a759046cf9bd1a8e9bb20d9d4, scope.project.id=f14d8cdc0ed5452ca04a981594be5974, user.id=61a6d10d178d4987839e337f1cab52cc) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:56.897117 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28484090-02af-4d21-8752-d5d08ec57cec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:56.898865 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28484090-02af-4d21-8752-d5d08ec57cec None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:56.905557 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 59/124] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:45:56 2022] GET /identity/v3/role_assignments?user.id=61a6d10d178d4987839e337f1cab52cc&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=bd9a605a759046cf9bd1a8e9bb20d9d4 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:56.913085 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28484090-02af-4d21-8752-d5d08ec57cec None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:56.918312 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269066bc-a899-4c88-941a-3eb67ed28d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:56.919651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269066bc-a899-4c88-941a-3eb67ed28d67 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:56.920104 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269066bc-a899-4c88-941a-3eb67ed28d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:56.920590 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269066bc-a899-4c88-941a-3eb67ed28d67 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:56.945793 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269066bc-a899-4c88-941a-3eb67ed28d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-269066bc-a899-4c88-941a-3eb67ed28d67 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:56.964956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:56.965878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:45:56.966680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:45:56.972367 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 66/125] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:45:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:56.972367 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269066bc-a899-4c88-941a-3eb67ed28d67 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:57.124905 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28484090-02af-4d21-8752-d5d08ec57cec None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:57.168692 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24397a1e-6dec-496a-9b8e-c30eb10408ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:57.168920 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269066bc-a899-4c88-941a-3eb67ed28d67 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:57.170279 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24397a1e-6dec-496a-9b8e-c30eb10408ab None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:57.170799 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24397a1e-6dec-496a-9b8e-c30eb10408ab None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:57.171317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24397a1e-6dec-496a-9b8e-c30eb10408ab None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974/users/c0ff018c6850486aa01127bb8c1bd8b2/roles/6a41401a07c443f0b5acd2e189145727` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:57.171422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:57.175852 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24397a1e-6dec-496a-9b8e-c30eb10408ab None admin] RBAC: Authorizing `identity:create_grant(project_id=f14d8cdc0ed5452ca04a981594be5974, user_id=c0ff018c6850486aa01127bb8c1bd8b2, role_id=6a41401a07c443f0b5acd2e189145727)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:57.179090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24397a1e-6dec-496a-9b8e-c30eb10408ab None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:57.209829 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5164a0-4a21-41af-a3be-90a15411a0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:57.210770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5164a0-4a21-41af-a3be-90a15411a0ef None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:57.211050 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5164a0-4a21-41af-a3be-90a15411a0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:57.211326 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5164a0-4a21-41af-a3be-90a15411a0ef None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:57.212912 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5164a0-4a21-41af-a3be-90a15411a0ef None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:57.214400 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5164a0-4a21-41af-a3be-90a15411a0ef None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:57.216419 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 67/126] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:45:56 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:57.222311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5164a0-4a21-41af-a3be-90a15411a0ef None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:57.284114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 60/127] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:45:56 2022] PUT /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974/users/c0ff018c6850486aa01127bb8c1bd8b2/roles/6a41401a07c443f0b5acd2e189145727 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:45:57.287299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24397a1e-6dec-496a-9b8e-c30eb10408ab None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:57.305147 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:57.396778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5164a0-4a21-41af-a3be-90a15411a0ef None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:57.434634 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24397a1e-6dec-496a-9b8e-c30eb10408ab None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:57.477467 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643daa88-7edd-4ef4-8155-e3ad0ff87106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:57.478554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643daa88-7edd-4ef4-8155-e3ad0ff87106 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:57.479053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643daa88-7edd-4ef4-8155-e3ad0ff87106 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:57.479537 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643daa88-7edd-4ef4-8155-e3ad0ff87106 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:57.480744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643daa88-7edd-4ef4-8155-e3ad0ff87106 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:57.481360 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643daa88-7edd-4ef4-8155-e3ad0ff87106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:57.483197 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643daa88-7edd-4ef4-8155-e3ad0ff87106 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:57.494845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 61/128] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:45:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:57.496147 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643daa88-7edd-4ef4-8155-e3ad0ff87106 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:57.550220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4549-3b07-43ba-897e-956966e0eb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:57.551170 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4549-3b07-43ba-897e-956966e0eb02 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:57.551463 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4549-3b07-43ba-897e-956966e0eb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:57.551743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4549-3b07-43ba-897e-956966e0eb02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:57.592064 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c4549-3b07-43ba-897e-956966e0eb02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:45:57.594477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-ce2c4549-3b07-43ba-897e-956966e0eb02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:45:57.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:45:57.610157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:45:57.610949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 68/129] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:45:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 394 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:45:57.611958 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4549-3b07-43ba-897e-956966e0eb02 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:57.664914 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643daa88-7edd-4ef4-8155-e3ad0ff87106 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:57.665081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:57.730653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bcebd-2453-4e11-88ca-05b647481c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:57.731589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bcebd-2453-4e11-88ca-05b647481c91 None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:57.731871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bcebd-2453-4e11-88ca-05b647481c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:57.732144 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bcebd-2453-4e11-88ca-05b647481c91 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974/users/3f3a6a2da4254702bc7bd15361b6c9ae/roles/815bd48c3df0405babfd5b53430fd3ea` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:57.735995 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bcebd-2453-4e11-88ca-05b647481c91 None admin] RBAC: Authorizing `identity:create_grant(project_id=f14d8cdc0ed5452ca04a981594be5974, user_id=3f3a6a2da4254702bc7bd15361b6c9ae, role_id=815bd48c3df0405babfd5b53430fd3ea)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:57.737686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bcebd-2453-4e11-88ca-05b647481c91 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:57.821945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4549-3b07-43ba-897e-956966e0eb02 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:57.854932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 62/130] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:45:57 2022] PUT /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974/users/3f3a6a2da4254702bc7bd15361b6c9ae/roles/815bd48c3df0405babfd5b53430fd3ea => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:45:57.855952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bcebd-2453-4e11-88ca-05b647481c91 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:57.897808 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170bbce-31a7-4a0b-b15b-b4568a238dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:57.899911 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170bbce-31a7-4a0b-b15b-b4568a238dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:57.900844 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170bbce-31a7-4a0b-b15b-b4568a238dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:57.901613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170bbce-31a7-4a0b-b15b-b4568a238dd4 None admin] PATH_INFO: `/v3/users/c0ff018c6850486aa01127bb8c1bd8b2` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:57.904732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170bbce-31a7-4a0b-b15b-b4568a238dd4 None admin] RBAC: Authorizing `identity:get_user(user_id=c0ff018c6850486aa01127bb8c1bd8b2)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:57.908926 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170bbce-31a7-4a0b-b15b-b4568a238dd4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:57.914036 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 69/131] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:57 2022] GET /identity/v3/users/c0ff018c6850486aa01127bb8c1bd8b2 => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:57.921488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170bbce-31a7-4a0b-b15b-b4568a238dd4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:58.112509 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:58.117115 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170bbce-31a7-4a0b-b15b-b4568a238dd4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.164112 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bcebd-2453-4e11-88ca-05b647481c91 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.164308 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a850da5-67ce-46f5-9ab0-9ec3a92594c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.165367 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a850da5-67ce-46f5-9ab0-9ec3a92594c9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.165755 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a850da5-67ce-46f5-9ab0-9ec3a92594c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.166130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a850da5-67ce-46f5-9ab0-9ec3a92594c9 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.171009 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a850da5-67ce-46f5-9ab0-9ec3a92594c9 None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.172811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a850da5-67ce-46f5-9ab0-9ec3a92594c9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:58.175233 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 70/132] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:45:57 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:58.181629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a850da5-67ce-46f5-9ab0-9ec3a92594c9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:58.201098 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:58.210510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25019c3-adfb-447d-9654-b46673159759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.211584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25019c3-adfb-447d-9654-b46673159759 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.212076 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25019c3-adfb-447d-9654-b46673159759 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.212664 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25019c3-adfb-447d-9654-b46673159759 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.213878 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25019c3-adfb-447d-9654-b46673159759 None admin] RBAC: Adding query filter params (name=member) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:45:58.214432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25019c3-adfb-447d-9654-b46673159759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.216109 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25019c3-adfb-447d-9654-b46673159759 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:58.226883 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 63/133] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:45:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:58.233269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25019c3-adfb-447d-9654-b46673159759 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:58.296882 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a850da5-67ce-46f5-9ab0-9ec3a92594c9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.334958 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d4b2b-b7b9-48cf-8b3a-1dacbb99b909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.335884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d4b2b-b7b9-48cf-8b3a-1dacbb99b909 None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.336165 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d4b2b-b7b9-48cf-8b3a-1dacbb99b909 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.336471 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d4b2b-b7b9-48cf-8b3a-1dacbb99b909 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974/users/c0ff018c6850486aa01127bb8c1bd8b2/roles/cb116a5614cf4a48ad99840d95f96bb3` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.338972 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d4b2b-b7b9-48cf-8b3a-1dacbb99b909 None admin] RBAC: Authorizing `identity:create_grant(project_id=f14d8cdc0ed5452ca04a981594be5974, user_id=c0ff018c6850486aa01127bb8c1bd8b2, role_id=cb116a5614cf4a48ad99840d95f96bb3)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.340492 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0d4b2b-b7b9-48cf-8b3a-1dacbb99b909 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:58.375546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25019c3-adfb-447d-9654-b46673159759 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.412666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31ceff-9bba-4570-8f57-f70941c1ce5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.413634 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31ceff-9bba-4570-8f57-f70941c1ce5a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.413922 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31ceff-9bba-4570-8f57-f70941c1ce5a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.414203 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e31ceff-9bba-4570-8f57-f70941c1ce5a None admin] PATH_INFO: `/v3/users/c0ff018c6850486aa01127bb8c1bd8b2` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.415461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31ceff-9bba-4570-8f57-f70941c1ce5a None admin] RBAC: Authorizing `identity:get_user(user_id=c0ff018c6850486aa01127bb8c1bd8b2)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.416982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e31ceff-9bba-4570-8f57-f70941c1ce5a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:58.418990 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 64/134] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:45:58 2022] GET /identity/v3/users/c0ff018c6850486aa01127bb8c1bd8b2 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:45:58.426005 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31ceff-9bba-4570-8f57-f70941c1ce5a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:58.432007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 71/135] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:45:58 2022] PUT /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974/users/c0ff018c6850486aa01127bb8c1bd8b2/roles/cb116a5614cf4a48ad99840d95f96bb3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:45:58.528763 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:58.595226 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e31ceff-9bba-4570-8f57-f70941c1ce5a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.648371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194026d-6258-4244-abc4-62cd2653f100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.649286 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194026d-6258-4244-abc4-62cd2653f100 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.649575 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194026d-6258-4244-abc4-62cd2653f100 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.649856 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194026d-6258-4244-abc4-62cd2653f100 None admin] PATH_INFO: `/v3/projects/5beb6dc9dc3f45d5919111c3e6c396e7` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.651214 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194026d-6258-4244-abc4-62cd2653f100 None admin] RBAC: Authorizing `identity:get_project(project_id=5beb6dc9dc3f45d5919111c3e6c396e7)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.652859 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194026d-6258-4244-abc4-62cd2653f100 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:58.654861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 65/136] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:45:58 2022] GET /identity/v3/projects/5beb6dc9dc3f45d5919111c3e6c396e7 => generated 312 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:45:58.665530 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d4b2b-b7b9-48cf-8b3a-1dacbb99b909 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:45:58.715925 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:45:58.837211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d4b2b-b7b9-48cf-8b3a-1dacbb99b909 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:45:58.925286 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fd5ab7-9045-40d3-b010-09c97e13fd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:45:58.926928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd5ab7-9045-40d3-b010-09c97e13fd4e None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:58.927421 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd5ab7-9045-40d3-b010-09c97e13fd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:58.927934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fd5ab7-9045-40d3-b010-09c97e13fd4e None admin] PATH_INFO: `/v3/projects/5beb6dc9dc3f45d5919111c3e6c396e7/users/c0ff018c6850486aa01127bb8c1bd8b2/roles/6a41401a07c443f0b5acd2e189145727` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:58.933167 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd5ab7-9045-40d3-b010-09c97e13fd4e None admin] RBAC: Authorizing `identity:create_grant(project_id=5beb6dc9dc3f45d5919111c3e6c396e7, user_id=c0ff018c6850486aa01127bb8c1bd8b2, role_id=6a41401a07c443f0b5acd2e189145727)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:45:58.935763 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fd5ab7-9045-40d3-b010-09c97e13fd4e None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:45:59.041560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 72/137] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:45:58 2022] PUT /identity/v3/projects/5beb6dc9dc3f45d5919111c3e6c396e7/users/c0ff018c6850486aa01127bb8c1bd8b2/roles/6a41401a07c443f0b5acd2e189145727 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:45:59.663749 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff9a2-0090-4d3a-9c79-67c120bc99b4 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:59.664231 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff9a2-0090-4d3a-9c79-67c120bc99b4 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:59.664231 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eff9a2-0090-4d3a-9c79-67c120bc99b4 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:59.665658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 66/138] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:45:59.682049 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ddf6a-e1a1-43c3-a030-a31aded79266 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:45:59.682671 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ddf6a-e1a1-43c3-a030-a31aded79266 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:45:59.682977 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ddf6a-e1a1-43c3-a030-a31aded79266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:45:59.713425 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-550ddf6a-e1a1-43c3-a030-a31aded79266 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:45:59.715122 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-550ddf6a-e1a1-43c3-a030-a31aded79266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:45:59.806988 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 73/139] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:45:59.857948 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eff9a2-0090-4d3a-9c79-67c120bc99b4 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:00.002136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eff9a2-0090-4d3a-9c79-67c120bc99b4 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:00.039973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:00.047529 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fc6c40-7f6b-4c42-879f-d93594e475ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:00.048361 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc6c40-7f6b-4c42-879f-d93594e475ab None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:00.048682 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc6c40-7f6b-4c42-879f-d93594e475ab None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:00.049500 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fc6c40-7f6b-4c42-879f-d93594e475ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:00.050719 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc6c40-7f6b-4c42-879f-d93594e475ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:00.052196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fc6c40-7f6b-4c42-879f-d93594e475ab None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:00.054111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 67/140] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:45:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:00.059896 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ddf6a-e1a1-43c3-a030-a31aded79266 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:00.092362 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:00.194140 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ddf6a-e1a1-43c3-a030-a31aded79266 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:00.240660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80227e-b318-4515-b7f5-6a51ca82996d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:00.241677 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80227e-b318-4515-b7f5-6a51ca82996d None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:00.242165 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80227e-b318-4515-b7f5-6a51ca82996d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:00.242715 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80227e-b318-4515-b7f5-6a51ca82996d None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:00.243658 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80227e-b318-4515-b7f5-6a51ca82996d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:00.245441 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80227e-b318-4515-b7f5-6a51ca82996d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:00.359814 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 74/141] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:46:00 2022] POST /identity/v3/users => generated 283 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:00.542102 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947ae21-7401-438f-a023-f52f07b79235 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:00.542394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947ae21-7401-438f-a023-f52f07b79235 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:00.542956 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947ae21-7401-438f-a023-f52f07b79235 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:00.544601 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 68/142] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:00.550454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b7e96-172f-470b-8a9b-b8f8e64e3a06 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:00.550862 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b7e96-172f-470b-8a9b-b8f8e64e3a06 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:00.551225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b7e96-172f-470b-8a9b-b8f8e64e3a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:00.576033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-b21b7e96-172f-470b-8a9b-b8f8e64e3a06 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:00.577315 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-b21b7e96-172f-470b-8a9b-b8f8e64e3a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:00.681974 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 75/143] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:00.713652 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947ae21-7401-438f-a023-f52f07b79235 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:00.865251 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947ae21-7401-438f-a023-f52f07b79235 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:00.921127 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a4e84-4df1-4f9a-aea6-971353fae4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:00.921964 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a4e84-4df1-4f9a-aea6-971353fae4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:00.922255 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a4e84-4df1-4f9a-aea6-971353fae4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:00.922537 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a4e84-4df1-4f9a-aea6-971353fae4d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:00.938584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04a4e84-4df1-4f9a-aea6-971353fae4d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-e04a4e84-4df1-4f9a-aea6-971353fae4d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:00.947291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:00.948174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:00.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 69/144] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:00.956005 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b7e96-172f-470b-8a9b-b8f8e64e3a06 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:00.964363 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:01.097872 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b7e96-172f-470b-8a9b-b8f8e64e3a06 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:01.118879 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382f16e-bdca-4d23-b0d7-d07bfb6827e4 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:01.119193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382f16e-bdca-4d23-b0d7-d07bfb6827e4 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:01.119441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382f16e-bdca-4d23-b0d7-d07bfb6827e4 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:01.120902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 70/145] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:01.128322 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d7d9f-cc8c-463a-88fa-ed6b658d7367 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:01.128644 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d7d9f-cc8c-463a-88fa-ed6b658d7367 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:01.128921 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d7d9f-cc8c-463a-88fa-ed6b658d7367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:01.161445 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-1d0d7d9f-cc8c-463a-88fa-ed6b658d7367 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:01.163466 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-1d0d7d9f-cc8c-463a-88fa-ed6b658d7367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:01.170007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff5ce3-35df-4765-ac63-7d5d72f3b41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:01.171023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff5ce3-35df-4765-ac63-7d5d72f3b41f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:01.171524 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff5ce3-35df-4765-ac63-7d5d72f3b41f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:01.171987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff5ce3-35df-4765-ac63-7d5d72f3b41f None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:01.172988 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff5ce3-35df-4765-ac63-7d5d72f3b41f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:01.175233 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff5ce3-35df-4765-ac63-7d5d72f3b41f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:01.176879 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff5ce3-35df-4765-ac63-7d5d72f3b41f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:01.194386 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 76/146] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:01.202270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff5ce3-35df-4765-ac63-7d5d72f3b41f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:01.274467 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:01.310580 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 71/147] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:01.346518 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d7d9f-cc8c-463a-88fa-ed6b658d7367 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:01.351110 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:01.417399 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff5ce3-35df-4765-ac63-7d5d72f3b41f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:01.494190 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56f0bf-80a4-4dbd-b115-383245988975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:01.495005 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56f0bf-80a4-4dbd-b115-383245988975 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:01.495593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56f0bf-80a4-4dbd-b115-383245988975 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:01.495593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56f0bf-80a4-4dbd-b115-383245988975 None admin] PATH_INFO: `/v3/users/61a6d10d178d4987839e337f1cab52cc` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:01.497067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56f0bf-80a4-4dbd-b115-383245988975 None admin] RBAC: Authorizing `identity:get_user(user_id=61a6d10d178d4987839e337f1cab52cc)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:01.498558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56f0bf-80a4-4dbd-b115-383245988975 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:01.500241 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 77/148] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:01 2022] GET /identity/v3/users/61a6d10d178d4987839e337f1cab52cc => generated 289 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:01.506367 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56f0bf-80a4-4dbd-b115-383245988975 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:01.529758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d7d9f-cc8c-463a-88fa-ed6b658d7367 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:01.574848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91507fe5-0852-4b1c-ac49-6115736cb013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:01.576023 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91507fe5-0852-4b1c-ac49-6115736cb013 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:01.576526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91507fe5-0852-4b1c-ac49-6115736cb013 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:01.577004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91507fe5-0852-4b1c-ac49-6115736cb013 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:01.590193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91507fe5-0852-4b1c-ac49-6115736cb013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-91507fe5-0852-4b1c-ac49-6115736cb013 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:01.599220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:01.600129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:01.600975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:01.602581 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 72/149] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:01.604075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5917a-863c-45b5-bde5-2483c164bb5b None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:01.608443 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5917a-863c-45b5-bde5-2483c164bb5b None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:01.608443 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5917a-863c-45b5-bde5-2483c164bb5b None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:01.608443 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 73/150] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:01.610713 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5917a-863c-45b5-bde5-2483c164bb5b None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:01.694461 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56f0bf-80a4-4dbd-b115-383245988975 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:01.698308 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:01.733130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bf53e-4b9b-4745-a3a6-b89547188a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:01.734015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bf53e-4b9b-4745-a3a6-b89547188a0d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:01.734297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bf53e-4b9b-4745-a3a6-b89547188a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:01.734573 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bf53e-4b9b-4745-a3a6-b89547188a0d None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:01.735880 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bf53e-4b9b-4745-a3a6-b89547188a0d None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:01.738424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9bf53e-4b9b-4745-a3a6-b89547188a0d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:01.740637 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 78/151] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:01 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:01.742427 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc3a1-fdd3-4c21-b80a-ee5eb86487bb None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:01.742939 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc3a1-fdd3-4c21-b80a-ee5eb86487bb None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:01.743389 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbfc3a1-fdd3-4c21-b80a-ee5eb86487bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:01.749094 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:01.763663 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-efbfc3a1-fdd3-4c21-b80a-ee5eb86487bb None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:01.765262 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-efbfc3a1-fdd3-4c21-b80a-ee5eb86487bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:01.841978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 79/152] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:01.846868 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfc3a1-fdd3-4c21-b80a-ee5eb86487bb None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:01.873750 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5917a-863c-45b5-bde5-2483c164bb5b None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:01.985098 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbfc3a1-fdd3-4c21-b80a-ee5eb86487bb None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:02.017913 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9efbe3b-edd1-46b1-95b4-0598bb715f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.018638 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efbe3b-edd1-46b1-95b4-0598bb715f06 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.018998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efbe3b-edd1-46b1-95b4-0598bb715f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.019508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efbe3b-edd1-46b1-95b4-0598bb715f06 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.021160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9efbe3b-edd1-46b1-95b4-0598bb715f06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:02.021806 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9efbe3b-edd1-46b1-95b4-0598bb715f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:02.023323 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:02.024244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9efbe3b-edd1-46b1-95b4-0598bb715f06 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:02.027486 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8409bda-6f8c-4fc6-ab07-bba22fceb8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.029093 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8409bda-6f8c-4fc6-ab07-bba22fceb8a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.029791 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8409bda-6f8c-4fc6-ab07-bba22fceb8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.030267 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8409bda-6f8c-4fc6-ab07-bba22fceb8a9 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974/users/61a6d10d178d4987839e337f1cab52cc/roles/bd9a605a759046cf9bd1a8e9bb20d9d4` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.033612 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8409bda-6f8c-4fc6-ab07-bba22fceb8a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f14d8cdc0ed5452ca04a981594be5974, user_id=61a6d10d178d4987839e337f1cab52cc, role_id=bd9a605a759046cf9bd1a8e9bb20d9d4)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:02.035466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8409bda-6f8c-4fc6-ab07-bba22fceb8a9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:02.042632 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 74/153] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:02.043821 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9efbe3b-edd1-46b1-95b4-0598bb715f06 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:02.143539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 80/154] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:46:01 2022] PUT /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974/users/61a6d10d178d4987839e337f1cab52cc/roles/bd9a605a759046cf9bd1a8e9bb20d9d4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:02.144873 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8409bda-6f8c-4fc6-ab07-bba22fceb8a9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:02.211432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9efbe3b-edd1-46b1-95b4-0598bb715f06 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:02.227508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:02.288526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecac7cb-84ca-4e3e-9237-f24bfdd89977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.291052 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecac7cb-84ca-4e3e-9237-f24bfdd89977 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.291330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecac7cb-84ca-4e3e-9237-f24bfdd89977 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.291605 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecac7cb-84ca-4e3e-9237-f24bfdd89977 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.322912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecac7cb-84ca-4e3e-9237-f24bfdd89977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-cecac7cb-84ca-4e3e-9237-f24bfdd89977 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:02.332160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:02.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:02.334027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:02.338141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 75/155] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:46:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:02.345935 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecac7cb-84ca-4e3e-9237-f24bfdd89977 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:02.359327 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8409bda-6f8c-4fc6-ab07-bba22fceb8a9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:02.404272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3fddc-4590-4db2-b9e4-a79d5b4eb5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.405443 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3fddc-4590-4db2-b9e4-a79d5b4eb5fd None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.405876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3fddc-4590-4db2-b9e4-a79d5b4eb5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.406294 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3fddc-4590-4db2-b9e4-a79d5b4eb5fd None admin] PATH_INFO: `/v3/users/c0ff018c6850486aa01127bb8c1bd8b2` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.408759 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3fddc-4590-4db2-b9e4-a79d5b4eb5fd None admin] RBAC: Authorizing `identity:get_user(user_id=c0ff018c6850486aa01127bb8c1bd8b2)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:02.410770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3fddc-4590-4db2-b9e4-a79d5b4eb5fd None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:02.412815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 81/156] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:02 2022] GET /identity/v3/users/c0ff018c6850486aa01127bb8c1bd8b2 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:02.423302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3fddc-4590-4db2-b9e4-a79d5b4eb5fd None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:02.521904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:02.582711 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecac7cb-84ca-4e3e-9237-f24bfdd89977 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:02.619332 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3fddc-4590-4db2-b9e4-a79d5b4eb5fd None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:02.665128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6d73f-f92e-4569-a27b-da81e14f7b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.665817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6d73f-f92e-4569-a27b-da81e14f7b59 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.666103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6d73f-f92e-4569-a27b-da81e14f7b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.666374 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6d73f-f92e-4569-a27b-da81e14f7b59 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.667128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6d73f-f92e-4569-a27b-da81e14f7b59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:02.667499 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6d73f-f92e-4569-a27b-da81e14f7b59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:02.669110 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6d73f-f92e-4569-a27b-da81e14f7b59 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:02.688259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 76/157] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:02.695662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6d73f-f92e-4569-a27b-da81e14f7b59 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:02.718153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f753010-604d-4d31-abb7-9135b82c3397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.718813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f753010-604d-4d31-abb7-9135b82c3397 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.719096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f753010-604d-4d31-abb7-9135b82c3397 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.719371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f753010-604d-4d31-abb7-9135b82c3397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.743939 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:02.820835 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f753010-604d-4d31-abb7-9135b82c3397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:02.823060 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f753010-604d-4d31-abb7-9135b82c3397 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:02.882687 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6d73f-f92e-4569-a27b-da81e14f7b59 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:02.944363 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:02.987576 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9535f40-3b7e-4b3f-a687-fefb29c32bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:02.988484 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9535f40-3b7e-4b3f-a687-fefb29c32bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:02.988761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9535f40-3b7e-4b3f-a687-fefb29c32bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:02.989030 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9535f40-3b7e-4b3f-a687-fefb29c32bc5 None admin] PATH_INFO: `/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:02.993097 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9535f40-3b7e-4b3f-a687-fefb29c32bc5 None admin] RBAC: Authorizing `identity:get_user(user_id=3f3a6a2da4254702bc7bd15361b6c9ae)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:02.994584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9535f40-3b7e-4b3f-a687-fefb29c32bc5 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:02.996682 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 77/158] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:02 2022] GET /identity/v3/users/3f3a6a2da4254702bc7bd15361b6c9ae => generated 247 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:03.000357 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa0012-227f-41ee-8c30-366bd3c435f6 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.000357 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa0012-227f-41ee-8c30-366bd3c435f6 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.000357 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa0012-227f-41ee-8c30-366bd3c435f6 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.000357 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 78/159] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:03.001347 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a422e-c926-4fbd-80a1-5cca255d11b1 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.001657 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a422e-c926-4fbd-80a1-5cca255d11b1 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.001937 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a422e-c926-4fbd-80a1-5cca255d11b1 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.003257 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 79/160] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:03.009442 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a422e-c926-4fbd-80a1-5cca255d11b1 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:03.030519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 82/161] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:46:03.032673 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07fbec-88d7-44b2-84e4-2a4b974c8a5d None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.033732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07fbec-88d7-44b2-84e4-2a4b974c8a5d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.034231 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07fbec-88d7-44b2-84e4-2a4b974c8a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.065349 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-ea07fbec-88d7-44b2-84e4-2a4b974c8a5d None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:03.066973 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-ea07fbec-88d7-44b2-84e4-2a4b974c8a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:03.169297 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:03.182202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a422e-c926-4fbd-80a1-5cca255d11b1 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:03.238428 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492f9b4-946f-48e5-8c23-4478ac3ddf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:03.239083 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492f9b4-946f-48e5-8c23-4478ac3ddf91 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.239362 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492f9b4-946f-48e5-8c23-4478ac3ddf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.239633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492f9b4-946f-48e5-8c23-4478ac3ddf91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.290315 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 83/162] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:03.291802 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e672-f808-4aab-8041-dfd65defa7c5 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.292100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e672-f808-4aab-8041-dfd65defa7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.292399 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f32e672-f808-4aab-8041-dfd65defa7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.320785 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-1f32e672-f808-4aab-8041-dfd65defa7c5 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:03.322016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-1f32e672-f808-4aab-8041-dfd65defa7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:03.359538 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492f9b4-946f-48e5-8c23-4478ac3ddf91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:03.361259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492f9b4-946f-48e5-8c23-4478ac3ddf91 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:03.456443 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 84/163] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:03.456443 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32e672-f808-4aab-8041-dfd65defa7c5 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:03.490466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:03.560512 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:03.582921 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 80/164] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:03.584160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492f9b4-946f-48e5-8c23-4478ac3ddf91 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:03.613486 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f32e672-f808-4aab-8041-dfd65defa7c5 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:03.674091 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35320b-0078-4587-bdad-08734a044644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:03.675061 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35320b-0078-4587-bdad-08734a044644 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.675343 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35320b-0078-4587-bdad-08734a044644 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.675619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35320b-0078-4587-bdad-08734a044644 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.676981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35320b-0078-4587-bdad-08734a044644 None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:03.678721 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35320b-0078-4587-bdad-08734a044644 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:03.683238 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 85/165] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:03 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:03.684097 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35320b-0078-4587-bdad-08734a044644 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:03.764374 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492f9b4-946f-48e5-8c23-4478ac3ddf91 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:03.823785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6ab27-f097-47ab-9a3f-6a2053129504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:03.824637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6ab27-f097-47ab-9a3f-6a2053129504 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.824922 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6ab27-f097-47ab-9a3f-6a2053129504 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6ab27-f097-47ab-9a3f-6a2053129504 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.846738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6ab27-f097-47ab-9a3f-6a2053129504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:03.860788 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-04f6ab27-f097-47ab-9a3f-6a2053129504 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:03.863433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:03.864371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:03.865211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:03.872838 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 81/166] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:46:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:03.873723 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6ab27-f097-47ab-9a3f-6a2053129504 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:03.874788 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35320b-0078-4587-bdad-08734a044644 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:03.956827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca473c6-fd44-4928-8a23-ef9b9f401502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:03.957695 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca473c6-fd44-4928-8a23-ef9b9f401502 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:03.958019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca473c6-fd44-4928-8a23-ef9b9f401502 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:03.958304 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca473c6-fd44-4928-8a23-ef9b9f401502 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:03.978801 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca473c6-fd44-4928-8a23-ef9b9f401502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:03.993787 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-fca473c6-fd44-4928-8a23-ef9b9f401502 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:03.997505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:03.998482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:03.999254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 86/167] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:04.005324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca473c6-fd44-4928-8a23-ef9b9f401502 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:04.102350 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6ab27-f097-47ab-9a3f-6a2053129504 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:04.182498 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca473c6-fd44-4928-8a23-ef9b9f401502 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:04.199656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811cde0-4b8b-462d-9de0-f8e48db22203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:04.200759 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811cde0-4b8b-462d-9de0-f8e48db22203 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:04.201073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811cde0-4b8b-462d-9de0-f8e48db22203 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:04.201419 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811cde0-4b8b-462d-9de0-f8e48db22203 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:04.205213 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811cde0-4b8b-462d-9de0-f8e48db22203 None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:04.207362 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811cde0-4b8b-462d-9de0-f8e48db22203 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:04.209513 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 82/168] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:03 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:04.210117 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811cde0-4b8b-462d-9de0-f8e48db22203 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:04.269761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6feee-abc4-411c-b767-42acdf154d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:04.270441 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6feee-abc4-411c-b767-42acdf154d48 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:04.270730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6feee-abc4-411c-b767-42acdf154d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:04.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6feee-abc4-411c-b767-42acdf154d48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:04.271871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6feee-abc4-411c-b767-42acdf154d48 None admin] RBAC: Adding query filter params (role.id=6a41401a07c443f0b5acd2e189145727, scope.project.id=f14d8cdc0ed5452ca04a981594be5974, user.id=c0ff018c6850486aa01127bb8c1bd8b2) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:04.272184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6feee-abc4-411c-b767-42acdf154d48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:04.273670 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6feee-abc4-411c-b767-42acdf154d48 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:04.286669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 87/169] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:03 2022] GET /identity/v3/role_assignments?user.id=c0ff018c6850486aa01127bb8c1bd8b2&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=6a41401a07c443f0b5acd2e189145727 => generated 608 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:04.287619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6feee-abc4-411c-b767-42acdf154d48 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:04.318563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:04.432022 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811cde0-4b8b-462d-9de0-f8e48db22203 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:04.488970 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:04.518358 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e55d65-9b77-4e83-9d91-7f159215ee90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:04.519648 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55d65-9b77-4e83-9d91-7f159215ee90 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:04.519942 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55d65-9b77-4e83-9d91-7f159215ee90 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:04.520215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e55d65-9b77-4e83-9d91-7f159215ee90 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:04.521510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e55d65-9b77-4e83-9d91-7f159215ee90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:04.521992 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e55d65-9b77-4e83-9d91-7f159215ee90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:04.526304 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e55d65-9b77-4e83-9d91-7f159215ee90 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:04.542375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 83/170] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:04.543266 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e55d65-9b77-4e83-9d91-7f159215ee90 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:04.574569 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6feee-abc4-411c-b767-42acdf154d48 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:04.660873 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c6f2a-ae7f-4d8e-91b1-0055c2a8bfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:04.661553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c6f2a-ae7f-4d8e-91b1-0055c2a8bfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:04.661836 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c6f2a-ae7f-4d8e-91b1-0055c2a8bfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:04.662112 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c6f2a-ae7f-4d8e-91b1-0055c2a8bfd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:04.662877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c6f2a-ae7f-4d8e-91b1-0055c2a8bfd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:04.663186 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c6f2a-ae7f-4d8e-91b1-0055c2a8bfd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:04.664599 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c6f2a-ae7f-4d8e-91b1-0055c2a8bfd6 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:04.688114 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 88/171] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:04.688114 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c6f2a-ae7f-4d8e-91b1-0055c2a8bfd6 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:04.743812 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e55d65-9b77-4e83-9d91-7f159215ee90 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:04.820802 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4d8ef-329e-4545-9522-ac24b1ff2d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:04.821911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4d8ef-329e-4545-9522-ac24b1ff2d7e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:04.822408 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4d8ef-329e-4545-9522-ac24b1ff2d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:04.822890 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4d8ef-329e-4545-9522-ac24b1ff2d7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:04.824038 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4d8ef-329e-4545-9522-ac24b1ff2d7e None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=f14d8cdc0ed5452ca04a981594be5974, user.id=3f3a6a2da4254702bc7bd15361b6c9ae) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:04.824648 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4d8ef-329e-4545-9522-ac24b1ff2d7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:04.826404 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4d8ef-329e-4545-9522-ac24b1ff2d7e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:04.837773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 84/172] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:04 2022] GET /identity/v3/role_assignments?user.id=3f3a6a2da4254702bc7bd15361b6c9ae&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:04.839044 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4d8ef-329e-4545-9522-ac24b1ff2d7e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:04.863423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:04.894914 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c6f2a-ae7f-4d8e-91b1-0055c2a8bfd6 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:04.941011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e3759-c791-4363-9b29-4e86245d894b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:04.941814 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e3759-c791-4363-9b29-4e86245d894b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:04.942099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e3759-c791-4363-9b29-4e86245d894b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:04.942367 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e3759-c791-4363-9b29-4e86245d894b None admin] PATH_INFO: `/v3/users/c0ff018c6850486aa01127bb8c1bd8b2` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:04.943676 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e3759-c791-4363-9b29-4e86245d894b None admin] RBAC: Authorizing `identity:get_user(user_id=c0ff018c6850486aa01127bb8c1bd8b2)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:04.945186 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e3759-c791-4363-9b29-4e86245d894b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:04.946873 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 89/173] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:04 2022] GET /identity/v3/users/c0ff018c6850486aa01127bb8c1bd8b2 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:04.948173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611ea6f-6876-44ee-b257-35bfe0de95d4 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:04.948523 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611ea6f-6876-44ee-b257-35bfe0de95d4 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:04.948825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611ea6f-6876-44ee-b257-35bfe0de95d4 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:04.957024 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 90/174] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:04.961085 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611ea6f-6876-44ee-b257-35bfe0de95d4 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:04.961764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:05.047640 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4d8ef-329e-4545-9522-ac24b1ff2d7e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:05.119565 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c620aa2-050e-4cc1-aaec-dbcf145859c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:05.120892 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c620aa2-050e-4cc1-aaec-dbcf145859c2 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:05.121419 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c620aa2-050e-4cc1-aaec-dbcf145859c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:05.122005 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c620aa2-050e-4cc1-aaec-dbcf145859c2 None admin] PATH_INFO: `/v3/users/c0ff018c6850486aa01127bb8c1bd8b2` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:05.124375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611ea6f-6876-44ee-b257-35bfe0de95d4 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:05.125363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c620aa2-050e-4cc1-aaec-dbcf145859c2 None admin] RBAC: Authorizing `identity:get_user(user_id=c0ff018c6850486aa01127bb8c1bd8b2)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:05.127561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c620aa2-050e-4cc1-aaec-dbcf145859c2 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:05.129744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 85/175] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:04 2022] GET /identity/v3/users/c0ff018c6850486aa01127bb8c1bd8b2 => generated 275 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:05.130911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5b24b-6de6-4078-984b-e0dc5bc55d41 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:05.131275 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5b24b-6de6-4078-984b-e0dc5bc55d41 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:05.131630 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5b24b-6de6-4078-984b-e0dc5bc55d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:05.175133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-d9a5b24b-6de6-4078-984b-e0dc5bc55d41 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:05.176432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-d9a5b24b-6de6-4078-984b-e0dc5bc55d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:05.182300 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc18de-5b2d-429f-ba2c-1c8040cec11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:05.182978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc18de-5b2d-429f-ba2c-1c8040cec11a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:05.183701 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc18de-5b2d-429f-ba2c-1c8040cec11a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:05.183701 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc18de-5b2d-429f-ba2c-1c8040cec11a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:05.211929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:05.320859 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 86/176] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:05.321756 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5b24b-6de6-4078-984b-e0dc5bc55d41 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:05.331282 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc18de-5b2d-429f-ba2c-1c8040cec11a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:05.332806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc18de-5b2d-429f-ba2c-1c8040cec11a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:05.430802 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:05.542117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 91/177] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:05.543011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc18de-5b2d-429f-ba2c-1c8040cec11a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:05.554576 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5b24b-6de6-4078-984b-e0dc5bc55d41 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:05.600707 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:05.610438 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0e193-dc21-4a3c-b588-f8b27f14d892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:05.611647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0e193-dc21-4a3c-b588-f8b27f14d892 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:05.611929 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0e193-dc21-4a3c-b588-f8b27f14d892 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:05.612207 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0e193-dc21-4a3c-b588-f8b27f14d892 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:05.658616 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc18de-5b2d-429f-ba2c-1c8040cec11a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:05.698638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f1f20-c0a7-4fa7-8463-b60d9c9e3559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:05.699439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1f20-c0a7-4fa7-8463-b60d9c9e3559 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:05.699720 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1f20-c0a7-4fa7-8463-b60d9c9e3559 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:05.699989 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f1f20-c0a7-4fa7-8463-b60d9c9e3559 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:05.714898 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f1f20-c0a7-4fa7-8463-b60d9c9e3559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-918f1f20-c0a7-4fa7-8463-b60d9c9e3559 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:05.722991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:05.723888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 92/178] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:46:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:05.725189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f1f20-c0a7-4fa7-8463-b60d9c9e3559 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:05.742062 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0e193-dc21-4a3c-b588-f8b27f14d892 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:05.743735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0e193-dc21-4a3c-b588-f8b27f14d892 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:05.799851 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:05.844521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f1f20-c0a7-4fa7-8463-b60d9c9e3559 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:05.846304 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:05.862864 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 87/179] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:05.864267 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e87ee-b1bb-46d9-a669-0cf2cb0ccf7f None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:05.864577 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e87ee-b1bb-46d9-a669-0cf2cb0ccf7f None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:05.864847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e87ee-b1bb-46d9-a669-0cf2cb0ccf7f None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:05.866135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 88/180] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:05.870116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612553-414e-4180-8ec5-fa2611912ae0 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:05.870407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612553-414e-4180-8ec5-fa2611912ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:05.870686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a612553-414e-4180-8ec5-fa2611912ae0 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:05.872590 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 89/181] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:05.873287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a612553-414e-4180-8ec5-fa2611912ae0 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:05.894841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bb63d-c538-4c0f-9da7-b2ad5e7cbdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:05.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bb63d-c538-4c0f-9da7-b2ad5e7cbdec None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:05.896610 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bb63d-c538-4c0f-9da7-b2ad5e7cbdec None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:05.896610 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bb63d-c538-4c0f-9da7-b2ad5e7cbdec None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:05.897839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bb63d-c538-4c0f-9da7-b2ad5e7cbdec None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:05.899336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bb63d-c538-4c0f-9da7-b2ad5e7cbdec None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:05.901243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 93/182] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:05 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:05.901243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bb63d-c538-4c0f-9da7-b2ad5e7cbdec None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:05.985231 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a612553-414e-4180-8ec5-fa2611912ae0 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:06.017473 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bb63d-c538-4c0f-9da7-b2ad5e7cbdec None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:06.020211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb975d-1017-4443-997c-8bc196e52bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:06.020907 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb975d-1017-4443-997c-8bc196e52bee None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.021210 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb975d-1017-4443-997c-8bc196e52bee None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.021464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb975d-1017-4443-997c-8bc196e52bee None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.022212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb975d-1017-4443-997c-8bc196e52bee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:06.022515 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb975d-1017-4443-997c-8bc196e52bee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:06.023931 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb975d-1017-4443-997c-8bc196e52bee None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:06.035693 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 90/183] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:06.035693 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179d953-dd9f-4427-99e5-a475b0ead02c None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.035693 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179d953-dd9f-4427-99e5-a475b0ead02c None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.035693 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179d953-dd9f-4427-99e5-a475b0ead02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.052599 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-d179d953-dd9f-4427-99e5-a475b0ead02c None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:06.053961 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-d179d953-dd9f-4427-99e5-a475b0ead02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:06.056686 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e13f9ac-c091-416e-99bf-ff6fbb4a34d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:06.056833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:06.057558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13f9ac-c091-416e-99bf-ff6fbb4a34d0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.057839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13f9ac-c091-416e-99bf-ff6fbb4a34d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.058141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e13f9ac-c091-416e-99bf-ff6fbb4a34d0 None admin] PATH_INFO: `/v3/projects/5beb6dc9dc3f45d5919111c3e6c396e7` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.059353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13f9ac-c091-416e-99bf-ff6fbb4a34d0 None admin] RBAC: Authorizing `identity:get_project(project_id=5beb6dc9dc3f45d5919111c3e6c396e7)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:06.060838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e13f9ac-c091-416e-99bf-ff6fbb4a34d0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:06.067189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 94/184] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:05 2022] GET /identity/v3/projects/5beb6dc9dc3f45d5919111c3e6c396e7 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:06.067189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac24c5-5a7f-4e88-9b1c-5747bec72955 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.067189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac24c5-5a7f-4e88-9b1c-5747bec72955 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.067189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac24c5-5a7f-4e88-9b1c-5747bec72955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.081346 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-3aac24c5-5a7f-4e88-9b1c-5747bec72955 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:06.082624 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-3aac24c5-5a7f-4e88-9b1c-5747bec72955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:06.114102 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:06.129017 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 91/185] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:06.130099 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179d953-dd9f-4427-99e5-a475b0ead02c None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:06.166247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 95/186] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:06.167594 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8cb30-15c4-4798-adc9-5c105d7883df None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.167884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8cb30-15c4-4798-adc9-5c105d7883df None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.168164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8cb30-15c4-4798-adc9-5c105d7883df None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.169729 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 96/187] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:06.170181 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8cb30-15c4-4798-adc9-5c105d7883df None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:06.240152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179d953-dd9f-4427-99e5-a475b0ead02c None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:06.277392 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2373a6-6bb2-43d0-9e42-21a456ad52c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:06.278055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2373a6-6bb2-43d0-9e42-21a456ad52c9 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.278334 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2373a6-6bb2-43d0-9e42-21a456ad52c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.278610 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2373a6-6bb2-43d0-9e42-21a456ad52c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.279467 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2373a6-6bb2-43d0-9e42-21a456ad52c9 None admin] RBAC: Adding query filter params (role.id=cb116a5614cf4a48ad99840d95f96bb3, scope.project.id=f14d8cdc0ed5452ca04a981594be5974, user.id=c0ff018c6850486aa01127bb8c1bd8b2) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:06.279774 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2373a6-6bb2-43d0-9e42-21a456ad52c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:06.281394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2373a6-6bb2-43d0-9e42-21a456ad52c9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:06.282656 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8cb30-15c4-4798-adc9-5c105d7883df None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:06.288818 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 92/188] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:06 2022] GET /identity/v3/role_assignments?user.id=c0ff018c6850486aa01127bb8c1bd8b2&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=cb116a5614cf4a48ad99840d95f96bb3 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:06.292311 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2373a6-6bb2-43d0-9e42-21a456ad52c9 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:06.296448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:06.327755 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182a3d9-1ee7-44dc-baf1-6ab95c51aa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:06.328614 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182a3d9-1ee7-44dc-baf1-6ab95c51aa2f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.328910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182a3d9-1ee7-44dc-baf1-6ab95c51aa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.329168 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e182a3d9-1ee7-44dc-baf1-6ab95c51aa2f None admin] PATH_INFO: `/v3/users/4c6b548fdba34d1280f0a5a80a1e7684` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.346955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182a3d9-1ee7-44dc-baf1-6ab95c51aa2f None admin] RBAC: Authorizing `identity:get_user(user_id=4c6b548fdba34d1280f0a5a80a1e7684)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:06.348590 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e182a3d9-1ee7-44dc-baf1-6ab95c51aa2f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:06.350465 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 97/189] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:06 2022] GET /identity/v3/users/4c6b548fdba34d1280f0a5a80a1e7684 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:06.351284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182a3d9-1ee7-44dc-baf1-6ab95c51aa2f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:06.385841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:06.427288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2373a6-6bb2-43d0-9e42-21a456ad52c9 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:06.470213 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5a352-ed69-468d-b72f-068d08493d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:06.471391 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5a352-ed69-468d-b72f-068d08493d53 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.471677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5a352-ed69-468d-b72f-068d08493d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.471953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5a352-ed69-468d-b72f-068d08493d53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.472575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e182a3d9-1ee7-44dc-baf1-6ab95c51aa2f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:06.472849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5a352-ed69-468d-b72f-068d08493d53 None admin] RBAC: Adding query filter params (role.id=6a41401a07c443f0b5acd2e189145727, scope.project.id=5beb6dc9dc3f45d5919111c3e6c396e7, user.id=c0ff018c6850486aa01127bb8c1bd8b2) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:06.473280 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5a352-ed69-468d-b72f-068d08493d53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:06.474786 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5a352-ed69-468d-b72f-068d08493d53 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:06.488542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 93/190] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:06 2022] GET /identity/v3/role_assignments?user.id=c0ff018c6850486aa01127bb8c1bd8b2&scope.project.id=5beb6dc9dc3f45d5919111c3e6c396e7&role.id=6a41401a07c443f0b5acd2e189145727 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:06.488542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4bfa9-2ab0-42a4-9c0c-521071e86b42 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.488542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4bfa9-2ab0-42a4-9c0c-521071e86b42 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.488542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4bfa9-2ab0-42a4-9c0c-521071e86b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.503139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-12f4bfa9-2ab0-42a4-9c0c-521071e86b42 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:06.504406 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-12f4bfa9-2ab0-42a4-9c0c-521071e86b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:06.544436 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e3239-55da-4ce4-9148-34d48d2d0682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:06.545251 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3239-55da-4ce4-9148-34d48d2d0682 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.545535 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3239-55da-4ce4-9148-34d48d2d0682 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.545809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e3239-55da-4ce4-9148-34d48d2d0682 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.547026 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e3239-55da-4ce4-9148-34d48d2d0682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:06.549422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e3239-55da-4ce4-9148-34d48d2d0682 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:06.550304 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 98/191] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:06.554158 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e3239-55da-4ce4-9148-34d48d2d0682 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:06.600319 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:06.613413 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 94/192] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:06.614339 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4bfa9-2ab0-42a4-9c0c-521071e86b42 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:06.700794 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:06.712573 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e3239-55da-4ce4-9148-34d48d2d0682 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:06.740123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4bfa9-2ab0-42a4-9c0c-521071e86b42 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:06.760413 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04460e04-424f-495c-b4dd-e6e9bda48113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:06.761196 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04460e04-424f-495c-b4dd-e6e9bda48113 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.761480 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04460e04-424f-495c-b4dd-e6e9bda48113 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.761751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04460e04-424f-495c-b4dd-e6e9bda48113 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.773603 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04460e04-424f-495c-b4dd-e6e9bda48113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:06.781635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad74cef-48e9-48c4-8f8d-bc4c33905d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-04460e04-424f-495c-b4dd-e6e9bda48113 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:06.782053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:06.782956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:06.783764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:06.784389 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad74cef-48e9-48c4-8f8d-bc4c33905d58 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.784389 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad74cef-48e9-48c4-8f8d-bc4c33905d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.784389 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad74cef-48e9-48c4-8f8d-bc4c33905d58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.784559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 99/193] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:46:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:06.784559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04460e04-424f-495c-b4dd-e6e9bda48113 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:06.854130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:06.857589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad74cef-48e9-48c4-8f8d-bc4c33905d58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:06.859159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad74cef-48e9-48c4-8f8d-bc4c33905d58 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:06.893833 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04460e04-424f-495c-b4dd-e6e9bda48113 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:06.927348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1bb44-b169-4f79-8fb9-81e34a1ca5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:06.928281 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1bb44-b169-4f79-8fb9-81e34a1ca5f9 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:06.928720 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1bb44-b169-4f79-8fb9-81e34a1ca5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:06.928989 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1bb44-b169-4f79-8fb9-81e34a1ca5f9 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:06.929698 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1bb44-b169-4f79-8fb9-81e34a1ca5f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:06.931131 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1bb44-b169-4f79-8fb9-81e34a1ca5f9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:06.959558 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 95/194] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:06.960571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad74cef-48e9-48c4-8f8d-bc4c33905d58 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.030849 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 100/195] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:46:06 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:07.031729 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1bb44-b169-4f79-8fb9-81e34a1ca5f9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.046138 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:07.065002 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:07.072437 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad74cef-48e9-48c4-8f8d-bc4c33905d58 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:07.118141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32adbd-b5d2-44da-9c9c-b7af2e85f0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:07.118141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32adbd-b5d2-44da-9c9c-b7af2e85f0ac None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.118141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32adbd-b5d2-44da-9c9c-b7af2e85f0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.118141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32adbd-b5d2-44da-9c9c-b7af2e85f0ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.135106 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb32adbd-b5d2-44da-9c9c-b7af2e85f0ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-eb32adbd-b5d2-44da-9c9c-b7af2e85f0ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:07.143773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:07.144858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:07.145700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 96/196] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:07.146343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32adbd-b5d2-44da-9c9c-b7af2e85f0ac None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.163045 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1bb44-b169-4f79-8fb9-81e34a1ca5f9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:07.208131 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fb26c-b28a-472d-a2f8-cc48db9fbd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:07.208877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fb26c-b28a-472d-a2f8-cc48db9fbd6f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.209747 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fb26c-b28a-472d-a2f8-cc48db9fbd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.210178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fb26c-b28a-472d-a2f8-cc48db9fbd6f None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.211148 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fb26c-b28a-472d-a2f8-cc48db9fbd6f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:07.211453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fb26c-b28a-472d-a2f8-cc48db9fbd6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:07.213074 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fb26c-b28a-472d-a2f8-cc48db9fbd6f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:07.231393 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 101/197] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:07.231393 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fb26c-b28a-472d-a2f8-cc48db9fbd6f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.273644 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32adbd-b5d2-44da-9c9c-b7af2e85f0ac None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:07.320242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:07.324955 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe9ff8-83a6-4193-ad35-a35083c1401f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:07.325992 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe9ff8-83a6-4193-ad35-a35083c1401f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.326525 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe9ff8-83a6-4193-ad35-a35083c1401f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.326907 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe9ff8-83a6-4193-ad35-a35083c1401f None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.329231 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe9ff8-83a6-4193-ad35-a35083c1401f None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:07.330757 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe9ff8-83a6-4193-ad35-a35083c1401f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:07.332658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 97/198] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:07.333331 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe9ff8-83a6-4193-ad35-a35083c1401f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.366985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fb26c-b28a-472d-a2f8-cc48db9fbd6f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:07.380203 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:07.409829 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa83f5-9aa0-42d8-9002-28af0dfd359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:07.410697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa83f5-9aa0-42d8-9002-28af0dfd359a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.410859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa83f5-9aa0-42d8-9002-28af0dfd359a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.411100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa83f5-9aa0-42d8-9002-28af0dfd359a None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.411918 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa83f5-9aa0-42d8-9002-28af0dfd359a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:07.412215 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa83f5-9aa0-42d8-9002-28af0dfd359a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:07.413757 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa83f5-9aa0-42d8-9002-28af0dfd359a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:07.424656 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 102/199] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:07.425777 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa83f5-9aa0-42d8-9002-28af0dfd359a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.491054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe9ff8-83a6-4193-ad35-a35083c1401f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:07.554927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e767253-d591-4de7-a997-6d8ada323da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:07.555912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e767253-d591-4de7-a997-6d8ada323da0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.556210 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e767253-d591-4de7-a997-6d8ada323da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.556606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e767253-d591-4de7-a997-6d8ada323da0 None admin] PATH_INFO: `/v3/users/4c6b548fdba34d1280f0a5a80a1e7684` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.557973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e767253-d591-4de7-a997-6d8ada323da0 None admin] RBAC: Authorizing `identity:get_user(user_id=4c6b548fdba34d1280f0a5a80a1e7684)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:07.559497 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e767253-d591-4de7-a997-6d8ada323da0 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:07.561799 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 98/200] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/users/4c6b548fdba34d1280f0a5a80a1e7684 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:07.562610 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e767253-d591-4de7-a997-6d8ada323da0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.609954 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa83f5-9aa0-42d8-9002-28af0dfd359a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:07.655029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36e290-3fcd-46b1-a98b-e51882d0b622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:07.655634 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36e290-3fcd-46b1-a98b-e51882d0b622 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.655918 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36e290-3fcd-46b1-a98b-e51882d0b622 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.656197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36e290-3fcd-46b1-a98b-e51882d0b622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.657111 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36e290-3fcd-46b1-a98b-e51882d0b622 None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61, user.id=4c6b548fdba34d1280f0a5a80a1e7684) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:07.657430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36e290-3fcd-46b1-a98b-e51882d0b622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:07.659520 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36e290-3fcd-46b1-a98b-e51882d0b622 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:07.661689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:07.672371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 103/201] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/role_assignments?user.id=4c6b548fdba34d1280f0a5a80a1e7684&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:07.672371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36e290-3fcd-46b1-a98b-e51882d0b622 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.719607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:07.734207 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e767253-d591-4de7-a997-6d8ada323da0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:07.792122 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e0896-3706-4842-81b9-31cbe97da20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:07.798740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e0896-3706-4842-81b9-31cbe97da20f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.799372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e0896-3706-4842-81b9-31cbe97da20f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.799804 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e0896-3706-4842-81b9-31cbe97da20f None admin] PATH_INFO: `/v3/users/61a6d10d178d4987839e337f1cab52cc` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:07.801425 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e0896-3706-4842-81b9-31cbe97da20f None admin] RBAC: Authorizing `identity:get_user(user_id=61a6d10d178d4987839e337f1cab52cc)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:07.803347 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e0896-3706-4842-81b9-31cbe97da20f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:07.805366 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 99/202] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/users/61a6d10d178d4987839e337f1cab52cc => generated 289 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:07.811489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e0896-3706-4842-81b9-31cbe97da20f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:07.867789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36e290-3fcd-46b1-a98b-e51882d0b622 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:07.941903 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a09d4-de0e-4fea-8d55-d65a61bb8d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:07.942485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a09d4-de0e-4fea-8d55-d65a61bb8d2b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:07.942767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a09d4-de0e-4fea-8d55-d65a61bb8d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:07.943039 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a09d4-de0e-4fea-8d55-d65a61bb8d2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:08.048867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:08.054247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a09d4-de0e-4fea-8d55-d65a61bb8d2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:08.055784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a09d4-de0e-4fea-8d55-d65a61bb8d2b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:08.078945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e0896-3706-4842-81b9-31cbe97da20f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:08.136067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256f8c6-de44-4458-aaa5-9464301c93c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:08.136757 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f8c6-de44-4458-aaa5-9464301c93c7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:08.137044 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f8c6-de44-4458-aaa5-9464301c93c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:08.137321 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f8c6-de44-4458-aaa5-9464301c93c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:08.175737 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:08.245244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256f8c6-de44-4458-aaa5-9464301c93c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:08.246651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256f8c6-de44-4458-aaa5-9464301c93c7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:08.281960 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 104/203] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:08.290812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a09d4-de0e-4fea-8d55-d65a61bb8d2b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:08.447672 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 100/204] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:46:08.454693 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256f8c6-de44-4458-aaa5-9464301c93c7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:08.464861 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:08.471760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a09d4-de0e-4fea-8d55-d65a61bb8d2b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:08.554321 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce52f4-3955-468c-97c2-ced1f27e7280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:08.555348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce52f4-3955-468c-97c2-ced1f27e7280 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:08.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce52f4-3955-468c-97c2-ced1f27e7280 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:08.555946 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce52f4-3955-468c-97c2-ced1f27e7280 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:08.558489 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce52f4-3955-468c-97c2-ced1f27e7280 None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:08.560025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce52f4-3955-468c-97c2-ced1f27e7280 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:08.562144 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 105/205] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:08 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:08.567253 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:08.568591 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce52f4-3955-468c-97c2-ced1f27e7280 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:08.635227 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256f8c6-de44-4458-aaa5-9464301c93c7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:08.683938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4146f-9fb2-453f-b76b-1beae80ca6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:08.684845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4146f-9fb2-453f-b76b-1beae80ca6ab None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:08.685130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4146f-9fb2-453f-b76b-1beae80ca6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:08.685407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4146f-9fb2-453f-b76b-1beae80ca6ab None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:08.687704 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4146f-9fb2-453f-b76b-1beae80ca6ab None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:08.689243 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d4146f-9fb2-453f-b76b-1beae80ca6ab None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:08.692103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 101/206] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:08 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:08.696916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4146f-9fb2-453f-b76b-1beae80ca6ab None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:08.763023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce52f4-3955-468c-97c2-ced1f27e7280 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:08.809621 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f940e6-b951-4e16-893a-704c33a2fd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:08.810157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f940e6-b951-4e16-893a-704c33a2fd73 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:08.810388 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f940e6-b951-4e16-893a-704c33a2fd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:08.810667 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f940e6-b951-4e16-893a-704c33a2fd73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:08.811511 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f940e6-b951-4e16-893a-704c33a2fd73 None admin] RBAC: Adding query filter params (role.id=cb116a5614cf4a48ad99840d95f96bb3, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61, user.id=4c6b548fdba34d1280f0a5a80a1e7684) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:08.811822 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f940e6-b951-4e16-893a-704c33a2fd73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:08.819353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f940e6-b951-4e16-893a-704c33a2fd73 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:08.819353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 106/207] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:08 2022] GET /identity/v3/role_assignments?user.id=4c6b548fdba34d1280f0a5a80a1e7684&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=cb116a5614cf4a48ad99840d95f96bb3 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:08.898510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4146f-9fb2-453f-b76b-1beae80ca6ab None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:08.944367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:08.951079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372d2f6-dc5f-4e08-b58b-dac7e531d615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:08.952016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372d2f6-dc5f-4e08-b58b-dac7e531d615 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:08.952477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372d2f6-dc5f-4e08-b58b-dac7e531d615 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:08.952893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372d2f6-dc5f-4e08-b58b-dac7e531d615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:08.955340 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372d2f6-dc5f-4e08-b58b-dac7e531d615 None admin] RBAC: Adding query filter params (role.id=bd9a605a759046cf9bd1a8e9bb20d9d4, scope.project.id=f14d8cdc0ed5452ca04a981594be5974, user.id=61a6d10d178d4987839e337f1cab52cc) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:08.955340 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372d2f6-dc5f-4e08-b58b-dac7e531d615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:08.956608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372d2f6-dc5f-4e08-b58b-dac7e531d615 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:08.968425 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 102/208] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:08 2022] GET /identity/v3/role_assignments?user.id=61a6d10d178d4987839e337f1cab52cc&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=bd9a605a759046cf9bd1a8e9bb20d9d4 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:09.743695 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cfea2-e9ee-404e-9cc2-073ad43ef641 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:09.744203 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cfea2-e9ee-404e-9cc2-073ad43ef641 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:09.744203 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cfea2-e9ee-404e-9cc2-073ad43ef641 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:09.745909 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 107/209] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:09.751535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e49c5a-d3ac-4e4a-bbb4-74df11529d84 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:09.752001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e49c5a-d3ac-4e4a-bbb4-74df11529d84 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:09.752099 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e49c5a-d3ac-4e4a-bbb4-74df11529d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:09.771131 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-12e49c5a-d3ac-4e4a-bbb4-74df11529d84 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:09.772393 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-12e49c5a-d3ac-4e4a-bbb4-74df11529d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:09.849321 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 103/210] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:09.879923 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cfea2-e9ee-404e-9cc2-073ad43ef641 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:09.911317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:09.978358 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e12869-7893-4c6c-8d55-d5847e958a3e None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:09.979005 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e12869-7893-4c6c-8d55-d5847e958a3e None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:09.979487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e12869-7893-4c6c-8d55-d5847e958a3e None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:09.981503 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 104/211] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:09.987338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40d31f-94ea-4d3c-ae19-1ecac0444d59 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:09.987843 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40d31f-94ea-4d3c-ae19-1ecac0444d59 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:09.988411 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40d31f-94ea-4d3c-ae19-1ecac0444d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:09.997114 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cfea2-e9ee-404e-9cc2-073ad43ef641 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:10.007262 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-1e40d31f-94ea-4d3c-ae19-1ecac0444d59 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:10.008884 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-1e40d31f-94ea-4d3c-ae19-1ecac0444d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:10.034829 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e550fc4-1c7d-4aa6-8298-e51dade3451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:10.036122 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e550fc4-1c7d-4aa6-8298-e51dade3451d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:10.036639 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e550fc4-1c7d-4aa6-8298-e51dade3451d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:10.037138 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e550fc4-1c7d-4aa6-8298-e51dade3451d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:10.048509 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e550fc4-1c7d-4aa6-8298-e51dade3451d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-4e550fc4-1c7d-4aa6-8298-e51dade3451d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:10.057247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:10.058175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:10.058979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:10.061856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 108/212] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:10.067528 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e550fc4-1c7d-4aa6-8298-e51dade3451d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:10.097344 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 105/213] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:10.121848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40d31f-94ea-4d3c-ae19-1ecac0444d59 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:10.186989 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:10.192064 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:10.205207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e550fc4-1c7d-4aa6-8298-e51dade3451d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:10.241854 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296463d1-4ba9-4ccb-a9cf-e9c4dc0cd860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:10.242524 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296463d1-4ba9-4ccb-a9cf-e9c4dc0cd860 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:10.242809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296463d1-4ba9-4ccb-a9cf-e9c4dc0cd860 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:10.243085 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296463d1-4ba9-4ccb-a9cf-e9c4dc0cd860 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:10.243859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296463d1-4ba9-4ccb-a9cf-e9c4dc0cd860 None admin] RBAC: Adding query filter params (name=member) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:10.244128 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296463d1-4ba9-4ccb-a9cf-e9c4dc0cd860 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:10.246938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40d31f-94ea-4d3c-ae19-1ecac0444d59 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:10.247636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296463d1-4ba9-4ccb-a9cf-e9c4dc0cd860 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:10.257326 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 109/214] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:10.262633 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296463d1-4ba9-4ccb-a9cf-e9c4dc0cd860 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:10.292511 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec72bc-4bf5-40b3-804b-2fc4ec278ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:10.294198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec72bc-4bf5-40b3-804b-2fc4ec278ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:10.294640 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec72bc-4bf5-40b3-804b-2fc4ec278ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:10.295215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec72bc-4bf5-40b3-804b-2fc4ec278ea0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:10.297210 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec72bc-4bf5-40b3-804b-2fc4ec278ea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:10.299042 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec72bc-4bf5-40b3-804b-2fc4ec278ea0 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:10.302034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 106/215] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:10.308383 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec72bc-4bf5-40b3-804b-2fc4ec278ea0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:10.375828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296463d1-4ba9-4ccb-a9cf-e9c4dc0cd860 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:10.412866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd7d39-1f9d-4cbf-89ec-f954f8477aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:10.413747 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd7d39-1f9d-4cbf-89ec-f954f8477aff None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:10.414033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd7d39-1f9d-4cbf-89ec-f954f8477aff None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:10.414308 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd7d39-1f9d-4cbf-89ec-f954f8477aff None admin] PATH_INFO: `/v3/users/8e7b764c1fb44e628cefe267e29d65fb` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:10.424460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec72bc-4bf5-40b3-804b-2fc4ec278ea0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:10.429062 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd7d39-1f9d-4cbf-89ec-f954f8477aff None admin] RBAC: Authorizing `identity:get_user(user_id=8e7b764c1fb44e628cefe267e29d65fb)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:10.430523 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd7d39-1f9d-4cbf-89ec-f954f8477aff None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:10.432687 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 110/216] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:10 2022] GET /identity/v3/users/8e7b764c1fb44e628cefe267e29d65fb => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:10.434077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a9e89-afff-4692-968c-33a47762a39d None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:10.434367 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a9e89-afff-4692-968c-33a47762a39d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:10.434643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69a9e89-afff-4692-968c-33a47762a39d None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:10.436266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 111/217] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:10.441269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a9e89-afff-4692-968c-33a47762a39d None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:10.453004 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:10.455953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:10.467373 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822aeb3c-8c1a-4e61-82e1-cd5afc55780d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:10.468046 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822aeb3c-8c1a-4e61-82e1-cd5afc55780d None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:10.468325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822aeb3c-8c1a-4e61-82e1-cd5afc55780d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:10.468646 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822aeb3c-8c1a-4e61-82e1-cd5afc55780d None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:10.469375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822aeb3c-8c1a-4e61-82e1-cd5afc55780d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:10.470862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822aeb3c-8c1a-4e61-82e1-cd5afc55780d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:10.546647 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69a9e89-afff-4692-968c-33a47762a39d None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:10.579871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 107/218] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:46:10 2022] POST /identity/v3/users => generated 297 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:46:10.588568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31fa7a-92c4-4210-99c6-3097b346e52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:10.588568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31fa7a-92c4-4210-99c6-3097b346e52b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:10.588568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31fa7a-92c4-4210-99c6-3097b346e52b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:10.588568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c31fa7a-92c4-4210-99c6-3097b346e52b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:10.590767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f3d2-b984-4b5d-a3fb-9cddf41a199b None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:10.591059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f3d2-b984-4b5d-a3fb-9cddf41a199b None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:10.591339 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f3d2-b984-4b5d-a3fb-9cddf41a199b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:10.614940 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-1594f3d2-b984-4b5d-a3fb-9cddf41a199b None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:10.616184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-1594f3d2-b984-4b5d-a3fb-9cddf41a199b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:10.687047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31fa7a-92c4-4210-99c6-3097b346e52b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:10.690147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c31fa7a-92c4-4210-99c6-3097b346e52b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:10.708055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:10.717506 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 108/219] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:10.739832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f3d2-b984-4b5d-a3fb-9cddf41a199b None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:10.824990 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 112/220] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:10.833206 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31fa7a-92c4-4210-99c6-3097b346e52b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:10.864325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f3d2-b984-4b5d-a3fb-9cddf41a199b None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:10.876925 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:10.904270 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c00804-fec7-4379-883b-b8a141647af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:10.906700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c00804-fec7-4379-883b-b8a141647af2 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:10.906700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c00804-fec7-4379-883b-b8a141647af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:10.906700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c00804-fec7-4379-883b-b8a141647af2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:10.922313 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c00804-fec7-4379-883b-b8a141647af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-52c00804-fec7-4379-883b-b8a141647af2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:10.934645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:10.940208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:10.941090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 109/221] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:46:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:10.942261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c00804-fec7-4379-883b-b8a141647af2 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:10.973269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c31fa7a-92c4-4210-99c6-3097b346e52b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:10.988148 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:11.027063 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8824794-e44e-495d-b603-437e8f06e64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:11.028091 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8824794-e44e-495d-b603-437e8f06e64e None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:11.028505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8824794-e44e-495d-b603-437e8f06e64e None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:11.029035 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8824794-e44e-495d-b603-437e8f06e64e None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:11.030921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8824794-e44e-495d-b603-437e8f06e64e None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:11.032772 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8824794-e44e-495d-b603-437e8f06e64e None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:11.037015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 113/222] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:10 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:11.045486 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8824794-e44e-495d-b603-437e8f06e64e None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:11.074955 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c00804-fec7-4379-883b-b8a141647af2 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:11.125261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ee53b-af48-448c-aee9-f5634eb191a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:11.126166 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ee53b-af48-448c-aee9-f5634eb191a3 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:11.126972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ee53b-af48-448c-aee9-f5634eb191a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:11.127249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ee53b-af48-448c-aee9-f5634eb191a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:11.128013 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ee53b-af48-448c-aee9-f5634eb191a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:11.128321 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ee53b-af48-448c-aee9-f5634eb191a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:11.130480 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ee53b-af48-448c-aee9-f5634eb191a3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:11.141043 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 110/223] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:11.146706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ee53b-af48-448c-aee9-f5634eb191a3 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:11.172651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8824794-e44e-495d-b603-437e8f06e64e None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:11.191522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:11.211303 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d02fa-8f94-4a6f-b91e-a828ebb17945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:11.212215 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d02fa-8f94-4a6f-b91e-a828ebb17945 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:11.212983 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d02fa-8f94-4a6f-b91e-a828ebb17945 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:11.213396 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d02fa-8f94-4a6f-b91e-a828ebb17945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:11.214379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d02fa-8f94-4a6f-b91e-a828ebb17945 None admin] RBAC: Adding query filter params (role.id=6a41401a07c443f0b5acd2e189145727, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61, user.id=8e7b764c1fb44e628cefe267e29d65fb) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:11.214854 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d02fa-8f94-4a6f-b91e-a828ebb17945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:11.216457 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d02fa-8f94-4a6f-b91e-a828ebb17945 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:11.223674 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 114/224] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:11 2022] GET /identity/v3/role_assignments?user.id=8e7b764c1fb44e628cefe267e29d65fb&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=6a41401a07c443f0b5acd2e189145727 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:11.275603 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ee53b-af48-448c-aee9-f5634eb191a3 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:11.317703 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f11045-ba66-403a-8f93-d2c2792528ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:11.318599 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11045-ba66-403a-8f93-d2c2792528ed None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:11.318883 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11045-ba66-403a-8f93-d2c2792528ed None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:11.319154 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f11045-ba66-403a-8f93-d2c2792528ed None admin] PATH_INFO: `/v3/users/4c6b548fdba34d1280f0a5a80a1e7684` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:11.320478 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f11045-ba66-403a-8f93-d2c2792528ed None admin] RBAC: Authorizing `identity:get_user(user_id=4c6b548fdba34d1280f0a5a80a1e7684)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:11.321998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f11045-ba66-403a-8f93-d2c2792528ed None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:11.323907 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 111/225] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:11 2022] GET /identity/v3/users/4c6b548fdba34d1280f0a5a80a1e7684 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:11.329376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d02fa-8f94-4a6f-b91e-a828ebb17945 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:11.376281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2765a-c9e4-471e-9614-128d037f11cd None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:11.376614 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2765a-c9e4-471e-9614-128d037f11cd None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:11.376772 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2765a-c9e4-471e-9614-128d037f11cd None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:11.378178 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 112/226] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:11.384247 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac228629-6e40-4592-a363-915a48f4e8ea None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:11.384578 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac228629-6e40-4592-a363-915a48f4e8ea None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:11.384826 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac228629-6e40-4592-a363-915a48f4e8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:11.387251 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:11.418537 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-ac228629-6e40-4592-a363-915a48f4e8ea None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:11.419786 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-ac228629-6e40-4592-a363-915a48f4e8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:11.458771 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d02fa-8f94-4a6f-b91e-a828ebb17945 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:11.524451 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbda2f-56de-463f-b6ae-a3379a99caa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:11.525462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbda2f-56de-463f-b6ae-a3379a99caa0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:11.525716 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbda2f-56de-463f-b6ae-a3379a99caa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:11.526021 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbda2f-56de-463f-b6ae-a3379a99caa0 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:11.527284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbda2f-56de-463f-b6ae-a3379a99caa0 None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:11.527995 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 113/227] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:11.532789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccbda2f-56de-463f-b6ae-a3379a99caa0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:11.532789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 115/228] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:11 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:11.536442 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac228629-6e40-4592-a363-915a48f4e8ea None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:11.567564 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbda2f-56de-463f-b6ae-a3379a99caa0 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:11.572989 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:11.654407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac228629-6e40-4592-a363-915a48f4e8ea None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:11.693417 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fcc6e-9150-4710-8e37-8d7b263a0532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:11.694657 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fcc6e-9150-4710-8e37-8d7b263a0532 None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:11.695153 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fcc6e-9150-4710-8e37-8d7b263a0532 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:11.695617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fcc6e-9150-4710-8e37-8d7b263a0532 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/users/4c6b548fdba34d1280f0a5a80a1e7684/roles/815bd48c3df0405babfd5b53430fd3ea` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:11.697863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fcc6e-9150-4710-8e37-8d7b263a0532 None admin] RBAC: Authorizing `identity:create_grant(project_id=03030ff4e6b54acfb48d93cd4178fd61, user_id=4c6b548fdba34d1280f0a5a80a1e7684, role_id=815bd48c3df0405babfd5b53430fd3ea)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:11.699576 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fcc6e-9150-4710-8e37-8d7b263a0532 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:11.726503 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbda2f-56de-463f-b6ae-a3379a99caa0 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:11.737941 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:11.762090 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4898a1-c859-4139-a4b9-cee6c97bdb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:11.762908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4898a1-c859-4139-a4b9-cee6c97bdb53 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:11.763195 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4898a1-c859-4139-a4b9-cee6c97bdb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:11.763468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4898a1-c859-4139-a4b9-cee6c97bdb53 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:11.775372 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4898a1-c859-4139-a4b9-cee6c97bdb53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-3f4898a1-c859-4139-a4b9-cee6c97bdb53 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:11.783678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:11.784585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:11.785357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 116/229] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:46:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:11.799969 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4898a1-c859-4139-a4b9-cee6c97bdb53 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:11.804274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 114/230] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:46:11 2022] PUT /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/users/4c6b548fdba34d1280f0a5a80a1e7684/roles/815bd48c3df0405babfd5b53430fd3ea => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:11.925470 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4898a1-c859-4139-a4b9-cee6c97bdb53 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:11.962186 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9291605-5ce9-48ba-9786-bff471f499e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:11.962863 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9291605-5ce9-48ba-9786-bff471f499e1 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:11.963136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9291605-5ce9-48ba-9786-bff471f499e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:11.963413 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9291605-5ce9-48ba-9786-bff471f499e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:11.964173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9291605-5ce9-48ba-9786-bff471f499e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:11.964543 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9291605-5ce9-48ba-9786-bff471f499e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:11.965943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9291605-5ce9-48ba-9786-bff471f499e1 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:11.975931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 117/231] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:11.985580 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fcc6e-9150-4710-8e37-8d7b263a0532 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:12.092398 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:12.112513 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fcc6e-9150-4710-8e37-8d7b263a0532 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:12.164913 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce5e3d-6ecf-4ab1-8621-d0fe71368453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:12.165801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce5e3d-6ecf-4ab1-8621-d0fe71368453 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:12.166141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce5e3d-6ecf-4ab1-8621-d0fe71368453 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:12.167357 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce5e3d-6ecf-4ab1-8621-d0fe71368453 None admin] PATH_INFO: `/v3/users/4c6b548fdba34d1280f0a5a80a1e7684` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:12.168936 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce5e3d-6ecf-4ab1-8621-d0fe71368453 None admin] RBAC: Authorizing `identity:get_user(user_id=4c6b548fdba34d1280f0a5a80a1e7684)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:12.170430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce5e3d-6ecf-4ab1-8621-d0fe71368453 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:12.172441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 115/232] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:11 2022] GET /identity/v3/users/4c6b548fdba34d1280f0a5a80a1e7684 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:12.178465 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9291605-5ce9-48ba-9786-bff471f499e1 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:12.220625 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:12.295720 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9291605-5ce9-48ba-9786-bff471f499e1 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:12.334302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b15a43-eb45-4f2b-abf3-f56246506df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:12.335179 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b15a43-eb45-4f2b-abf3-f56246506df4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:12.338132 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b15a43-eb45-4f2b-abf3-f56246506df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:12.338132 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b15a43-eb45-4f2b-abf3-f56246506df4 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:12.345461 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b15a43-eb45-4f2b-abf3-f56246506df4 None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:12.345461 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b15a43-eb45-4f2b-abf3-f56246506df4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:12.346923 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 118/233] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:12 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:12.353378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce5e3d-6ecf-4ab1-8621-d0fe71368453 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:12.527111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce5e3d-6ecf-4ab1-8621-d0fe71368453 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:12.575674 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7bead6-3573-4d83-aa02-bed9edbe1c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:12.576603 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bead6-3573-4d83-aa02-bed9edbe1c4d None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:12.576964 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bead6-3573-4d83-aa02-bed9edbe1c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:12.577247 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7bead6-3573-4d83-aa02-bed9edbe1c4d None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/users/4c6b548fdba34d1280f0a5a80a1e7684/roles/cb116a5614cf4a48ad99840d95f96bb3` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:12.579568 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7bead6-3573-4d83-aa02-bed9edbe1c4d None admin] RBAC: Authorizing `identity:create_grant(project_id=03030ff4e6b54acfb48d93cd4178fd61, user_id=4c6b548fdba34d1280f0a5a80a1e7684, role_id=cb116a5614cf4a48ad99840d95f96bb3)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:12.581085 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7bead6-3573-4d83-aa02-bed9edbe1c4d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:12.687919 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 116/234] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:46:12 2022] PUT /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/users/4c6b548fdba34d1280f0a5a80a1e7684/roles/cb116a5614cf4a48ad99840d95f96bb3 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:13.835977 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25872358-3612-4b9c-be9f-6113fb0e6f8a None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:13.836533 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25872358-3612-4b9c-be9f-6113fb0e6f8a None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:13.836533 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25872358-3612-4b9c-be9f-6113fb0e6f8a None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:13.838266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 119/235] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:13.844410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08e9c0-d598-4a24-8867-3f01493dde76 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:13.844926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08e9c0-d598-4a24-8867-3f01493dde76 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:13.845000 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08e9c0-d598-4a24-8867-3f01493dde76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:13.869372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-5d08e9c0-d598-4a24-8867-3f01493dde76 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:13.870597 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-5d08e9c0-d598-4a24-8867-3f01493dde76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:13.883340 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:13.889996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b3b11-e6df-4b94-9ac3-79fdf16df998 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:13.890285 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b3b11-e6df-4b94-9ac3-79fdf16df998 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:13.890567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b3b11-e6df-4b94-9ac3-79fdf16df998 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:13.892044 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 120/236] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:13.897491 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fc8fd-4164-4532-86d7-35d5367f7ed9 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:13.897786 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fc8fd-4164-4532-86d7-35d5367f7ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:13.898069 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fc8fd-4164-4532-86d7-35d5367f7ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:13.928298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-e40fc8fd-4164-4532-86d7-35d5367f7ed9 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:13.929552 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-e40fc8fd-4164-4532-86d7-35d5367f7ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:13.989734 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 117/237] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:14.027671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08e9c0-d598-4a24-8867-3f01493dde76 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:14.055342 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 121/238] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:14.078736 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fc8fd-4164-4532-86d7-35d5367f7ed9 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:14.112218 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:14.167077 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08e9c0-d598-4a24-8867-3f01493dde76 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:14.193184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fc8fd-4164-4532-86d7-35d5367f7ed9 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:14.220403 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38807b88-3488-4894-abbb-960a96149e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:14.221496 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38807b88-3488-4894-abbb-960a96149e0a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:14.221784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38807b88-3488-4894-abbb-960a96149e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:14.222063 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38807b88-3488-4894-abbb-960a96149e0a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:14.231845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526f932-26fb-49e2-ab03-20ce9eb2f78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:14.232888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526f932-26fb-49e2-ab03-20ce9eb2f78a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:14.233385 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526f932-26fb-49e2-ab03-20ce9eb2f78a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:14.233757 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526f932-26fb-49e2-ab03-20ce9eb2f78a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:14.234931 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38807b88-3488-4894-abbb-960a96149e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:14.235149 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526f932-26fb-49e2-ab03-20ce9eb2f78a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:14.236757 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526f932-26fb-49e2-ab03-20ce9eb2f78a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:14.238511 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 122/239] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:14.239819 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b13619-ed58-41ae-8d30-9a718cb0baa8 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:14.244550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b13619-ed58-41ae-8d30-9a718cb0baa8 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:14.244941 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b13619-ed58-41ae-8d30-9a718cb0baa8 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-38807b88-3488-4894-abbb-960a96149e0a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:14.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:14.246776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 118/240] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:46:14.248276 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38807b88-3488-4894-abbb-960a96149e0a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:14.249593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 123/241] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:14.254623 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea96b62-6e07-4bef-a81f-812eca267ac3 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:14.254988 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea96b62-6e07-4bef-a81f-812eca267ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:14.255346 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea96b62-6e07-4bef-a81f-812eca267ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:14.264284 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:14.276633 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-8ea96b62-6e07-4bef-a81f-812eca267ac3 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:14.280782 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-8ea96b62-6e07-4bef-a81f-812eca267ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:14.372876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 124/242] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:14.377474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea96b62-6e07-4bef-a81f-812eca267ac3 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:14.382072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38807b88-3488-4894-abbb-960a96149e0a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:14.423890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:14.436708 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647b8cc-ab2b-4be6-ac70-ebc9116627f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:14.437391 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647b8cc-ab2b-4be6-ac70-ebc9116627f1 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:14.437735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647b8cc-ab2b-4be6-ac70-ebc9116627f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:14.438012 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647b8cc-ab2b-4be6-ac70-ebc9116627f1 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:14.439315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647b8cc-ab2b-4be6-ac70-ebc9116627f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:14.441401 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647b8cc-ab2b-4be6-ac70-ebc9116627f1 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:14.506371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea96b62-6e07-4bef-a81f-812eca267ac3 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:14.540676 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577df2e-82d4-4b25-b78d-da740ca7d7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:14.541347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577df2e-82d4-4b25-b78d-da740ca7d7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:14.541629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577df2e-82d4-4b25-b78d-da740ca7d7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:14.541898 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577df2e-82d4-4b25-b78d-da740ca7d7d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:14.542642 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577df2e-82d4-4b25-b78d-da740ca7d7d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:14.542945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577df2e-82d4-4b25-b78d-da740ca7d7d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:14.544422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577df2e-82d4-4b25-b78d-da740ca7d7d6 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:14.553673 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 125/243] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:14.554896 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577df2e-82d4-4b25-b78d-da740ca7d7d6 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:14.567803 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 119/244] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:46:14 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:14.569047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77332f95-90ea-4285-9174-393e07ee7cad None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:14.569336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77332f95-90ea-4285-9174-393e07ee7cad None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:14.569968 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77332f95-90ea-4285-9174-393e07ee7cad None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:14.571499 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 120/245] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:14.576464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77332f95-90ea-4285-9174-393e07ee7cad None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:14.665776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:14.699950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:14.712671 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577df2e-82d4-4b25-b78d-da740ca7d7d6 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:14.728518 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77332f95-90ea-4285-9174-393e07ee7cad None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:14.754982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98926014-ee06-45e5-a487-86e5e96a374c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:14.756200 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98926014-ee06-45e5-a487-86e5e96a374c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:14.756770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98926014-ee06-45e5-a487-86e5e96a374c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:14.757248 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98926014-ee06-45e5-a487-86e5e96a374c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:14.772621 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98926014-ee06-45e5-a487-86e5e96a374c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:14.774617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c459dd2-2daf-4636-aeba-4a580a149c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:14.775535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c459dd2-2daf-4636-aeba-4a580a149c87 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:14.775821 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c459dd2-2daf-4636-aeba-4a580a149c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:14.776096 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c459dd2-2daf-4636-aeba-4a580a149c87 None admin] PATH_INFO: `/v3/users/cb6403f6c76d44718ed67ced1af5af84` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-98926014-ee06-45e5-a487-86e5e96a374c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:14.783018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:14.783948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 126/246] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:14.787746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73a664-2aa5-49db-8529-87bddc181b61 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:14.788467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73a664-2aa5-49db-8529-87bddc181b61 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:14.788467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73a664-2aa5-49db-8529-87bddc181b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:14.801870 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c459dd2-2daf-4636-aeba-4a580a149c87 None admin] RBAC: Authorizing `identity:get_user(user_id=cb6403f6c76d44718ed67ced1af5af84)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:14.803626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c459dd2-2daf-4636-aeba-4a580a149c87 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:14.805686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 121/247] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:14 2022] GET /identity/v3/users/cb6403f6c76d44718ed67ced1af5af84 => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:14.805843 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-8e73a664-2aa5-49db-8529-87bddc181b61 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:14.808384 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c459dd2-2daf-4636-aeba-4a580a149c87 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:14.808516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-8e73a664-2aa5-49db-8529-87bddc181b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:14.892654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 127/248] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:14.893580 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73a664-2aa5-49db-8529-87bddc181b61 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:14.954906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c459dd2-2daf-4636-aeba-4a580a149c87 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:14.987151 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:15.003505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91542d-f373-489f-a94d-d6323e94c685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:15.004153 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91542d-f373-489f-a94d-d6323e94c685 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:15.005049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91542d-f373-489f-a94d-d6323e94c685 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:15.005049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91542d-f373-489f-a94d-d6323e94c685 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:15.010105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91542d-f373-489f-a94d-d6323e94c685 None admin] RBAC: Adding query filter params (name=member) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:15.010105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91542d-f373-489f-a94d-d6323e94c685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:15.010105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91542d-f373-489f-a94d-d6323e94c685 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:15.020472 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 122/249] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:15.021696 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91542d-f373-489f-a94d-d6323e94c685 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:15.024613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:15.054329 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e73a664-2aa5-49db-8529-87bddc181b61 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:15.108821 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381f002-5129-44e9-968e-4e80ecdcc57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:15.109486 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381f002-5129-44e9-968e-4e80ecdcc57f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:15.109774 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381f002-5129-44e9-968e-4e80ecdcc57f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:15.110053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381f002-5129-44e9-968e-4e80ecdcc57f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:15.159107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91542d-f373-489f-a94d-d6323e94c685 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:15.186135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381f002-5129-44e9-968e-4e80ecdcc57f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:15.187618 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381f002-5129-44e9-968e-4e80ecdcc57f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:15.212718 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c579a51-6a5e-49c4-8ca9-4259c35221ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:15.213517 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c579a51-6a5e-49c4-8ca9-4259c35221ba None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:15.217419 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c579a51-6a5e-49c4-8ca9-4259c35221ba None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:15.217665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c579a51-6a5e-49c4-8ca9-4259c35221ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:15.239805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c579a51-6a5e-49c4-8ca9-4259c35221ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-6c579a51-6a5e-49c4-8ca9-4259c35221ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:15.252435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:15.253394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:15.254171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:15.260779 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 123/250] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:46:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:15.261970 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c579a51-6a5e-49c4-8ca9-4259c35221ba None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:15.308754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:15.325580 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 128/251] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:15.333110 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381f002-5129-44e9-968e-4e80ecdcc57f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:15.366667 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:15.424524 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c579a51-6a5e-49c4-8ca9-4259c35221ba None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:15.470138 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94323172-7855-4389-9352-8a567a37a3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:15.471589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323172-7855-4389-9352-8a567a37a3dd None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:15.472081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323172-7855-4389-9352-8a567a37a3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:15.472608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323172-7855-4389-9352-8a567a37a3dd None admin] PATH_INFO: `/v3/users/8e7b764c1fb44e628cefe267e29d65fb` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:15.476667 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94323172-7855-4389-9352-8a567a37a3dd None admin] RBAC: Authorizing `identity:get_user(user_id=8e7b764c1fb44e628cefe267e29d65fb)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:15.479454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94323172-7855-4389-9352-8a567a37a3dd None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:15.482644 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 124/252] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:15 2022] GET /identity/v3/users/8e7b764c1fb44e628cefe267e29d65fb => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:15.484059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94323172-7855-4389-9352-8a567a37a3dd None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:15.604931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381f002-5129-44e9-968e-4e80ecdcc57f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:15.672143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef0f60-0cb3-4e7d-8df0-79830583c162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:15.673472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef0f60-0cb3-4e7d-8df0-79830583c162 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:15.673764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef0f60-0cb3-4e7d-8df0-79830583c162 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:15.674043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef0f60-0cb3-4e7d-8df0-79830583c162 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:15.674918 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef0f60-0cb3-4e7d-8df0-79830583c162 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:15.675234 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef0f60-0cb3-4e7d-8df0-79830583c162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:15.676676 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef0f60-0cb3-4e7d-8df0-79830583c162 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:15.687650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94323172-7855-4389-9352-8a567a37a3dd None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:15.696360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 129/253] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:15.697142 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef0f60-0cb3-4e7d-8df0-79830583c162 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:15.699302 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:15.726406 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46dc97-8d3a-44e8-8875-fa9fe69ef54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:15.727466 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46dc97-8d3a-44e8-8875-fa9fe69ef54c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:15.727747 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46dc97-8d3a-44e8-8875-fa9fe69ef54c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:15.728025 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46dc97-8d3a-44e8-8875-fa9fe69ef54c None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:15.729304 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46dc97-8d3a-44e8-8875-fa9fe69ef54c None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:15.731396 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e46dc97-8d3a-44e8-8875-fa9fe69ef54c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:15.734218 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 125/254] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:15 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:15.735773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46dc97-8d3a-44e8-8875-fa9fe69ef54c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:15.790984 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:15.865160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46dc97-8d3a-44e8-8875-fa9fe69ef54c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:15.904012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef0f60-0cb3-4e7d-8df0-79830583c162 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:15.910454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653cc4d-be56-4d6d-ba8a-1ee9a1126d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:15.911367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653cc4d-be56-4d6d-ba8a-1ee9a1126d61 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:15.911735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653cc4d-be56-4d6d-ba8a-1ee9a1126d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:15.912209 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653cc4d-be56-4d6d-ba8a-1ee9a1126d61 None admin] PATH_INFO: `/v3/users/4c6b548fdba34d1280f0a5a80a1e7684` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:15.914314 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653cc4d-be56-4d6d-ba8a-1ee9a1126d61 None admin] RBAC: Authorizing `identity:get_user(user_id=4c6b548fdba34d1280f0a5a80a1e7684)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:15.916845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653cc4d-be56-4d6d-ba8a-1ee9a1126d61 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:15.919656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 126/255] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:15 2022] GET /identity/v3/users/4c6b548fdba34d1280f0a5a80a1e7684 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:15.920693 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653cc4d-be56-4d6d-ba8a-1ee9a1126d61 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:15.946366 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea710172-b2bc-4d94-8126-6b6bae83efa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:15.947347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710172-b2bc-4d94-8126-6b6bae83efa5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:15.947734 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710172-b2bc-4d94-8126-6b6bae83efa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:15.948028 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea710172-b2bc-4d94-8126-6b6bae83efa5 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:15.949305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea710172-b2bc-4d94-8126-6b6bae83efa5 None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:15.950925 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea710172-b2bc-4d94-8126-6b6bae83efa5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:15.958205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 130/256] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:15 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:15.959616 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1756d-0009-44cd-8b79-bf0d13879021 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:15.959907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1756d-0009-44cd-8b79-bf0d13879021 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:15.960526 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1756d-0009-44cd-8b79-bf0d13879021 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:15.962599 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 131/257] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:15.968136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1756d-0009-44cd-8b79-bf0d13879021 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:15.979054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:16.062772 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653cc4d-be56-4d6d-ba8a-1ee9a1126d61 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:16.096046 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1756d-0009-44cd-8b79-bf0d13879021 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:16.109144 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:16.122548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2fe84-3656-47d4-878c-228fad9b92c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:16.122548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2fe84-3656-47d4-878c-228fad9b92c7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.122548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2fe84-3656-47d4-878c-228fad9b92c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.122548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2fe84-3656-47d4-878c-228fad9b92c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.126751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2fe84-3656-47d4-878c-228fad9b92c7 None admin] RBAC: Adding query filter params (role.id=bd9a605a759046cf9bd1a8e9bb20d9d4, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61, user.id=cb6403f6c76d44718ed67ced1af5af84) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:16.126751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2fe84-3656-47d4-878c-228fad9b92c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:16.126751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2fe84-3656-47d4-878c-228fad9b92c7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:16.129926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 127/258] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:15 2022] GET /identity/v3/role_assignments?user.id=cb6403f6c76d44718ed67ced1af5af84&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=bd9a605a759046cf9bd1a8e9bb20d9d4 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:16.136443 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2fe84-3656-47d4-878c-228fad9b92c7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:16.153608 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4443a-b42a-4d50-8252-4b5fe8a4dd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:16.154635 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4443a-b42a-4d50-8252-4b5fe8a4dd20 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.155126 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4443a-b42a-4d50-8252-4b5fe8a4dd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.155555 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4443a-b42a-4d50-8252-4b5fe8a4dd20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.260481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2fe84-3656-47d4-878c-228fad9b92c7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:16.265485 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:16.289398 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4443a-b42a-4d50-8252-4b5fe8a4dd20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:16.290916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4443a-b42a-4d50-8252-4b5fe8a4dd20 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:16.296284 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7611ff-c006-41e7-abe4-0b851d2433fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:16.297572 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7611ff-c006-41e7-abe4-0b851d2433fa None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.298654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7611ff-c006-41e7-abe4-0b851d2433fa None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.299078 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7611ff-c006-41e7-abe4-0b851d2433fa None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/users/8e7b764c1fb44e628cefe267e29d65fb/roles/6a41401a07c443f0b5acd2e189145727` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.301525 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7611ff-c006-41e7-abe4-0b851d2433fa None admin] RBAC: Authorizing `identity:create_grant(project_id=03030ff4e6b54acfb48d93cd4178fd61, user_id=8e7b764c1fb44e628cefe267e29d65fb, role_id=6a41401a07c443f0b5acd2e189145727)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:16.304714 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7611ff-c006-41e7-abe4-0b851d2433fa None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:16.399898 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 128/259] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:46:16 2022] PUT /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/users/8e7b764c1fb44e628cefe267e29d65fb/roles/6a41401a07c443f0b5acd2e189145727 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:16.401636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c176341-a9de-423f-8588-b81158a4d152 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.402106 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c176341-a9de-423f-8588-b81158a4d152 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.402613 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c176341-a9de-423f-8588-b81158a4d152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.448805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-6c176341-a9de-423f-8588-b81158a4d152 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:16.451218 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-6c176341-a9de-423f-8588-b81158a4d152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:16.518768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 132/260] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:16.524131 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4443a-b42a-4d50-8252-4b5fe8a4dd20 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:16.550600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:16.576516 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 129/261] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:16.627245 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c176341-a9de-423f-8588-b81158a4d152 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:16.732397 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4443a-b42a-4d50-8252-4b5fe8a4dd20 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:16.783151 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646f5a9-7452-455f-8dad-63ab401febbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:16.784252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646f5a9-7452-455f-8dad-63ab401febbe None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.784698 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646f5a9-7452-455f-8dad-63ab401febbe None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.785111 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646f5a9-7452-455f-8dad-63ab401febbe None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.786515 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646f5a9-7452-455f-8dad-63ab401febbe None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:16.788169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646f5a9-7452-455f-8dad-63ab401febbe None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:16.790401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 133/262] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:16 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:16.801429 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646f5a9-7452-455f-8dad-63ab401febbe None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:16.865451 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:16.870624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c176341-a9de-423f-8588-b81158a4d152 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:16.900778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:16.923718 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646f5a9-7452-455f-8dad-63ab401febbe None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:16.937675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2e439-9db7-4bbc-a3ef-72b3c0e6c36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:16.938546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2e439-9db7-4bbc-a3ef-72b3c0e6c36e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.938835 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2e439-9db7-4bbc-a3ef-72b3c0e6c36e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.939114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2e439-9db7-4bbc-a3ef-72b3c0e6c36e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.958458 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2e439-9db7-4bbc-a3ef-72b3c0e6c36e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:16.962995 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e698ee-f9c3-40a6-9159-a8452bb0b2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:16.964152 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e698ee-f9c3-40a6-9159-a8452bb0b2cf None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:16.964804 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e698ee-f9c3-40a6-9159-a8452bb0b2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:16.965225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e698ee-f9c3-40a6-9159-a8452bb0b2cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:16.966269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e698ee-f9c3-40a6-9159-a8452bb0b2cf None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61, user.id=4c6b548fdba34d1280f0a5a80a1e7684) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:16.966724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e698ee-f9c3-40a6-9159-a8452bb0b2cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:16.968311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e698ee-f9c3-40a6-9159-a8452bb0b2cf None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-9bd2e439-9db7-4bbc-a3ef-72b3c0e6c36e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:16.975105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:16.976278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:16.977193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 130/263] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:46:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:16.979695 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 134/264] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:16 2022] GET /identity/v3/role_assignments?user.id=4c6b548fdba34d1280f0a5a80a1e7684&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:16.984311 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2e439-9db7-4bbc-a3ef-72b3c0e6c36e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:17.136503 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2e439-9db7-4bbc-a3ef-72b3c0e6c36e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:17.188136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09744720-cde3-4b46-9bcc-c036b822d68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:17.189406 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09744720-cde3-4b46-9bcc-c036b822d68e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.189695 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09744720-cde3-4b46-9bcc-c036b822d68e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.189977 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09744720-cde3-4b46-9bcc-c036b822d68e None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.190764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09744720-cde3-4b46-9bcc-c036b822d68e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:17.191080 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09744720-cde3-4b46-9bcc-c036b822d68e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:17.192595 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09744720-cde3-4b46-9bcc-c036b822d68e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:17.208031 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 131/265] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:17.214403 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e698ee-f9c3-40a6-9159-a8452bb0b2cf None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:17.397929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e698ee-f9c3-40a6-9159-a8452bb0b2cf None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:17.445455 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef607e6-5640-407c-b794-d25f32a267bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:17.446334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef607e6-5640-407c-b794-d25f32a267bf None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.446617 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef607e6-5640-407c-b794-d25f32a267bf None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.446943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef607e6-5640-407c-b794-d25f32a267bf None admin] PATH_INFO: `/v3/users/4c6b548fdba34d1280f0a5a80a1e7684` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.450806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef607e6-5640-407c-b794-d25f32a267bf None admin] RBAC: Authorizing `identity:get_user(user_id=4c6b548fdba34d1280f0a5a80a1e7684)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:17.452323 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef607e6-5640-407c-b794-d25f32a267bf None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:17.454947 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 135/266] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:17 2022] GET /identity/v3/users/4c6b548fdba34d1280f0a5a80a1e7684 => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:17.460300 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09744720-cde3-4b46-9bcc-c036b822d68e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:17.487086 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0583e0-b5bc-4938-902a-04d361f20f35 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.487374 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0583e0-b5bc-4938-902a-04d361f20f35 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.487651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0583e0-b5bc-4938-902a-04d361f20f35 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.489355 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 136/267] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:17.498755 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a05996-195b-4243-9b36-968108c7bb78 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.501386 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a05996-195b-4243-9b36-968108c7bb78 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.501386 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a05996-195b-4243-9b36-968108c7bb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.508631 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:17.524318 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-f0a05996-195b-4243-9b36-968108c7bb78 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:17.526293 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-f0a05996-195b-4243-9b36-968108c7bb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:17.530543 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:17.612363 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 137/268] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:17.612363 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7a83c-4b9d-4b5a-a2b7-8199ab617cb8 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.612363 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7a83c-4b9d-4b5a-a2b7-8199ab617cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.612363 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7a83c-4b9d-4b5a-a2b7-8199ab617cb8 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.612363 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 138/269] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:17.622381 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f26e6-343e-47ba-b8a1-5ff5c7437a01 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.622680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f26e6-343e-47ba-b8a1-5ff5c7437a01 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.622934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f26e6-343e-47ba-b8a1-5ff5c7437a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.649981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-f13f26e6-343e-47ba-b8a1-5ff5c7437a01 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:17.651524 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-f13f26e6-343e-47ba-b8a1-5ff5c7437a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:17.667817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09744720-cde3-4b46-9bcc-c036b822d68e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:17.711459 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652ff73-96bc-4316-82cf-8d6a79565a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:17.712112 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ff73-96bc-4316-82cf-8d6a79565a8e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.712418 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ff73-96bc-4316-82cf-8d6a79565a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.712693 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ff73-96bc-4316-82cf-8d6a79565a8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.756218 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 139/270] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:17.756218 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f26e6-343e-47ba-b8a1-5ff5c7437a01 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:17.820905 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652ff73-96bc-4316-82cf-8d6a79565a8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:17.822724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652ff73-96bc-4316-82cf-8d6a79565a8e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:17.840931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:17.848910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:17.872322 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f26e6-343e-47ba-b8a1-5ff5c7437a01 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:17.910385 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899530d-746d-42ca-a5d9-3bfd11dd7e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:17.911364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899530d-746d-42ca-a5d9-3bfd11dd7e0d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:17.911364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899530d-746d-42ca-a5d9-3bfd11dd7e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:17.911662 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899530d-746d-42ca-a5d9-3bfd11dd7e0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:17.913037 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899530d-746d-42ca-a5d9-3bfd11dd7e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:17.914670 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899530d-746d-42ca-a5d9-3bfd11dd7e0d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:17.920737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 140/271] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:17.920737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899530d-746d-42ca-a5d9-3bfd11dd7e0d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:17.943227 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 132/272] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:17.944532 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652ff73-96bc-4316-82cf-8d6a79565a8e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:18.027041 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899530d-746d-42ca-a5d9-3bfd11dd7e0d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:18.055635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652ff73-96bc-4316-82cf-8d6a79565a8e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:18.065427 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc5d4d-c0cb-4a36-814d-24a568033367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.066298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5d4d-c0cb-4a36-814d-24a568033367 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.066579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5d4d-c0cb-4a36-814d-24a568033367 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.066855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5d4d-c0cb-4a36-814d-24a568033367 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.076923 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:18.082456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc5d4d-c0cb-4a36-814d-24a568033367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:18.087500 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-59dc5d4d-c0cb-4a36-814d-24a568033367 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:18.091124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:18.100147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 141/273] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:18.101106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc5d4d-c0cb-4a36-814d-24a568033367 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:18.109701 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba369864-10b5-44a9-891b-c43c39849e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.111182 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba369864-10b5-44a9-891b-c43c39849e5d None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.111749 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba369864-10b5-44a9-891b-c43c39849e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.112464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba369864-10b5-44a9-891b-c43c39849e5d None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.113354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba369864-10b5-44a9-891b-c43c39849e5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:18.115537 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba369864-10b5-44a9-891b-c43c39849e5d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:18.206034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc5d4d-c0cb-4a36-814d-24a568033367 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:18.235789 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 133/274] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:46:17 2022] POST /identity/v3/users => generated 293 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:18.245419 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba369864-10b5-44a9-891b-c43c39849e5d None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:18.262069 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250cfd0-bc9f-4fa4-a256-45d6c38f314a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.264627 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250cfd0-bc9f-4fa4-a256-45d6c38f314a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.265348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250cfd0-bc9f-4fa4-a256-45d6c38f314a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.265855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250cfd0-bc9f-4fa4-a256-45d6c38f314a None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.270809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250cfd0-bc9f-4fa4-a256-45d6c38f314a None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:18.273643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4250cfd0-bc9f-4fa4-a256-45d6c38f314a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:18.277978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 142/275] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:18 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:18.285574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250cfd0-bc9f-4fa4-a256-45d6c38f314a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:18.402463 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:18.443504 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250cfd0-bc9f-4fa4-a256-45d6c38f314a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:18.487172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a450c8-b359-4e13-9acd-2e6da217c239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.487776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a450c8-b359-4e13-9acd-2e6da217c239 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.488060 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a450c8-b359-4e13-9acd-2e6da217c239 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.488368 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a450c8-b359-4e13-9acd-2e6da217c239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.489356 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a450c8-b359-4e13-9acd-2e6da217c239 None admin] RBAC: Adding query filter params (role.id=cb116a5614cf4a48ad99840d95f96bb3, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61, user.id=4c6b548fdba34d1280f0a5a80a1e7684) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:18.489726 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a450c8-b359-4e13-9acd-2e6da217c239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:18.491279 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a450c8-b359-4e13-9acd-2e6da217c239 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:18.509293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba369864-10b5-44a9-891b-c43c39849e5d None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:18.551197 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902531-5249-4e7f-8208-11f3ebe27b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.555009 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 143/276] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:18 2022] GET /identity/v3/role_assignments?user.id=4c6b548fdba34d1280f0a5a80a1e7684&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=cb116a5614cf4a48ad99840d95f96bb3 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:18.556118 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902531-5249-4e7f-8208-11f3ebe27b88 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.556679 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902531-5249-4e7f-8208-11f3ebe27b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.557167 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902531-5249-4e7f-8208-11f3ebe27b88 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.558136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74902531-5249-4e7f-8208-11f3ebe27b88 None admin] RBAC: Adding query filter params (name=member) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:18.558638 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74902531-5249-4e7f-8208-11f3ebe27b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:18.560305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74902531-5249-4e7f-8208-11f3ebe27b88 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:18.579669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 134/277] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:18.583647 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a450c8-b359-4e13-9acd-2e6da217c239 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:18.745169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a450c8-b359-4e13-9acd-2e6da217c239 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:18.779155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:18.787095 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa11cb-0141-4eb0-8cd4-789a852c6ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.787996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa11cb-0141-4eb0-8cd4-789a852c6ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.788286 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa11cb-0141-4eb0-8cd4-789a852c6ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.803173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa11cb-0141-4eb0-8cd4-789a852c6ed5 None admin] PATH_INFO: `/v3/users/e7bddafa59b0415596d9a55b9c000242` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.808555 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa11cb-0141-4eb0-8cd4-789a852c6ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=e7bddafa59b0415596d9a55b9c000242)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:18.810290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa11cb-0141-4eb0-8cd4-789a852c6ed5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:18.812472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 144/278] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:18 2022] GET /identity/v3/users/e7bddafa59b0415596d9a55b9c000242 => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:18.818850 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902531-5249-4e7f-8208-11f3ebe27b88 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:18.881432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:18.958706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902531-5249-4e7f-8208-11f3ebe27b88 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:18.997269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4adab9-882e-44dd-a9d2-182422634801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:18.997846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4adab9-882e-44dd-a9d2-182422634801 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:18.998125 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4adab9-882e-44dd-a9d2-182422634801 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:18.998438 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4adab9-882e-44dd-a9d2-182422634801 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:18.999299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4adab9-882e-44dd-a9d2-182422634801 None admin] RBAC: Adding query filter params (role.id=6a41401a07c443f0b5acd2e189145727, user.id=e7bddafa59b0415596d9a55b9c000242, scope.system=all) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:18.999611 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4adab9-882e-44dd-a9d2-182422634801 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:19.001653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4adab9-882e-44dd-a9d2-182422634801 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:19.007722 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 135/279] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 19:46:18 2022] GET /identity/v3/role_assignments?user.id=e7bddafa59b0415596d9a55b9c000242&scope.system=all&role.id=6a41401a07c443f0b5acd2e189145727 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:19.256868 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869e379-1398-497f-8702-c57be1a2527f None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:19.257452 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322e15-6b2e-4f77-8fa6-d51cad74fb72 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:19.257780 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869e379-1398-497f-8702-c57be1a2527f None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:19.257780 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869e379-1398-497f-8702-c57be1a2527f None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:19.258453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322e15-6b2e-4f77-8fa6-d51cad74fb72 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:19.258949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322e15-6b2e-4f77-8fa6-d51cad74fb72 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:19.259046 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 136/280] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:19.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 145/281] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:19.268286 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65039b-1225-4d8d-850d-4816c2a11378 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:19.268596 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65039b-1225-4d8d-850d-4816c2a11378 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:19.268881 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65039b-1225-4d8d-850d-4816c2a11378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:19.275774 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e681e3-0dd7-41c8-9a71-5b94bc2569c9 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:19.276077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e681e3-0dd7-41c8-9a71-5b94bc2569c9 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:19.276330 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e681e3-0dd7-41c8-9a71-5b94bc2569c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:19.306781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-bc65039b-1225-4d8d-850d-4816c2a11378 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:19.308016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-bc65039b-1225-4d8d-850d-4816c2a11378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:19.311103 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-23e681e3-0dd7-41c8-9a71-5b94bc2569c9 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:19.312276 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-23e681e3-0dd7-41c8-9a71-5b94bc2569c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:19.435760 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 137/282] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:19.466672 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65039b-1225-4d8d-850d-4816c2a11378 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:19.480659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 146/283] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 19:46:19.515246 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e681e3-0dd7-41c8-9a71-5b94bc2569c9 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:19.532784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:19.626669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e681e3-0dd7-41c8-9a71-5b94bc2569c9 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:19.645800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:19.646314 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65039b-1225-4d8d-850d-4816c2a11378 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:19.668100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f6f9d-2cb1-4938-b273-f007601cdb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:19.669227 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f6f9d-2cb1-4938-b273-f007601cdb0d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:19.669662 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f6f9d-2cb1-4938-b273-f007601cdb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:19.671655 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f6f9d-2cb1-4938-b273-f007601cdb0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:19.693187 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f6f9d-2cb1-4938-b273-f007601cdb0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-d52f6f9d-2cb1-4938-b273-f007601cdb0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:19.704482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:19.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:19.706264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:19.707352 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 147/284] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:19.716167 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f6f9d-2cb1-4938-b273-f007601cdb0d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:19.732865 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341132b-5c47-442c-943e-c09eef34ec02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:19.733769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341132b-5c47-442c-943e-c09eef34ec02 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:19.734561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341132b-5c47-442c-943e-c09eef34ec02 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:19.734671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341132b-5c47-442c-943e-c09eef34ec02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:19.765687 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341132b-5c47-442c-943e-c09eef34ec02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-d341132b-5c47-442c-943e-c09eef34ec02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:19.774716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:19.775675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:19.776521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:19.783818 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 138/285] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:46:19.786375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341132b-5c47-442c-943e-c09eef34ec02 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:19.847975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f6f9d-2cb1-4938-b273-f007601cdb0d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:19.911953 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c6f78-f7d1-4606-87ff-e1c03c9f6dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:19.913052 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c6f78-f7d1-4606-87ff-e1c03c9f6dea None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:19.913545 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c6f78-f7d1-4606-87ff-e1c03c9f6dea None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:19.914023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c6f78-f7d1-4606-87ff-e1c03c9f6dea None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:19.914985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c6f78-f7d1-4606-87ff-e1c03c9f6dea None admin] RBAC: Adding query filter params (name=reader) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:19.915512 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c6f78-f7d1-4606-87ff-e1c03c9f6dea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:19.917152 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c6f78-f7d1-4606-87ff-e1c03c9f6dea None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:19.927210 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 148/286] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:19.933523 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c6f78-f7d1-4606-87ff-e1c03c9f6dea None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:19.978621 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:19.996157 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:20.017350 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341132b-5c47-442c-943e-c09eef34ec02 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:20.077021 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c6f78-f7d1-4606-87ff-e1c03c9f6dea None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:20.094119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dee18-1834-4f48-926f-122c44f1ead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:20.094897 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dee18-1834-4f48-926f-122c44f1ead7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:20.095140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dee18-1834-4f48-926f-122c44f1ead7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:20.095422 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dee18-1834-4f48-926f-122c44f1ead7 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:20.096189 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dee18-1834-4f48-926f-122c44f1ead7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:20.096529 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dee18-1834-4f48-926f-122c44f1ead7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:20.097965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131dee18-1834-4f48-926f-122c44f1ead7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:20.122046 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 139/287] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:20.127620 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d010de4-776d-4136-bf41-1c70339c439f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:20.127980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dee18-1834-4f48-926f-122c44f1ead7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:20.129200 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d010de4-776d-4136-bf41-1c70339c439f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:20.129748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d010de4-776d-4136-bf41-1c70339c439f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:20.130230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d010de4-776d-4136-bf41-1c70339c439f None admin] PATH_INFO: `/v3/users/cb6403f6c76d44718ed67ced1af5af84` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:20.131810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d010de4-776d-4136-bf41-1c70339c439f None admin] RBAC: Authorizing `identity:get_user(user_id=cb6403f6c76d44718ed67ced1af5af84)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:20.133541 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d010de4-776d-4136-bf41-1c70339c439f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:20.135881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 149/288] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:19 2022] GET /identity/v3/users/cb6403f6c76d44718ed67ced1af5af84 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:20.142376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d010de4-776d-4136-bf41-1c70339c439f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:20.274489 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d010de4-776d-4136-bf41-1c70339c439f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:20.304725 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:20.307345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dee18-1834-4f48-926f-122c44f1ead7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:20.332240 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c12e6-0455-41cd-ae62-55e7e8ac27d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:20.333250 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c12e6-0455-41cd-ae62-55e7e8ac27d8 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:20.333548 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c12e6-0455-41cd-ae62-55e7e8ac27d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:20.333835 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c12e6-0455-41cd-ae62-55e7e8ac27d8 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:20.352184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c12e6-0455-41cd-ae62-55e7e8ac27d8 None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:20.353987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c12e6-0455-41cd-ae62-55e7e8ac27d8 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:20.356392 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 150/289] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:20 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:20.363164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c12e6-0455-41cd-ae62-55e7e8ac27d8 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:20.369217 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2df67-4271-4fad-96ec-68a81a9deb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:20.371928 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2df67-4271-4fad-96ec-68a81a9deb2f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:20.372234 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2df67-4271-4fad-96ec-68a81a9deb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:20.372541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2df67-4271-4fad-96ec-68a81a9deb2f None admin] PATH_INFO: `/v3/users/8e7b764c1fb44e628cefe267e29d65fb` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:20.378445 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2df67-4271-4fad-96ec-68a81a9deb2f None admin] RBAC: Authorizing `identity:get_user(user_id=8e7b764c1fb44e628cefe267e29d65fb)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:20.380158 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2df67-4271-4fad-96ec-68a81a9deb2f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:20.386684 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 140/290] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:20 2022] GET /identity/v3/users/8e7b764c1fb44e628cefe267e29d65fb => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:20.389990 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2df67-4271-4fad-96ec-68a81a9deb2f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:20.447556 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:20.573361 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c12e6-0455-41cd-ae62-55e7e8ac27d8 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:20.573850 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2df67-4271-4fad-96ec-68a81a9deb2f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:20.642493 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd8da4-eed4-4987-929c-c2921f589581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:20.643167 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd8da4-eed4-4987-929c-c2921f589581 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:20.643456 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd8da4-eed4-4987-929c-c2921f589581 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:20.643738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd8da4-eed4-4987-929c-c2921f589581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:20.646021 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e94144-7d58-4433-9031-3fc5ca3f87d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:20.646922 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e94144-7d58-4433-9031-3fc5ca3f87d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:20.647207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e94144-7d58-4433-9031-3fc5ca3f87d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:20.647487 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e94144-7d58-4433-9031-3fc5ca3f87d0 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/users/cb6403f6c76d44718ed67ced1af5af84/roles/bd9a605a759046cf9bd1a8e9bb20d9d4` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:20.651999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e94144-7d58-4433-9031-3fc5ca3f87d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=03030ff4e6b54acfb48d93cd4178fd61, user_id=cb6403f6c76d44718ed67ced1af5af84, role_id=bd9a605a759046cf9bd1a8e9bb20d9d4)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:20.653479 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e94144-7d58-4433-9031-3fc5ca3f87d0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:20.729863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd8da4-eed4-4987-929c-c2921f589581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:20.731426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd8da4-eed4-4987-929c-c2921f589581 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:20.750654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 151/291] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:46:20 2022] PUT /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/users/cb6403f6c76d44718ed67ced1af5af84/roles/bd9a605a759046cf9bd1a8e9bb20d9d4 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:20.781750 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:20.887696 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 141/292] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:20.893867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e94144-7d58-4433-9031-3fc5ca3f87d0 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:20.944857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:21.035205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e94144-7d58-4433-9031-3fc5ca3f87d0 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:21.076804 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfef7de-22a3-4aed-bc10-9c3f93414de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:21.077610 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfef7de-22a3-4aed-bc10-9c3f93414de2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.077896 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfef7de-22a3-4aed-bc10-9c3f93414de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.078159 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfef7de-22a3-4aed-bc10-9c3f93414de2 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.080017 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfef7de-22a3-4aed-bc10-9c3f93414de2 None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:21.081802 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfef7de-22a3-4aed-bc10-9c3f93414de2 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:21.083767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 152/293] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:20 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:21.094350 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd8da4-eed4-4987-929c-c2921f589581 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:21.239350 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd8da4-eed4-4987-929c-c2921f589581 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:21.276196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:21.287806 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583b855-d9a7-44a4-a7fa-a5bfa256f5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:21.288524 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583b855-d9a7-44a4-a7fa-a5bfa256f5bf None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.292822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583b855-d9a7-44a4-a7fa-a5bfa256f5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.292822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583b855-d9a7-44a4-a7fa-a5bfa256f5bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.292822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583b855-d9a7-44a4-a7fa-a5bfa256f5bf None admin] RBAC: Adding query filter params (role.id=6a41401a07c443f0b5acd2e189145727, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61, user.id=8e7b764c1fb44e628cefe267e29d65fb) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:21.292822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583b855-d9a7-44a4-a7fa-a5bfa256f5bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:21.292822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6583b855-d9a7-44a4-a7fa-a5bfa256f5bf None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:21.306185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 142/294] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:21 2022] GET /identity/v3/role_assignments?user.id=8e7b764c1fb44e628cefe267e29d65fb&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=6a41401a07c443f0b5acd2e189145727 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:21.505283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e6230-8a0c-4aea-b331-76f0655e94ab None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.505803 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e6230-8a0c-4aea-b331-76f0655e94ab None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.506074 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e6230-8a0c-4aea-b331-76f0655e94ab None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.508419 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 153/295] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:21.515153 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb679f18-68a1-49a4-b999-7ad81fa277a5 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.515676 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb679f18-68a1-49a4-b999-7ad81fa277a5 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.516037 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb679f18-68a1-49a4-b999-7ad81fa277a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.548999 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-eb679f18-68a1-49a4-b999-7ad81fa277a5 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:21.550986 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-eb679f18-68a1-49a4-b999-7ad81fa277a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:21.709987 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 143/296] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:21.735747 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e6230-8a0c-4aea-b331-76f0655e94ab None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:21.848645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894f4a3-8eac-48a4-bc1c-cef26fb38a98 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.848942 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894f4a3-8eac-48a4-bc1c-cef26fb38a98 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.849224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894f4a3-8eac-48a4-bc1c-cef26fb38a98 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.851260 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 144/297] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:21.857098 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6e776-c9e7-4de0-b638-e64573ce5acc None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.857698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6e776-c9e7-4de0-b638-e64573ce5acc None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.858187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6e776-c9e7-4de0-b638-e64573ce5acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.861114 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e6230-8a0c-4aea-b331-76f0655e94ab None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:21.863727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:21.879377 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-b7b6e776-c9e7-4de0-b638-e64573ce5acc None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:21.882608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-b7b6e776-c9e7-4de0-b638-e64573ce5acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:21.899062 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc61daf-87fe-4517-b239-c8b5ca29a4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:21.899852 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc61daf-87fe-4517-b239-c8b5ca29a4bf None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.900137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc61daf-87fe-4517-b239-c8b5ca29a4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.900423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc61daf-87fe-4517-b239-c8b5ca29a4bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.911309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc61daf-87fe-4517-b239-c8b5ca29a4bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-7cc61daf-87fe-4517-b239-c8b5ca29a4bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:21.919402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:21.920462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:21.921371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 154/298] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:46:21.922124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf4923-c0ed-4c17-aee6-68709bbace04 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.922197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf4923-c0ed-4c17-aee6-68709bbace04 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.923227 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf4923-c0ed-4c17-aee6-68709bbace04 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.928541 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 155/299] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:21.930595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf4923-c0ed-4c17-aee6-68709bbace04 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:21.965222 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 145/300] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:21.966700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7ef29-095b-446c-a6c4-66ae1cfbcfaa None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:21.966999 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7ef29-095b-446c-a6c4-66ae1cfbcfaa None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:21.967274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7ef29-095b-446c-a6c4-66ae1cfbcfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:21.986101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-e2d7ef29-095b-446c-a6c4-66ae1cfbcfaa None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:21.987293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-e2d7ef29-095b-446c-a6c4-66ae1cfbcfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:22.038872 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:22.054733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf4923-c0ed-4c17-aee6-68709bbace04 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:22.068421 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 146/301] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:22.069515 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7ef29-095b-446c-a6c4-66ae1cfbcfaa None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:22.101209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c52cb-ecca-4fa2-b469-9cdeaf09e635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:22.102164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c52cb-ecca-4fa2-b469-9cdeaf09e635 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:22.102640 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c52cb-ecca-4fa2-b469-9cdeaf09e635 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:22.103163 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c52cb-ecca-4fa2-b469-9cdeaf09e635 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:22.104221 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c52cb-ecca-4fa2-b469-9cdeaf09e635 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:22.104736 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c52cb-ecca-4fa2-b469-9cdeaf09e635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:22.106362 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c52cb-ecca-4fa2-b469-9cdeaf09e635 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:22.115656 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 156/302] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:22.121070 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c52cb-ecca-4fa2-b469-9cdeaf09e635 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:22.161909 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:22.176529 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7ef29-095b-446c-a6c4-66ae1cfbcfaa None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:22.215678 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3af6bd-ad39-4155-a2df-36e84149f62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:22.216834 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3af6bd-ad39-4155-a2df-36e84149f62c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:22.217311 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3af6bd-ad39-4155-a2df-36e84149f62c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:22.217769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3af6bd-ad39-4155-a2df-36e84149f62c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:22.231572 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3af6bd-ad39-4155-a2df-36e84149f62c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-5c3af6bd-ad39-4155-a2df-36e84149f62c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:22.239625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:22.240526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:22.241400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 147/303] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:22.242054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3af6bd-ad39-4155-a2df-36e84149f62c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:22.249474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c52cb-ecca-4fa2-b469-9cdeaf09e635 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:22.286970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287a39a-efd9-4399-8156-d1d9fb925a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:22.287776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287a39a-efd9-4399-8156-d1d9fb925a16 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:22.288069 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287a39a-efd9-4399-8156-d1d9fb925a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:22.288415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287a39a-efd9-4399-8156-d1d9fb925a16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:22.289660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287a39a-efd9-4399-8156-d1d9fb925a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:22.291133 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287a39a-efd9-4399-8156-d1d9fb925a16 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:22.293179 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 157/304] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:22.294030 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287a39a-efd9-4399-8156-d1d9fb925a16 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:22.298227 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:22.375495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3af6bd-ad39-4155-a2df-36e84149f62c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:22.407975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:22.413384 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f701f6b-e497-453a-8062-1a592badf7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:22.414233 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f701f6b-e497-453a-8062-1a592badf7cd None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:22.414516 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f701f6b-e497-453a-8062-1a592badf7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:22.414792 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f701f6b-e497-453a-8062-1a592badf7cd None admin] PATH_INFO: `/v3/users/ce10c9e8d17a4af792860121d2bc0cb9` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:22.415068 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287a39a-efd9-4399-8156-d1d9fb925a16 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:22.430157 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f701f6b-e497-453a-8062-1a592badf7cd None admin] RBAC: Authorizing `identity:get_user(user_id=ce10c9e8d17a4af792860121d2bc0cb9)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:22.431978 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f701f6b-e497-453a-8062-1a592badf7cd None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:22.439024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 148/305] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:22 2022] GET /identity/v3/users/ce10c9e8d17a4af792860121d2bc0cb9 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:22.439024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f701f6b-e497-453a-8062-1a592badf7cd None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:22.456683 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11664dd6-a5c7-47cc-84f7-bf8f7667d172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:22.457456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11664dd6-a5c7-47cc-84f7-bf8f7667d172 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:22.457767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11664dd6-a5c7-47cc-84f7-bf8f7667d172 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:22.458042 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11664dd6-a5c7-47cc-84f7-bf8f7667d172 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:22.458791 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11664dd6-a5c7-47cc-84f7-bf8f7667d172 None admin] RBAC: Adding query filter params (name=member) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:22.459094 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11664dd6-a5c7-47cc-84f7-bf8f7667d172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:22.460612 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11664dd6-a5c7-47cc-84f7-bf8f7667d172 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:22.469996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 158/306] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:22.469996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11664dd6-a5c7-47cc-84f7-bf8f7667d172 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:22.544665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:22.550517 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f701f6b-e497-453a-8062-1a592badf7cd None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:22.583487 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11664dd6-a5c7-47cc-84f7-bf8f7667d172 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:22.589599 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3771b9-6184-4e2b-889c-aca6c05cba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:22.590266 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3771b9-6184-4e2b-889c-aca6c05cba74 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:22.590546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3771b9-6184-4e2b-889c-aca6c05cba74 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:22.591236 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3771b9-6184-4e2b-889c-aca6c05cba74 None admin] PATH_INFO: `/v3/groups` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:22.591932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3771b9-6184-4e2b-889c-aca6c05cba74 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:22.593794 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3771b9-6184-4e2b-889c-aca6c05cba74 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:22.616936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702a575-172e-41b3-873c-4e13fe7791bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:22.617679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a575-172e-41b3-873c-4e13fe7791bb None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:22.617963 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a575-172e-41b3-873c-4e13fe7791bb None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:22.618236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a575-172e-41b3-873c-4e13fe7791bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:22.619065 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702a575-172e-41b3-873c-4e13fe7791bb None admin] RBAC: Adding query filter params (role.id=bd9a605a759046cf9bd1a8e9bb20d9d4, user.id=ce10c9e8d17a4af792860121d2bc0cb9, scope.system=all) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:22.619375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702a575-172e-41b3-873c-4e13fe7791bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:22.620838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702a575-172e-41b3-873c-4e13fe7791bb None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:22.626307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 159/307] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 19:46:22 2022] GET /identity/v3/role_assignments?user.id=ce10c9e8d17a4af792860121d2bc0cb9&scope.system=all&role.id=bd9a605a759046cf9bd1a8e9bb20d9d4 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:22.627033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702a575-172e-41b3-873c-4e13fe7791bb None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:22.660201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:22.673020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 149/308] 10.222.0.88 () {60 vars in 1134 bytes} [Thu May 19 19:46:22 2022] POST /identity/v3/groups => generated 229 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:22.780261 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702a575-172e-41b3-873c-4e13fe7791bb None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:22.824838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0870085-9a40-411c-8041-4264c4c241cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:22.826179 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0870085-9a40-411c-8041-4264c4c241cd None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:22.826682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0870085-9a40-411c-8041-4264c4c241cd None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:22.827158 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0870085-9a40-411c-8041-4264c4c241cd None admin] PATH_INFO: `/v3/users/e7bddafa59b0415596d9a55b9c000242` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:22.828986 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0870085-9a40-411c-8041-4264c4c241cd None admin] RBAC: Authorizing `identity:get_user(user_id=e7bddafa59b0415596d9a55b9c000242)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:22.830806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0870085-9a40-411c-8041-4264c4c241cd None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:22.833205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 160/309] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:22 2022] GET /identity/v3/users/e7bddafa59b0415596d9a55b9c000242 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:22.839410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3771b9-6184-4e2b-889c-aca6c05cba74 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:22.957937 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3771b9-6184-4e2b-889c-aca6c05cba74 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:22.994619 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63140c6c-d16f-4fcf-908b-f3706ca3bda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:22.995756 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63140c6c-d16f-4fcf-908b-f3706ca3bda0 None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:22.996177 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63140c6c-d16f-4fcf-908b-f3706ca3bda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:22.996609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63140c6c-d16f-4fcf-908b-f3706ca3bda0 None admin] PATH_INFO: `/v3/system/users/e7bddafa59b0415596d9a55b9c000242/roles/6a41401a07c443f0b5acd2e189145727` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:22.998270 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63140c6c-d16f-4fcf-908b-f3706ca3bda0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e7bddafa59b0415596d9a55b9c000242, role_id=6a41401a07c443f0b5acd2e189145727)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:22.999941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63140c6c-d16f-4fcf-908b-f3706ca3bda0 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:23.092713 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 150/310] 10.222.0.88 () {58 vars in 1334 bytes} [Thu May 19 19:46:22 2022] PUT /identity/v3/system/users/e7bddafa59b0415596d9a55b9c000242/roles/6a41401a07c443f0b5acd2e189145727 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:23.460932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4d0bf-fff7-4a19-b6ed-771fa6c16846 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:23.461216 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4d0bf-fff7-4a19-b6ed-771fa6c16846 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:23.461490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4d0bf-fff7-4a19-b6ed-771fa6c16846 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:23.471236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 161/311] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:23.471360 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bdfcc-ec05-424c-abed-4dc7cd3a9ea7 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:23.471360 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bdfcc-ec05-424c-abed-4dc7cd3a9ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:23.471360 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390bdfcc-ec05-424c-abed-4dc7cd3a9ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:23.489013 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-390bdfcc-ec05-424c-abed-4dc7cd3a9ea7 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:23.490506 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-390bdfcc-ec05-424c-abed-4dc7cd3a9ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:23.511773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:23.579469 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 151/312] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:23.603081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4d0bf-fff7-4a19-b6ed-771fa6c16846 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:23.700404 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:23.714186 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4d0bf-fff7-4a19-b6ed-771fa6c16846 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:23.751627 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaff32-3e84-4b90-a182-2efc8799be5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:23.752770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaff32-3e84-4b90-a182-2efc8799be5b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:23.753237 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaff32-3e84-4b90-a182-2efc8799be5b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:23.753689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecaff32-3e84-4b90-a182-2efc8799be5b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:23.766713 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecaff32-3e84-4b90-a182-2efc8799be5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-0ecaff32-3e84-4b90-a182-2efc8799be5b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:23.775139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:23.776108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:23.776927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:23.779592 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 162/313] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:23.785597 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bdfcc-ec05-424c-abed-4dc7cd3a9ea7 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:23.898929 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390bdfcc-ec05-424c-abed-4dc7cd3a9ea7 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:23.936210 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15315da3-1754-4742-9a14-b9c165103464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:23.936943 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15315da3-1754-4742-9a14-b9c165103464 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:23.937234 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15315da3-1754-4742-9a14-b9c165103464 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:23.937511 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15315da3-1754-4742-9a14-b9c165103464 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:23.938348 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15315da3-1754-4742-9a14-b9c165103464 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:23.938657 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15315da3-1754-4742-9a14-b9c165103464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:23.940139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15315da3-1754-4742-9a14-b9c165103464 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:23.950750 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 152/314] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:23.956428 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaff32-3e84-4b90-a182-2efc8799be5b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:24.065674 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecaff32-3e84-4b90-a182-2efc8799be5b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:24.102393 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891eeed-2df8-4373-b6ce-d807d22feb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:24.103646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891eeed-2df8-4373-b6ce-d807d22feb2a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:24.104137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891eeed-2df8-4373-b6ce-d807d22feb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:24.104643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891eeed-2df8-4373-b6ce-d807d22feb2a None admin] PATH_INFO: `/v3/users/cb6403f6c76d44718ed67ced1af5af84` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:24.106133 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891eeed-2df8-4373-b6ce-d807d22feb2a None admin] RBAC: Authorizing `identity:get_user(user_id=cb6403f6c76d44718ed67ced1af5af84)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:24.107803 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891eeed-2df8-4373-b6ce-d807d22feb2a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:24.110297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 163/315] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:23 2022] GET /identity/v3/users/cb6403f6c76d44718ed67ced1af5af84 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:24.116366 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15315da3-1754-4742-9a14-b9c165103464 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:24.138119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:24.276739 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15315da3-1754-4742-9a14-b9c165103464 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:24.315543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c7c2f-c7e0-441b-81b8-3ab932085e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:24.316200 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c7c2f-c7e0-441b-81b8-3ab932085e56 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:24.316513 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c7c2f-c7e0-441b-81b8-3ab932085e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:24.316786 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c7c2f-c7e0-441b-81b8-3ab932085e56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:24.394038 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c7c2f-c7e0-441b-81b8-3ab932085e56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:24.395831 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c7c2f-c7e0-441b-81b8-3ab932085e56 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:24.411361 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:24.506417 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 153/316] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:24.514844 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891eeed-2df8-4373-b6ce-d807d22feb2a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:24.574975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:24.668698 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891eeed-2df8-4373-b6ce-d807d22feb2a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:24.706487 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ba9df-11af-48d3-ae18-eb56a19ac598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:24.707367 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ba9df-11af-48d3-ae18-eb56a19ac598 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:24.707648 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ba9df-11af-48d3-ae18-eb56a19ac598 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:24.707922 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ba9df-11af-48d3-ae18-eb56a19ac598 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:24.709417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ba9df-11af-48d3-ae18-eb56a19ac598 None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:24.710871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ba9df-11af-48d3-ae18-eb56a19ac598 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:24.713009 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 164/317] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:24 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:24.718927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c7c2f-c7e0-441b-81b8-3ab932085e56 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:24.851412 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c7c2f-c7e0-441b-81b8-3ab932085e56 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:24.876732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:24.890179 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de42519e-63c7-464e-ad5c-01ec289055ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:24.890917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42519e-63c7-464e-ad5c-01ec289055ce None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:24.891897 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42519e-63c7-464e-ad5c-01ec289055ce None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:24.892180 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42519e-63c7-464e-ad5c-01ec289055ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:24.893062 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42519e-63c7-464e-ad5c-01ec289055ce None admin] RBAC: Adding query filter params (role.id=bd9a605a759046cf9bd1a8e9bb20d9d4, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61, user.id=cb6403f6c76d44718ed67ced1af5af84) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:24.893375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42519e-63c7-464e-ad5c-01ec289055ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:24.894851 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de42519e-63c7-464e-ad5c-01ec289055ce None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:24.901899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 154/318] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:24 2022] GET /identity/v3/role_assignments?user.id=cb6403f6c76d44718ed67ced1af5af84&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=bd9a605a759046cf9bd1a8e9bb20d9d4 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:24.983006 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340bf38-b453-42a4-854b-56280873b4d5 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:24.983586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340bf38-b453-42a4-854b-56280873b4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:24.984021 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340bf38-b453-42a4-854b-56280873b4d5 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:24.986201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 165/319] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:24.992587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317006bc-d06f-409c-96c1-99f24ec58a3e None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:24.993151 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317006bc-d06f-409c-96c1-99f24ec58a3e None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:24.993578 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317006bc-d06f-409c-96c1-99f24ec58a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.014344 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-317006bc-d06f-409c-96c1-99f24ec58a3e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:25.016588 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-317006bc-d06f-409c-96c1-99f24ec58a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:25.099056 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 155/320] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:25.123096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340bf38-b453-42a4-854b-56280873b4d5 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:25.217465 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fd067-1a27-4605-a315-8100e680ce97 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:25.218176 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fd067-1a27-4605-a315-8100e680ce97 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:25.218613 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fd067-1a27-4605-a315-8100e680ce97 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.221942 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 156/321] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:25.228030 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caa18a-d6d0-458f-9469-1152690fd50f None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:25.228319 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caa18a-d6d0-458f-9469-1152690fd50f None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:25.228640 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49caa18a-d6d0-458f-9469-1152690fd50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.238401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:25.248937 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-49caa18a-d6d0-458f-9469-1152690fd50f None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:25.258186 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-49caa18a-d6d0-458f-9469-1152690fd50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:25.258343 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340bf38-b453-42a4-854b-56280873b4d5 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:25.297820 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d06fda-20e7-4c09-ab7a-ce2a2bac9782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:25.298864 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d06fda-20e7-4c09-ab7a-ce2a2bac9782 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:25.299147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d06fda-20e7-4c09-ab7a-ce2a2bac9782 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:25.299419 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d06fda-20e7-4c09-ab7a-ce2a2bac9782 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.311313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d06fda-20e7-4c09-ab7a-ce2a2bac9782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-d6d06fda-20e7-4c09-ab7a-ce2a2bac9782 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:25.320626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:25.321850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:25.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 166/322] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:25.328042 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d06fda-20e7-4c09-ab7a-ce2a2bac9782 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:25.357238 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 157/323] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:25.379406 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caa18a-d6d0-458f-9469-1152690fd50f None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:25.417457 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:25.448428 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d06fda-20e7-4c09-ab7a-ce2a2bac9782 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:25.485129 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcac3f9-587b-400c-994e-680118b7aae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:25.485797 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcac3f9-587b-400c-994e-680118b7aae2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:25.486086 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcac3f9-587b-400c-994e-680118b7aae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:25.486456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcac3f9-587b-400c-994e-680118b7aae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.487111 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcac3f9-587b-400c-994e-680118b7aae2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:25.487418 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcac3f9-587b-400c-994e-680118b7aae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:25.488835 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcac3f9-587b-400c-994e-680118b7aae2 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:25.490881 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49caa18a-d6d0-458f-9469-1152690fd50f None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:25.497547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 167/324] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:25.503379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59447a95-a683-4c33-a02c-2242cd407e03 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:25.503379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59447a95-a683-4c33-a02c-2242cd407e03 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:25.503379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59447a95-a683-4c33-a02c-2242cd407e03 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.503379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 168/325] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:25.505339 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59447a95-a683-4c33-a02c-2242cd407e03 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:25.514619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:25.533824 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd4d7b-9c8e-4d74-9e22-aa89a74177fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:25.534791 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd4d7b-9c8e-4d74-9e22-aa89a74177fe None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:25.535147 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd4d7b-9c8e-4d74-9e22-aa89a74177fe None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:25.535569 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd4d7b-9c8e-4d74-9e22-aa89a74177fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.537058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd4d7b-9c8e-4d74-9e22-aa89a74177fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:25.538816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd4d7b-9c8e-4d74-9e22-aa89a74177fe None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:25.540729 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 158/326] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:25.541887 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc941f5-17d1-4795-ba0d-9d9cf046584f None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:25.542165 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc941f5-17d1-4795-ba0d-9d9cf046584f None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:25.542445 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc941f5-17d1-4795-ba0d-9d9cf046584f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.559734 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-8fc941f5-17d1-4795-ba0d-9d9cf046584f None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:25.561069 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-8fc941f5-17d1-4795-ba0d-9d9cf046584f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:25.614734 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59447a95-a683-4c33-a02c-2242cd407e03 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:25.632598 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 159/327] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:25.633689 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc941f5-17d1-4795-ba0d-9d9cf046584f None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:25.653723 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa39d4-f273-4f7d-bd10-e17437e9ab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:25.654715 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa39d4-f273-4f7d-bd10-e17437e9ab44 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:25.654987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa39d4-f273-4f7d-bd10-e17437e9ab44 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:25.655261 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aa39d4-f273-4f7d-bd10-e17437e9ab44 None admin] PATH_INFO: `/v3/users/ce10c9e8d17a4af792860121d2bc0cb9` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.656720 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa39d4-f273-4f7d-bd10-e17437e9ab44 None admin] RBAC: Authorizing `identity:get_user(user_id=ce10c9e8d17a4af792860121d2bc0cb9)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:25.658164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aa39d4-f273-4f7d-bd10-e17437e9ab44 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:25.659821 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 169/328] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:25 2022] GET /identity/v3/users/ce10c9e8d17a4af792860121d2bc0cb9 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:25.660623 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa39d4-f273-4f7d-bd10-e17437e9ab44 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:25.677847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:25.740042 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:25.745350 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc941f5-17d1-4795-ba0d-9d9cf046584f None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:25.764646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aa39d4-f273-4f7d-bd10-e17437e9ab44 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:25.781513 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc519-db3e-4fa3-a89b-81e16e355bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:25.782238 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc519-db3e-4fa3-a89b-81e16e355bf9 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:25.782518 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc519-db3e-4fa3-a89b-81e16e355bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:25.782791 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bc519-db3e-4fa3-a89b-81e16e355bf9 None admin] PATH_INFO: `/v3/groups` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.783480 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bc519-db3e-4fa3-a89b-81e16e355bf9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:25.784989 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bc519-db3e-4fa3-a89b-81e16e355bf9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:25.800904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82242619-2e9e-4525-907b-e6439d5755e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:25.801697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82242619-2e9e-4525-907b-e6439d5755e0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:25.801978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82242619-2e9e-4525-907b-e6439d5755e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:25.802503 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82242619-2e9e-4525-907b-e6439d5755e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.816572 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82242619-2e9e-4525-907b-e6439d5755e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-82242619-2e9e-4525-907b-e6439d5755e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:25.824298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:25.825347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:25.826162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 170/329] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:25.826798 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82242619-2e9e-4525-907b-e6439d5755e0 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:25.886654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 160/330] 10.222.0.88 () {60 vars in 1134 bytes} [Thu May 19 19:46:25 2022] POST /identity/v3/groups => generated 226 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:25.887429 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc519-db3e-4fa3-a89b-81e16e355bf9 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:25.940738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82242619-2e9e-4525-907b-e6439d5755e0 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:25.994642 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8066f-c907-4011-8a63-ca7534464301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:25.995537 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8066f-c907-4011-8a63-ca7534464301 None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:25.995815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8066f-c907-4011-8a63-ca7534464301 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:25.996087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8066f-c907-4011-8a63-ca7534464301 None admin] PATH_INFO: `/v3/system/users/ce10c9e8d17a4af792860121d2bc0cb9/roles/bd9a605a759046cf9bd1a8e9bb20d9d4` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:25.997731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8066f-c907-4011-8a63-ca7534464301 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ce10c9e8d17a4af792860121d2bc0cb9, role_id=bd9a605a759046cf9bd1a8e9bb20d9d4)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:25.999156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8066f-c907-4011-8a63-ca7534464301 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:26.001768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:26.024131 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:26.031576 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bc519-db3e-4fa3-a89b-81e16e355bf9 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:26.072815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43947864-c8d0-442b-aeae-651509850690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:26.073484 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43947864-c8d0-442b-aeae-651509850690 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:26.073771 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43947864-c8d0-442b-aeae-651509850690 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:26.074040 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43947864-c8d0-442b-aeae-651509850690 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:26.074819 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43947864-c8d0-442b-aeae-651509850690 None admin] RBAC: Adding query filter params (name=member) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:26.075099 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43947864-c8d0-442b-aeae-651509850690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:26.076519 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43947864-c8d0-442b-aeae-651509850690 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:26.085511 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 161/331] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:26.091221 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43947864-c8d0-442b-aeae-651509850690 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:26.116386 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 171/332] 10.222.0.88 () {58 vars in 1334 bytes} [Thu May 19 19:46:25 2022] PUT /identity/v3/system/users/ce10c9e8d17a4af792860121d2bc0cb9/roles/bd9a605a759046cf9bd1a8e9bb20d9d4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:26.234167 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43947864-c8d0-442b-aeae-651509850690 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:26.286652 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e718b-fb0c-45d5-8673-690b9649db2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:26.287872 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e718b-fb0c-45d5-8673-690b9649db2c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:26.288324 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e718b-fb0c-45d5-8673-690b9649db2c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:26.288861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e718b-fb0c-45d5-8673-690b9649db2c None admin] PATH_INFO: `/v3/users/e7bddafa59b0415596d9a55b9c000242` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:26.290592 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e718b-fb0c-45d5-8673-690b9649db2c None admin] RBAC: Authorizing `identity:get_user(user_id=e7bddafa59b0415596d9a55b9c000242)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:26.292467 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e718b-fb0c-45d5-8673-690b9649db2c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:26.295043 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 162/333] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:26 2022] GET /identity/v3/users/e7bddafa59b0415596d9a55b9c000242 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:26.301209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8066f-c907-4011-8a63-ca7534464301 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:26.442202 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8066f-c907-4011-8a63-ca7534464301 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:26.480378 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4056b0-1076-435c-8bf9-971a1130dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:26.481287 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4056b0-1076-435c-8bf9-971a1130dc55 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:26.481776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4056b0-1076-435c-8bf9-971a1130dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:26.482200 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4056b0-1076-435c-8bf9-971a1130dc55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:26.483325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4056b0-1076-435c-8bf9-971a1130dc55 None admin] RBAC: Adding query filter params (role.id=6a41401a07c443f0b5acd2e189145727, user.id=e7bddafa59b0415596d9a55b9c000242, scope.system=all) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:26.483797 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4056b0-1076-435c-8bf9-971a1130dc55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:26.485514 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4056b0-1076-435c-8bf9-971a1130dc55 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:26.499029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 172/334] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 19:46:26 2022] GET /identity/v3/role_assignments?user.id=e7bddafa59b0415596d9a55b9c000242&scope.system=all&role.id=6a41401a07c443f0b5acd2e189145727 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:28.434216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5971f-eb92-44c2-a960-da604bf8c8ff None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:28.435187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5971f-eb92-44c2-a960-da604bf8c8ff None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:28.435697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5971f-eb92-44c2-a960-da604bf8c8ff None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:28.437360 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 163/335] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:28.443937 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b98c4d-af38-47b4-900d-5509a1bee8dc None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:28.444975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b98c4d-af38-47b4-900d-5509a1bee8dc None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:28.445493 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b98c4d-af38-47b4-900d-5509a1bee8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:28.466036 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-29b98c4d-af38-47b4-900d-5509a1bee8dc None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:28.467571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-29b98c4d-af38-47b4-900d-5509a1bee8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:28.542808 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:28.561370 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 173/336] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:28.603123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5971f-eb92-44c2-a960-da604bf8c8ff None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:28.650283 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:28.761774 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa5971f-eb92-44c2-a960-da604bf8c8ff None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:28.810516 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1feb867-ee48-466c-b33a-e41a9b13b33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:28.811964 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feb867-ee48-466c-b33a-e41a9b13b33d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:28.812592 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feb867-ee48-466c-b33a-e41a9b13b33d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:28.813281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feb867-ee48-466c-b33a-e41a9b13b33d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:28.828561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1feb867-ee48-466c-b33a-e41a9b13b33d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-f1feb867-ee48-466c-b33a-e41a9b13b33d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:28.836994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:28.837927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:28.839548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:28.848773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 164/337] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:28.856318 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b98c4d-af38-47b4-900d-5509a1bee8dc None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:29.027610 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296715c3-7e52-4159-980e-774d16092cfe None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.027927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296715c3-7e52-4159-980e-774d16092cfe None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.028165 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296715c3-7e52-4159-980e-774d16092cfe None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.029854 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 165/338] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:29.035550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b98c4d-af38-47b4-900d-5509a1bee8dc None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:29.055959 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326034d-06e2-4980-96b9-19ad47b97699 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.056279 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326034d-06e2-4980-96b9-19ad47b97699 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.056542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9326034d-06e2-4980-96b9-19ad47b97699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.088918 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8069f8-2316-49f9-876d-1b49d1aa2ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:29.089600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8069f8-2316-49f9-876d-1b49d1aa2ecd None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.089884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8069f8-2316-49f9-876d-1b49d1aa2ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.090162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8069f8-2316-49f9-876d-1b49d1aa2ecd None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.090919 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8069f8-2316-49f9-876d-1b49d1aa2ecd None admin] RBAC: Adding query filter params (name=member) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:29.091223 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8069f8-2316-49f9-876d-1b49d1aa2ecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:29.092747 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8069f8-2316-49f9-876d-1b49d1aa2ecd None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:29.096770 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-9326034d-06e2-4980-96b9-19ad47b97699 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:29.097979 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-9326034d-06e2-4980-96b9-19ad47b97699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:29.114163 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 174/339] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:29.122357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d798eb-b4e9-4c7d-bbaa-5d0c96deff04 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.122621 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d798eb-b4e9-4c7d-bbaa-5d0c96deff04 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.122932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d798eb-b4e9-4c7d-bbaa-5d0c96deff04 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.124633 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 175/340] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:29.126006 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d798eb-b4e9-4c7d-bbaa-5d0c96deff04 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:29.214815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 166/341] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:29.216155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b58c8-8575-4fc4-ad9f-e72586398591 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.216475 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b58c8-8575-4fc4-ad9f-e72586398591 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.216754 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b58c8-8575-4fc4-ad9f-e72586398591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.250845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-973b58c8-8575-4fc4-ad9f-e72586398591 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:29.252254 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-973b58c8-8575-4fc4-ad9f-e72586398591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:29.254706 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:29.284834 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:29.391066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 167/342] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:29.391655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b58c8-8575-4fc4-ad9f-e72586398591 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:29.392244 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d798eb-b4e9-4c7d-bbaa-5d0c96deff04 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:29.450062 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98bc6a-c315-4aa2-9d26-3d0e60dfb27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:29.450834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98bc6a-c315-4aa2-9d26-3d0e60dfb27d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.451016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98bc6a-c315-4aa2-9d26-3d0e60dfb27d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.451283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98bc6a-c315-4aa2-9d26-3d0e60dfb27d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.568851 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b58c8-8575-4fc4-ad9f-e72586398591 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:29.610472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98bc6a-c315-4aa2-9d26-3d0e60dfb27d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:29.612294 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98bc6a-c315-4aa2-9d26-3d0e60dfb27d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:29.620380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60562db4-6d07-460b-b47c-75528f108929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:29.620380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562db4-6d07-460b-b47c-75528f108929 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.620380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562db4-6d07-460b-b47c-75528f108929 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.620380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60562db4-6d07-460b-b47c-75528f108929 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.633629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60562db4-6d07-460b-b47c-75528f108929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-60562db4-6d07-460b-b47c-75528f108929 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:29.642249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:29.643181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 19:46:29.643973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:29.648861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 168/343] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:29.649885 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60562db4-6d07-460b-b47c-75528f108929 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:29.655194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:29.720693 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:29.804175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 176/344] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:29.806008 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98bc6a-c315-4aa2-9d26-3d0e60dfb27d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:29.826417 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60562db4-6d07-460b-b47c-75528f108929 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:29.928973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ca487-9ccf-4f59-bb78-b914704d1073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:29.930343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca487-9ccf-4f59-bb78-b914704d1073 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:29.932059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca487-9ccf-4f59-bb78-b914704d1073 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:29.932672 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ca487-9ccf-4f59-bb78-b914704d1073 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:29.936941 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98bc6a-c315-4aa2-9d26-3d0e60dfb27d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:29.952823 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ca487-9ccf-4f59-bb78-b914704d1073 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-317ca487-9ccf-4f59-bb78-b914704d1073 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:29.961781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:29.962766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:29.963740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 169/345] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:46:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:29.964554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ca487-9ccf-4f59-bb78-b914704d1073 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:29.998430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea191dd9-73f4-44da-805c-8c52626769a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:29.999517 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea191dd9-73f4-44da-805c-8c52626769a0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:30.000688 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea191dd9-73f4-44da-805c-8c52626769a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:30.001182 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea191dd9-73f4-44da-805c-8c52626769a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:30.002451 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea191dd9-73f4-44da-805c-8c52626769a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:30.003237 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea191dd9-73f4-44da-805c-8c52626769a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:30.007772 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea191dd9-73f4-44da-805c-8c52626769a0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:30.024243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 177/346] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:30.025459 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452dffc-bb36-4629-9df9-3e50e852251d None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:30.025747 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452dffc-bb36-4629-9df9-3e50e852251d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:30.026019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452dffc-bb36-4629-9df9-3e50e852251d None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:30.027375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 178/347] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:30.030515 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452dffc-bb36-4629-9df9-3e50e852251d None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:30.050117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:30.069478 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:30.119598 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ca487-9ccf-4f59-bb78-b914704d1073 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:30.161898 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452dffc-bb36-4629-9df9-3e50e852251d None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:30.171439 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7a7b6-1dab-48e8-8ce2-334b291a5a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:30.173316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7a7b6-1dab-48e8-8ce2-334b291a5a6f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:30.173836 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7a7b6-1dab-48e8-8ce2-334b291a5a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:30.174856 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7a7b6-1dab-48e8-8ce2-334b291a5a6f None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:30.176515 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7a7b6-1dab-48e8-8ce2-334b291a5a6f None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:30.179353 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7a7b6-1dab-48e8-8ce2-334b291a5a6f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:30.181840 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 170/348] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:29 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:30.183238 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7a7b6-1dab-48e8-8ce2-334b291a5a6f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:30.214384 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9024a7a-36f6-4ae0-b1b6-4011587da070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:30.215690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9024a7a-36f6-4ae0-b1b6-4011587da070 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:30.216072 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9024a7a-36f6-4ae0-b1b6-4011587da070 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:30.216476 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9024a7a-36f6-4ae0-b1b6-4011587da070 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:30.217361 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9024a7a-36f6-4ae0-b1b6-4011587da070 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:30.217763 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9024a7a-36f6-4ae0-b1b6-4011587da070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:30.220414 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9024a7a-36f6-4ae0-b1b6-4011587da070 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:30.232845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 179/349] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:30.235832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d83a0-9064-4df9-bbdf-8a2904014e57 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:30.236452 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d83a0-9064-4df9-bbdf-8a2904014e57 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:30.236838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d83a0-9064-4df9-bbdf-8a2904014e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:30.268529 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-778d83a0-9064-4df9-bbdf-8a2904014e57 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:30.269960 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-778d83a0-9064-4df9-bbdf-8a2904014e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:30.358589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7a7b6-1dab-48e8-8ce2-334b291a5a6f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:30.368786 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 180/350] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:30.372422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d83a0-9064-4df9-bbdf-8a2904014e57 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:30.392753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:30.433962 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:30.440327 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd4f09-b03a-4edb-bf50-d3ac87787d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:30.440327 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd4f09-b03a-4edb-bf50-d3ac87787d2c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:30.440586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd4f09-b03a-4edb-bf50-d3ac87787d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:30.440861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fd4f09-b03a-4edb-bf50-d3ac87787d2c None admin] PATH_INFO: `/v3/users/ce10c9e8d17a4af792860121d2bc0cb9` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:30.443126 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd4f09-b03a-4edb-bf50-d3ac87787d2c None admin] RBAC: Authorizing `identity:get_user(user_id=ce10c9e8d17a4af792860121d2bc0cb9)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:30.462866 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fd4f09-b03a-4edb-bf50-d3ac87787d2c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:30.462866 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 171/351] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:46:30 2022] GET /identity/v3/users/ce10c9e8d17a4af792860121d2bc0cb9 => generated 293 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:30.462866 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd4f09-b03a-4edb-bf50-d3ac87787d2c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:30.542479 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d83a0-9064-4df9-bbdf-8a2904014e57 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:30.592201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef5533-d4fc-4df8-a895-79351aeabb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:30.593489 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef5533-d4fc-4df8-a895-79351aeabb66 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:30.594417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef5533-d4fc-4df8-a895-79351aeabb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:30.594745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eef5533-d4fc-4df8-a895-79351aeabb66 None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:30.600024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fd4f09-b03a-4edb-bf50-d3ac87787d2c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:30.601988 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef5533-d4fc-4df8-a895-79351aeabb66 None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:30.603449 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eef5533-d4fc-4df8-a895-79351aeabb66 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:30.606267 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 181/352] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:30 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:30.606407 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef5533-d4fc-4df8-a895-79351aeabb66 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:30.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1868b-e751-4d13-8186-1e7f3bc0b602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:30.658882 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1868b-e751-4d13-8186-1e7f3bc0b602 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:30.659162 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1868b-e751-4d13-8186-1e7f3bc0b602 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:30.659429 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1868b-e751-4d13-8186-1e7f3bc0b602 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:30.731781 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:30.761591 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:30.764638 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1868b-e751-4d13-8186-1e7f3bc0b602 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:30.767817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1868b-e751-4d13-8186-1e7f3bc0b602 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:30.796383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eef5533-d4fc-4df8-a895-79351aeabb66 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:30.840478 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c26ead-48e0-4486-9314-8ce41e0db82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:30.842071 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c26ead-48e0-4486-9314-8ce41e0db82b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:30.842536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c26ead-48e0-4486-9314-8ce41e0db82b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:30.842817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c26ead-48e0-4486-9314-8ce41e0db82b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:30.868390 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c26ead-48e0-4486-9314-8ce41e0db82b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-95c26ead-48e0-4486-9314-8ce41e0db82b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:30.877931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:30.878901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:30.879853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 182/353] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:30.881101 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c26ead-48e0-4486-9314-8ce41e0db82b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:30.911798 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 172/354] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:30.914219 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3d920-dbe3-405d-9418-401529752713 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:30.914795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3d920-dbe3-405d-9418-401529752713 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:30.915279 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3d920-dbe3-405d-9418-401529752713 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:30.919234 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 173/355] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:30.919234 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3d920-dbe3-405d-9418-401529752713 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:31.031313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c26ead-48e0-4486-9314-8ce41e0db82b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:31.060921 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:31.065442 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:31.068873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3d920-dbe3-405d-9418-401529752713 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:31.081166 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1de421-fbed-4b94-983f-ba234302c507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:31.081688 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de421-fbed-4b94-983f-ba234302c507 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:31.081966 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de421-fbed-4b94-983f-ba234302c507 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:31.082240 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de421-fbed-4b94-983f-ba234302c507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:31.083142 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1de421-fbed-4b94-983f-ba234302c507 None admin] RBAC: Adding query filter params (role.id=bd9a605a759046cf9bd1a8e9bb20d9d4, user.id=ce10c9e8d17a4af792860121d2bc0cb9, scope.system=all) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:31.083389 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1de421-fbed-4b94-983f-ba234302c507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:31.084943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1de421-fbed-4b94-983f-ba234302c507 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:31.099171 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 183/356] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 19:46:30 2022] GET /identity/v3/role_assignments?user.id=ce10c9e8d17a4af792860121d2bc0cb9&scope.system=all&role.id=bd9a605a759046cf9bd1a8e9bb20d9d4 => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:31.100057 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1de421-fbed-4b94-983f-ba234302c507 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:31.120916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f16944-e7a8-46fc-94da-0cacce11d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:31.123095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f16944-e7a8-46fc-94da-0cacce11d062 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:31.123095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f16944-e7a8-46fc-94da-0cacce11d062 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:31.123095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f16944-e7a8-46fc-94da-0cacce11d062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:31.123095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f16944-e7a8-46fc-94da-0cacce11d062 None admin] RBAC: Adding query filter params (group.id=d97e60b0d6d04bff825c3d55f198ffc4, role.id=6a41401a07c443f0b5acd2e189145727, scope.project.id=f14d8cdc0ed5452ca04a981594be5974) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:31.123380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f16944-e7a8-46fc-94da-0cacce11d062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:31.125481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f16944-e7a8-46fc-94da-0cacce11d062 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:31.134913 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 174/357] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 19:46:30 2022] GET /identity/v3/role_assignments?group.id=d97e60b0d6d04bff825c3d55f198ffc4&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=6a41401a07c443f0b5acd2e189145727 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:31.136129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f16944-e7a8-46fc-94da-0cacce11d062 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:31.237372 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1de421-fbed-4b94-983f-ba234302c507 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:31.265719 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f16944-e7a8-46fc-94da-0cacce11d062 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:31.282780 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867ce8c-28f1-44ef-9ef2-cf45d6c8d949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:31.283567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867ce8c-28f1-44ef-9ef2-cf45d6c8d949 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:31.283925 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867ce8c-28f1-44ef-9ef2-cf45d6c8d949 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:31.284277 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867ce8c-28f1-44ef-9ef2-cf45d6c8d949 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:31.285289 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867ce8c-28f1-44ef-9ef2-cf45d6c8d949 None admin] RBAC: Adding query filter params (name=member) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:31.285613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867ce8c-28f1-44ef-9ef2-cf45d6c8d949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:31.287250 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867ce8c-28f1-44ef-9ef2-cf45d6c8d949 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:31.298733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 184/358] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:31.300310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd01160-1409-4de7-a437-619db2ad6268 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:31.301712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd01160-1409-4de7-a437-619db2ad6268 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:31.302405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd01160-1409-4de7-a437-619db2ad6268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:31.314917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1481b-9023-4a8a-a306-e150d831eb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:31.316138 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1481b-9023-4a8a-a306-e150d831eb0f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:31.316639 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1481b-9023-4a8a-a306-e150d831eb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:31.317133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1481b-9023-4a8a-a306-e150d831eb0f None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:31.318794 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1481b-9023-4a8a-a306-e150d831eb0f None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:31.320687 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1481b-9023-4a8a-a306-e150d831eb0f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:31.323054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 175/359] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:31 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:31.323259 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-7cd01160-1409-4de7-a437-619db2ad6268 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:31.324927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92b5d0-b812-4c4f-84ed-6840999ea094 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:31.325114 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-7cd01160-1409-4de7-a437-619db2ad6268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:31.325614 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92b5d0-b812-4c4f-84ed-6840999ea094 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:31.326076 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92b5d0-b812-4c4f-84ed-6840999ea094 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:31.328123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 176/360] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:31.329279 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92b5d0-b812-4c4f-84ed-6840999ea094 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:31.383458 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:31.386725 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:31.424892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 185/361] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:31.425865 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd01160-1409-4de7-a437-619db2ad6268 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:31.473196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92b5d0-b812-4c4f-84ed-6840999ea094 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:31.526541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0887c4-a7ad-46f8-a052-3d8bdb83d39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:31.527205 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0887c4-a7ad-46f8-a052-3d8bdb83d39b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:31.527819 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0887c4-a7ad-46f8-a052-3d8bdb83d39b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:31.528101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0887c4-a7ad-46f8-a052-3d8bdb83d39b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:31.560472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd01160-1409-4de7-a437-619db2ad6268 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:31.602476 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2fb3b-dba9-44c7-8687-6a8cb20f0480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:31.603459 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2fb3b-dba9-44c7-8687-6a8cb20f0480 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:31.603742 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2fb3b-dba9-44c7-8687-6a8cb20f0480 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:31.604130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2fb3b-dba9-44c7-8687-6a8cb20f0480 None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:31.605386 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2fb3b-dba9-44c7-8687-6a8cb20f0480 None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:31.606822 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2fb3b-dba9-44c7-8687-6a8cb20f0480 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:31.609978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 186/362] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:31 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:31.611998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1feb72-d6ba-4d24-8866-41ff5bd7bdea None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:31.612287 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1feb72-d6ba-4d24-8866-41ff5bd7bdea None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:31.612635 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1feb72-d6ba-4d24-8866-41ff5bd7bdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:31.618197 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0887c4-a7ad-46f8-a052-3d8bdb83d39b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:31.619800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0887c4-a7ad-46f8-a052-3d8bdb83d39b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:31.632974 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-ca1feb72-d6ba-4d24-8866-41ff5bd7bdea None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:31.634173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-ca1feb72-d6ba-4d24-8866-41ff5bd7bdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:31.652987 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:31.690484 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:31.732690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 187/363] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:31.733650 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1feb72-d6ba-4d24-8866-41ff5bd7bdea None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:31.746073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 177/364] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:31.747428 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0887c4-a7ad-46f8-a052-3d8bdb83d39b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:31.878399 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1feb72-d6ba-4d24-8866-41ff5bd7bdea None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:31.893822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0887c4-a7ad-46f8-a052-3d8bdb83d39b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:31.925416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30069b7-f7fb-4715-a73e-4dceabfdeb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:31.925876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30069b7-f7fb-4715-a73e-4dceabfdeb68 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:31.926165 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30069b7-f7fb-4715-a73e-4dceabfdeb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:31.926437 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30069b7-f7fb-4715-a73e-4dceabfdeb68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:31.940380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:31.943373 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30069b7-f7fb-4715-a73e-4dceabfdeb68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:31.949077 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e516fe-a844-4c7f-8dbd-b7faeef15f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:31.949743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e516fe-a844-4c7f-8dbd-b7faeef15f5b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:31.950025 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e516fe-a844-4c7f-8dbd-b7faeef15f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:31.966890 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e516fe-a844-4c7f-8dbd-b7faeef15f5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-a30069b7-f7fb-4715-a73e-4dceabfdeb68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:31.967023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:31.967881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:31.969761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:31.970390 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e516fe-a844-4c7f-8dbd-b7faeef15f5b None admin] RBAC: Adding query filter params (group.id=d97e60b0d6d04bff825c3d55f198ffc4, role.id=cb116a5614cf4a48ad99840d95f96bb3, scope.project.id=f14d8cdc0ed5452ca04a981594be5974) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:31.970390 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e516fe-a844-4c7f-8dbd-b7faeef15f5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:31.970532 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 188/365] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:46:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:31.970532 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30069b7-f7fb-4715-a73e-4dceabfdeb68 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:31.970650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e516fe-a844-4c7f-8dbd-b7faeef15f5b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:31.970650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 178/366] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 19:46:31 2022] GET /identity/v3/role_assignments?group.id=d97e60b0d6d04bff825c3d55f198ffc4&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=cb116a5614cf4a48ad99840d95f96bb3 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:31.970650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e516fe-a844-4c7f-8dbd-b7faeef15f5b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:32.029282 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:32.106486 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30069b7-f7fb-4715-a73e-4dceabfdeb68 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:32.107021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e516fe-a844-4c7f-8dbd-b7faeef15f5b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:32.154327 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9869fb17-5fea-4bab-a730-6e2c5d6ffc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:32.155596 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869fb17-5fea-4bab-a730-6e2c5d6ffc77 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:32.155596 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869fb17-5fea-4bab-a730-6e2c5d6ffc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:32.155787 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9869fb17-5fea-4bab-a730-6e2c5d6ffc77 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:32.157863 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9869fb17-5fea-4bab-a730-6e2c5d6ffc77 None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:32.159552 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9869fb17-5fea-4bab-a730-6e2c5d6ffc77 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:32.161776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 189/367] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:31 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:32.162573 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a192d11-b660-413f-8aec-d4d95213a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:32.162935 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9869fb17-5fea-4bab-a730-6e2c5d6ffc77 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:32.163885 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a192d11-b660-413f-8aec-d4d95213a40b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:32.164382 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a192d11-b660-413f-8aec-d4d95213a40b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:32.164861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a192d11-b660-413f-8aec-d4d95213a40b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:32.181578 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a192d11-b660-413f-8aec-d4d95213a40b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-4a192d11-b660-413f-8aec-d4d95213a40b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:32.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:32.193930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:32.194764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:32.198060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 179/368] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:46:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:32.199496 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a192d11-b660-413f-8aec-d4d95213a40b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:32.275146 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:32.306581 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9869fb17-5fea-4bab-a730-6e2c5d6ffc77 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:32.324713 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:32.326084 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a192d11-b660-413f-8aec-d4d95213a40b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:32.354643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f438c2-3d63-45bd-a753-adbbdfa6b12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:32.355603 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f438c2-3d63-45bd-a753-adbbdfa6b12c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:32.356033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f438c2-3d63-45bd-a753-adbbdfa6b12c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:32.356497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f438c2-3d63-45bd-a753-adbbdfa6b12c None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:32.357429 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f438c2-3d63-45bd-a753-adbbdfa6b12c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:32.357900 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f438c2-3d63-45bd-a753-adbbdfa6b12c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:32.359533 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f438c2-3d63-45bd-a753-adbbdfa6b12c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:32.369348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 190/369] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:32.370714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f438c2-3d63-45bd-a753-adbbdfa6b12c None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:32.381615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b957-e9ab-4fb8-be2e-08572e4954b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:32.382490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b957-e9ab-4fb8-be2e-08572e4954b8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:32.382777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b957-e9ab-4fb8-be2e-08572e4954b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:32.383052 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e13b957-e9ab-4fb8-be2e-08572e4954b8 None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:32.388159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13b957-e9ab-4fb8-be2e-08572e4954b8 None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:32.389691 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e13b957-e9ab-4fb8-be2e-08572e4954b8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:32.392287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 180/370] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:32 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:32.393108 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b957-e9ab-4fb8-be2e-08572e4954b8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:32.496455 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f438c2-3d63-45bd-a753-adbbdfa6b12c None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:32.512304 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e13b957-e9ab-4fb8-be2e-08572e4954b8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:32.537903 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8f5a8-5ef2-4990-a521-f05cfc01e7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:32.539004 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8f5a8-5ef2-4990-a521-f05cfc01e7cd None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:32.539512 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8f5a8-5ef2-4990-a521-f05cfc01e7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:32.539991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8f5a8-5ef2-4990-a521-f05cfc01e7cd None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:32.541023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8f5a8-5ef2-4990-a521-f05cfc01e7cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:32.541323 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:32.541756 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8f5a8-5ef2-4990-a521-f05cfc01e7cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:32.543336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8f5a8-5ef2-4990-a521-f05cfc01e7cd None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:32.554250 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da07018-91b4-4c66-b225-5c72ebb4e1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:32.554904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da07018-91b4-4c66-b225-5c72ebb4e1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:32.555192 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da07018-91b4-4c66-b225-5c72ebb4e1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:32.555460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da07018-91b4-4c66-b225-5c72ebb4e1a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:32.556273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 191/371] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:32.557389 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8f5a8-5ef2-4990-a521-f05cfc01e7cd None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:32.588693 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:32.633588 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da07018-91b4-4c66-b225-5c72ebb4e1a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:32.635086 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da07018-91b4-4c66-b225-5c72ebb4e1a4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:32.670659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8f5a8-5ef2-4990-a521-f05cfc01e7cd None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:32.708992 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6151af8-8944-4a75-a574-67da118c4471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:32.710009 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6151af8-8944-4a75-a574-67da118c4471 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:32.710520 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6151af8-8944-4a75-a574-67da118c4471 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:32.711025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6151af8-8944-4a75-a574-67da118c4471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:32.712075 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6151af8-8944-4a75-a574-67da118c4471 None admin] RBAC: Adding query filter params (group.id=d97e60b0d6d04bff825c3d55f198ffc4, role.id=6a41401a07c443f0b5acd2e189145727, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:32.712632 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6151af8-8944-4a75-a574-67da118c4471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:32.714411 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6151af8-8944-4a75-a574-67da118c4471 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:32.720800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 192/372] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 19:46:32 2022] GET /identity/v3/role_assignments?group.id=d97e60b0d6d04bff825c3d55f198ffc4&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=6a41401a07c443f0b5acd2e189145727 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:32.721901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6151af8-8944-4a75-a574-67da118c4471 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:32.744982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 181/373] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:32.751317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da07018-91b4-4c66-b225-5c72ebb4e1a4 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:32.779488 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:32.828434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:32.839939 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6151af8-8944-4a75-a574-67da118c4471 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:32.863771 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da07018-91b4-4c66-b225-5c72ebb4e1a4 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:32.881738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e6499-ec85-492c-8d1f-378262a50caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:32.883042 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e6499-ec85-492c-8d1f-378262a50caf None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:32.883533 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e6499-ec85-492c-8d1f-378262a50caf None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:32.884021 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e6499-ec85-492c-8d1f-378262a50caf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:32.902775 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04603832-8348-41e2-843d-6d40b5ae3974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:32.903630 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04603832-8348-41e2-843d-6d40b5ae3974 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:32.903913 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04603832-8348-41e2-843d-6d40b5ae3974 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:32.904188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04603832-8348-41e2-843d-6d40b5ae3974 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:32.907706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04603832-8348-41e2-843d-6d40b5ae3974 None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:32.909211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04603832-8348-41e2-843d-6d40b5ae3974 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:32.911004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 182/374] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:32 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:32.916608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04603832-8348-41e2-843d-6d40b5ae3974 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:32.970856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e6499-ec85-492c-8d1f-378262a50caf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:32.972754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e6499-ec85-492c-8d1f-378262a50caf None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:33.019854 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:33.029119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04603832-8348-41e2-843d-6d40b5ae3974 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:33.068270 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ee6b3-f9b8-42be-9520-15c023b5547c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:33.069620 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ee6b3-f9b8-42be-9520-15c023b5547c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:33.070117 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ee6b3-f9b8-42be-9520-15c023b5547c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:33.070708 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ee6b3-f9b8-42be-9520-15c023b5547c None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:33.072540 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ee6b3-f9b8-42be-9520-15c023b5547c None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:33.074231 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ee6b3-f9b8-42be-9520-15c023b5547c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:33.076289 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 183/375] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:32 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:33.082125 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ee6b3-f9b8-42be-9520-15c023b5547c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:33.085450 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:33.097989 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 193/376] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:33.104090 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e6499-ec85-492c-8d1f-378262a50caf None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:33.194708 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ee6b3-f9b8-42be-9520-15c023b5547c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:33.222654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e6499-ec85-492c-8d1f-378262a50caf None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:33.233363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228eb6a-6c3b-468c-88af-3a3c873fce77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:33.234994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228eb6a-6c3b-468c-88af-3a3c873fce77 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:33.234994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228eb6a-6c3b-468c-88af-3a3c873fce77 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:33.235437 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228eb6a-6c3b-468c-88af-3a3c873fce77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:33.236415 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228eb6a-6c3b-468c-88af-3a3c873fce77 None admin] RBAC: Adding query filter params (group.id=d97e60b0d6d04bff825c3d55f198ffc4, role.id=cb116a5614cf4a48ad99840d95f96bb3, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:33.236835 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228eb6a-6c3b-468c-88af-3a3c873fce77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:33.238451 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228eb6a-6c3b-468c-88af-3a3c873fce77 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:33.244990 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 184/377] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 19:46:33 2022] GET /identity/v3/role_assignments?group.id=d97e60b0d6d04bff825c3d55f198ffc4&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=cb116a5614cf4a48ad99840d95f96bb3 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:33.266082 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf9f70-3445-4064-8eeb-bb69960c37f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:33.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf9f70-3445-4064-8eeb-bb69960c37f7 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:33.267760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf9f70-3445-4064-8eeb-bb69960c37f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:33.268188 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf9f70-3445-4064-8eeb-bb69960c37f7 None admin] PATH_INFO: `/v3/projects/c59a6214d8424d94bb73ea602efdca3f` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:33.269749 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf9f70-3445-4064-8eeb-bb69960c37f7 None admin] RBAC: Authorizing `identity:get_project(project_id=c59a6214d8424d94bb73ea602efdca3f)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:33.271490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacf9f70-3445-4064-8eeb-bb69960c37f7 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:33.273966 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 194/378] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:33 2022] GET /identity/v3/projects/c59a6214d8424d94bb73ea602efdca3f => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:33.279593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228eb6a-6c3b-468c-88af-3a3c873fce77 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:33.315663 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:33.434396 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228eb6a-6c3b-468c-88af-3a3c873fce77 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:33.480742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a64c3a-ccc0-4d6b-82ff-b3108d97ac18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:33.481984 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a64c3a-ccc0-4d6b-82ff-b3108d97ac18 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:33.482454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a64c3a-ccc0-4d6b-82ff-b3108d97ac18 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:33.482880 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a64c3a-ccc0-4d6b-82ff-b3108d97ac18 None admin] PATH_INFO: `/v3/groups/4f22169376e44f31b711e5470ca9f32e` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:33.491178 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a64c3a-ccc0-4d6b-82ff-b3108d97ac18 None admin] RBAC: Authorizing `identity:get_group(group_id=4f22169376e44f31b711e5470ca9f32e)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:33.499523 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a64c3a-ccc0-4d6b-82ff-b3108d97ac18 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:33.499523 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 185/379] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:33 2022] GET /identity/v3/groups/4f22169376e44f31b711e5470ca9f32e => generated 229 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:33.503563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf9f70-3445-4064-8eeb-bb69960c37f7 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:33.578025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:33.625575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf9f70-3445-4064-8eeb-bb69960c37f7 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:33.663423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b5d51-ba25-4840-92c5-aa51ebc5e1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:33.664350 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b5d51-ba25-4840-92c5-aa51ebc5e1ca None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:33.664810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b5d51-ba25-4840-92c5-aa51ebc5e1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:33.665234 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b5d51-ba25-4840-92c5-aa51ebc5e1ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:33.666223 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b5d51-ba25-4840-92c5-aa51ebc5e1ca None admin] RBAC: Adding query filter params (group.id=4f22169376e44f31b711e5470ca9f32e, role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=c59a6214d8424d94bb73ea602efdca3f) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:33.666689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b5d51-ba25-4840-92c5-aa51ebc5e1ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:33.668355 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b5d51-ba25-4840-92c5-aa51ebc5e1ca None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:33.676015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 195/380] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 19:46:33 2022] GET /identity/v3/role_assignments?group.id=4f22169376e44f31b711e5470ca9f32e&scope.project.id=c59a6214d8424d94bb73ea602efdca3f&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:33.884935 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d471a-e10b-467f-bdd2-f16c0924330d None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:33.885785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d471a-e10b-467f-bdd2-f16c0924330d None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:33.886311 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d471a-e10b-467f-bdd2-f16c0924330d None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:33.888005 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 186/381] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:33.894504 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0daa5f-214b-45cc-bf3d-099e1dea7285 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:33.894797 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0daa5f-214b-45cc-bf3d-099e1dea7285 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:33.895072 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0daa5f-214b-45cc-bf3d-099e1dea7285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:33.913462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-7d0daa5f-214b-45cc-bf3d-099e1dea7285 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:33.914669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-7d0daa5f-214b-45cc-bf3d-099e1dea7285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:33.994282 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 196/382] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:34.032954 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d471a-e10b-467f-bdd2-f16c0924330d None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:34.148077 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:34.162833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d471a-e10b-467f-bdd2-f16c0924330d None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:34.205202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81021a-e92a-485e-a410-62643ce7831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:34.206029 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81021a-e92a-485e-a410-62643ce7831f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:34.206317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81021a-e92a-485e-a410-62643ce7831f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:34.206622 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81021a-e92a-485e-a410-62643ce7831f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:34.224929 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81021a-e92a-485e-a410-62643ce7831f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-db81021a-e92a-485e-a410-62643ce7831f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:34.232871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:34.233808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:34.235155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:34.237294 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 187/383] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:34.242465 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0daa5f-214b-45cc-bf3d-099e1dea7285 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:34.414328 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:34.526003 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0daa5f-214b-45cc-bf3d-099e1dea7285 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:34.569080 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b18b1e-e238-4ef7-9e61-9df51b597b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:34.570141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b18b1e-e238-4ef7-9e61-9df51b597b30 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:34.570624 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b18b1e-e238-4ef7-9e61-9df51b597b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:34.571096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b18b1e-e238-4ef7-9e61-9df51b597b30 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:34.572263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b18b1e-e238-4ef7-9e61-9df51b597b30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:34.572824 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b18b1e-e238-4ef7-9e61-9df51b597b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:34.574449 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b18b1e-e238-4ef7-9e61-9df51b597b30 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:34.588979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 197/384] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:34.595647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81021a-e92a-485e-a410-62643ce7831f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:34.751208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81021a-e92a-485e-a410-62643ce7831f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:34.801404 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e88c4a-04aa-4a66-8027-5dce38ee6721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:34.802550 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88c4a-04aa-4a66-8027-5dce38ee6721 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:34.802836 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88c4a-04aa-4a66-8027-5dce38ee6721 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:34.803112 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88c4a-04aa-4a66-8027-5dce38ee6721 None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:34.804350 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e88c4a-04aa-4a66-8027-5dce38ee6721 None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:34.805843 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e88c4a-04aa-4a66-8027-5dce38ee6721 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:34.808542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 188/385] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:34 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:34.813381 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b18b1e-e238-4ef7-9e61-9df51b597b30 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:34.964754 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eec99b-505d-4fe5-bf5e-381df0d47bb6 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:34.965266 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eec99b-505d-4fe5-bf5e-381df0d47bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:34.965373 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eec99b-505d-4fe5-bf5e-381df0d47bb6 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:34.966813 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 189/386] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:34.973312 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728cd6-ec69-4017-bedb-2a6b78f76b00 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:34.973925 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728cd6-ec69-4017-bedb-2a6b78f76b00 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:34.974403 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b728cd6-ec69-4017-bedb-2a6b78f76b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:34.979667 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b18b1e-e238-4ef7-9e61-9df51b597b30 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:35.002234 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:35.022977 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-9b728cd6-ec69-4017-bedb-2a6b78f76b00 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:35.028040 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-9b728cd6-ec69-4017-bedb-2a6b78f76b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:35.064083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58743e38-6af8-48ad-a82b-79145dddfbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:35.065635 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58743e38-6af8-48ad-a82b-79145dddfbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:35.066143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58743e38-6af8-48ad-a82b-79145dddfbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:35.066649 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58743e38-6af8-48ad-a82b-79145dddfbd9 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:35.071509 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58743e38-6af8-48ad-a82b-79145dddfbd9 None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:35.072838 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:35.074308 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58743e38-6af8-48ad-a82b-79145dddfbd9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:35.077673 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 198/387] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:34 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:35.084423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58743e38-6af8-48ad-a82b-79145dddfbd9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:35.213801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 190/388] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:35.248994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b728cd6-ec69-4017-bedb-2a6b78f76b00 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:35.321701 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58743e38-6af8-48ad-a82b-79145dddfbd9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:35.376216 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150f4fc-19a4-41cf-9162-25b98ff9d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:35.377063 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150f4fc-19a4-41cf-9162-25b98ff9d4be None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:35.377355 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150f4fc-19a4-41cf-9162-25b98ff9d4be None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:35.377633 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150f4fc-19a4-41cf-9162-25b98ff9d4be None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974/groups/d97e60b0d6d04bff825c3d55f198ffc4/roles/6a41401a07c443f0b5acd2e189145727` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:35.379632 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150f4fc-19a4-41cf-9162-25b98ff9d4be None admin] RBAC: Authorizing `identity:create_grant(project_id=f14d8cdc0ed5452ca04a981594be5974, group_id=d97e60b0d6d04bff825c3d55f198ffc4, role_id=6a41401a07c443f0b5acd2e189145727)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:35.381203 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7150f4fc-19a4-41cf-9162-25b98ff9d4be None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:35.419999 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b728cd6-ec69-4017-bedb-2a6b78f76b00 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:35.456942 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 199/389] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 19:46:35 2022] PUT /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974/groups/d97e60b0d6d04bff825c3d55f198ffc4/roles/6a41401a07c443f0b5acd2e189145727 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:35.469349 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0b1ae-e3c0-4a21-b653-d691863dce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:35.470246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0b1ae-e3c0-4a21-b653-d691863dce99 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:35.470531 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0b1ae-e3c0-4a21-b653-d691863dce99 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:35.470855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0b1ae-e3c0-4a21-b653-d691863dce99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:35.483650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0b1ae-e3c0-4a21-b653-d691863dce99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-44a0b1ae-e3c0-4a21-b653-d691863dce99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:35.493299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:35.494354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:35.495698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 191/390] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:46:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:35.501497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150f4fc-19a4-41cf-9162-25b98ff9d4be None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:35.569941 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:35.656530 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150f4fc-19a4-41cf-9162-25b98ff9d4be None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:35.704098 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc685d-f764-47a2-9839-c116e1ccaade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:35.704821 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc685d-f764-47a2-9839-c116e1ccaade None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:35.705163 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc685d-f764-47a2-9839-c116e1ccaade None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:35.705450 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc685d-f764-47a2-9839-c116e1ccaade None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:35.706215 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc685d-f764-47a2-9839-c116e1ccaade None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:35.706522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc685d-f764-47a2-9839-c116e1ccaade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:35.707909 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc685d-f764-47a2-9839-c116e1ccaade None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:35.721632 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 200/391] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:35.727811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0b1ae-e3c0-4a21-b653-d691863dce99 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:35.750223 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:35.918571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0b1ae-e3c0-4a21-b653-d691863dce99 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:35.962309 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb307682-6cb6-4a4d-83a9-bc5566d9528e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:35.963189 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb307682-6cb6-4a4d-83a9-bc5566d9528e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:35.963472 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb307682-6cb6-4a4d-83a9-bc5566d9528e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:35.963757 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb307682-6cb6-4a4d-83a9-bc5566d9528e None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:35.967505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb307682-6cb6-4a4d-83a9-bc5566d9528e None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:35.969539 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb307682-6cb6-4a4d-83a9-bc5566d9528e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:35.971577 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 192/392] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:35 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:35.977207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc685d-f764-47a2-9839-c116e1ccaade None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:35.992285 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ee1a7-6822-46f3-8aff-461759685e2c None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:35.992933 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ee1a7-6822-46f3-8aff-461759685e2c None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:35.993433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ee1a7-6822-46f3-8aff-461759685e2c None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:35.995073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 193/393] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:36.003070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25d15f-8c7e-47f9-8e2f-f835ece300dd None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:36.003702 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25d15f-8c7e-47f9-8e2f-f835ece300dd None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:36.004179 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25d15f-8c7e-47f9-8e2f-f835ece300dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:36.022071 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-ac25d15f-8c7e-47f9-8e2f-f835ece300dd None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:36.023699 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-ac25d15f-8c7e-47f9-8e2f-f835ece300dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:36.132440 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 194/394] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:36.154690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc685d-f764-47a2-9839-c116e1ccaade None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:36.167142 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:36.170591 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25d15f-8c7e-47f9-8e2f-f835ece300dd None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:36.181844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:36.205637 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1d4f2-c140-4407-b9fc-16d6c67fa523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:36.206572 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1d4f2-c140-4407-b9fc-16d6c67fa523 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:36.206866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1d4f2-c140-4407-b9fc-16d6c67fa523 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:36.207233 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1d4f2-c140-4407-b9fc-16d6c67fa523 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:36.208387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1d4f2-c140-4407-b9fc-16d6c67fa523 None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:36.213358 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1d4f2-c140-4407-b9fc-16d6c67fa523 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:36.220022 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 201/395] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:35 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:36.233069 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1d4f2-c140-4407-b9fc-16d6c67fa523 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:36.303825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25d15f-8c7e-47f9-8e2f-f835ece300dd None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:36.345866 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a54df-ae32-4491-b437-bd370f0471b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:36.346967 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a54df-ae32-4491-b437-bd370f0471b0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:36.347481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a54df-ae32-4491-b437-bd370f0471b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:36.347929 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a54df-ae32-4491-b437-bd370f0471b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:36.362893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a54df-ae32-4491-b437-bd370f0471b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-a94a54df-ae32-4491-b437-bd370f0471b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:36.372736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:36.373688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:36.374593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:36.377649 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 195/396] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:36.384465 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a54df-ae32-4491-b437-bd370f0471b0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:36.440426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1d4f2-c140-4407-b9fc-16d6c67fa523 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:36.458314 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:36.480802 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f77d3f-84f4-4e61-97eb-777a1d430ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:36.481846 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f77d3f-84f4-4e61-97eb-777a1d430ec3 None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:36.482487 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f77d3f-84f4-4e61-97eb-777a1d430ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:36.482487 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f77d3f-84f4-4e61-97eb-777a1d430ec3 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974/groups/d97e60b0d6d04bff825c3d55f198ffc4/roles/cb116a5614cf4a48ad99840d95f96bb3` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:36.485653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f77d3f-84f4-4e61-97eb-777a1d430ec3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f14d8cdc0ed5452ca04a981594be5974, group_id=d97e60b0d6d04bff825c3d55f198ffc4, role_id=cb116a5614cf4a48ad99840d95f96bb3)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:36.487205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f77d3f-84f4-4e61-97eb-777a1d430ec3 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:36.505708 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a54df-ae32-4491-b437-bd370f0471b0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:36.545278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaab341-a869-4074-a92b-b8429fa21269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:36.545974 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaab341-a869-4074-a92b-b8429fa21269 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:36.546273 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaab341-a869-4074-a92b-b8429fa21269 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:36.547147 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaab341-a869-4074-a92b-b8429fa21269 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:36.547971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaab341-a869-4074-a92b-b8429fa21269 None admin] RBAC: Adding query filter params (name=member) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:36.548321 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaab341-a869-4074-a92b-b8429fa21269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:36.549755 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaab341-a869-4074-a92b-b8429fa21269 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:36.567810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 196/397] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:36.567810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036b16f-e28f-4583-8f9c-e4784fb88fb4 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:36.567810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036b16f-e28f-4583-8f9c-e4784fb88fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:36.567810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036b16f-e28f-4583-8f9c-e4784fb88fb4 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:36.567810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 197/398] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:36.571803 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 202/399] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 19:46:36 2022] PUT /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974/groups/d97e60b0d6d04bff825c3d55f198ffc4/roles/cb116a5614cf4a48ad99840d95f96bb3 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:36.574302 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a74fc-92b8-42ea-9c78-bfb64d03f57f None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:36.574655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a74fc-92b8-42ea-9c78-bfb64d03f57f None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:36.575036 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a74fc-92b8-42ea-9c78-bfb64d03f57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:36.586518 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f77d3f-84f4-4e61-97eb-777a1d430ec3 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:36.600367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-da1a74fc-92b8-42ea-9c78-bfb64d03f57f None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:36.600367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-da1a74fc-92b8-42ea-9c78-bfb64d03f57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:36.662059 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:36.728385 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 198/400] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:36.804905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f77d3f-84f4-4e61-97eb-777a1d430ec3 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:36.817171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a74fc-92b8-42ea-9c78-bfb64d03f57f None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:36.854045 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a56f3-9de5-48b4-84a7-6e96594f51f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:36.854976 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a56f3-9de5-48b4-84a7-6e96594f51f4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:36.855218 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a56f3-9de5-48b4-84a7-6e96594f51f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:36.855477 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a56f3-9de5-48b4-84a7-6e96594f51f4 None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:36.857630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a56f3-9de5-48b4-84a7-6e96594f51f4 None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:36.859825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a56f3-9de5-48b4-84a7-6e96594f51f4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:36.865877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 203/401] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:36 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:36.867863 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a56f3-9de5-48b4-84a7-6e96594f51f4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:36.898763 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:36.974978 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a74fc-92b8-42ea-9c78-bfb64d03f57f None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:36.998274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:37.003022 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a56f3-9de5-48b4-84a7-6e96594f51f4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:37.016498 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7f6e7-386d-4197-800d-5f9edf822179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:37.020973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f6e7-386d-4197-800d-5f9edf822179 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:37.020973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f6e7-386d-4197-800d-5f9edf822179 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:37.020973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7f6e7-386d-4197-800d-5f9edf822179 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:37.030168 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7f6e7-386d-4197-800d-5f9edf822179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-61a7f6e7-386d-4197-800d-5f9edf822179 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:37.039388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:37.040521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:37.041492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:37.043805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 199/402] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:46:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:37.045262 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde13dab-cfa3-4660-9f5a-6ec99526dba2 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:37.045699 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde13dab-cfa3-4660-9f5a-6ec99526dba2 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:37.046119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde13dab-cfa3-4660-9f5a-6ec99526dba2 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:37.046571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4d16f-ad2d-4fe7-ac41-633a30fb45a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:37.047497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4d16f-ad2d-4fe7-ac41-633a30fb45a5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:37.047848 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4d16f-ad2d-4fe7-ac41-633a30fb45a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:37.048140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 200/403] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:37.056373 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4d16f-ad2d-4fe7-ac41-633a30fb45a5 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:37.056586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde13dab-cfa3-4660-9f5a-6ec99526dba2 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:37.056667 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4d16f-ad2d-4fe7-ac41-633a30fb45a5 None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:37.056667 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4d16f-ad2d-4fe7-ac41-633a30fb45a5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:37.056667 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 204/404] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:36 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:37.061953 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534940a3-a98b-4d06-a294-a8ee16ee99bd None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:37.062406 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534940a3-a98b-4d06-a294-a8ee16ee99bd None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:37.062844 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534940a3-a98b-4d06-a294-a8ee16ee99bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:37.081001 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-534940a3-a98b-4d06-a294-a8ee16ee99bd None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:37.082191 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-534940a3-a98b-4d06-a294-a8ee16ee99bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:37.168637 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 205/405] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:37.169651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534940a3-a98b-4d06-a294-a8ee16ee99bd None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:37.181452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde13dab-cfa3-4660-9f5a-6ec99526dba2 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:37.199410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:37.219939 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc3594-5ae8-455d-b3ed-1f4bc0d0f0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:37.220942 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc3594-5ae8-455d-b3ed-1f4bc0d0f0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:37.221478 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc3594-5ae8-455d-b3ed-1f4bc0d0f0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:37.221936 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc3594-5ae8-455d-b3ed-1f4bc0d0f0c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:37.222928 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc3594-5ae8-455d-b3ed-1f4bc0d0f0c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:37.223373 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc3594-5ae8-455d-b3ed-1f4bc0d0f0c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:37.224921 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc3594-5ae8-455d-b3ed-1f4bc0d0f0c5 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:37.241565 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 201/406] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:37.243150 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc3594-5ae8-455d-b3ed-1f4bc0d0f0c5 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:37.302315 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534940a3-a98b-4d06-a294-a8ee16ee99bd None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:37.304727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:37.338566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02434329-5978-41d4-9f61-21e9d07e85dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:37.339729 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02434329-5978-41d4-9f61-21e9d07e85dd None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:37.340024 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02434329-5978-41d4-9f61-21e9d07e85dd None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:37.340305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02434329-5978-41d4-9f61-21e9d07e85dd None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/groups/d97e60b0d6d04bff825c3d55f198ffc4/roles/6a41401a07c443f0b5acd2e189145727` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:37.343935 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02434329-5978-41d4-9f61-21e9d07e85dd None admin] RBAC: Authorizing `identity:create_grant(project_id=03030ff4e6b54acfb48d93cd4178fd61, group_id=d97e60b0d6d04bff825c3d55f198ffc4, role_id=6a41401a07c443f0b5acd2e189145727)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:37.345070 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02434329-5978-41d4-9f61-21e9d07e85dd None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:37.358657 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc3594-5ae8-455d-b3ed-1f4bc0d0f0c5 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:37.395627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ca525-369d-4cfd-b76a-26ff23117b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:37.396457 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ca525-369d-4cfd-b76a-26ff23117b2d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:37.396769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ca525-369d-4cfd-b76a-26ff23117b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:37.397215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ca525-369d-4cfd-b76a-26ff23117b2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:37.408884 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ca525-369d-4cfd-b76a-26ff23117b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-7b4ca525-369d-4cfd-b76a-26ff23117b2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:37.416904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:37.417810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:37.418586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 202/407] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:46:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:37.419359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ca525-369d-4cfd-b76a-26ff23117b2d None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:37.448441 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 206/408] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 19:46:37 2022] PUT /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/groups/d97e60b0d6d04bff825c3d55f198ffc4/roles/6a41401a07c443f0b5acd2e189145727 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:37.450070 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02434329-5978-41d4-9f61-21e9d07e85dd None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:37.478750 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:37.625785 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02434329-5978-41d4-9f61-21e9d07e85dd None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:37.634545 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ca525-369d-4cfd-b76a-26ff23117b2d None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:37.667245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a702d5-3c2d-456e-a2ea-d4144faf88f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:37.667932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a702d5-3c2d-456e-a2ea-d4144faf88f3 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:37.668219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a702d5-3c2d-456e-a2ea-d4144faf88f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:37.668526 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a702d5-3c2d-456e-a2ea-d4144faf88f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:37.669299 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a702d5-3c2d-456e-a2ea-d4144faf88f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:37.669606 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a702d5-3c2d-456e-a2ea-d4144faf88f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:37.672066 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a702d5-3c2d-456e-a2ea-d4144faf88f3 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:37.689549 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 207/409] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:37.699418 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a702d5-3c2d-456e-a2ea-d4144faf88f3 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:37.721819 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13b14f-2c7c-41ab-943f-4b8b7fccecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:37.722934 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13b14f-2c7c-41ab-943f-4b8b7fccecc6 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:37.723296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13b14f-2c7c-41ab-943f-4b8b7fccecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:37.723675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13b14f-2c7c-41ab-943f-4b8b7fccecc6 None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:37.725163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13b14f-2c7c-41ab-943f-4b8b7fccecc6 None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:37.726651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13b14f-2c7c-41ab-943f-4b8b7fccecc6 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:37.728588 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 203/410] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:37 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:37.733252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:37.737650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13b14f-2c7c-41ab-943f-4b8b7fccecc6 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:37.851683 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a702d5-3c2d-456e-a2ea-d4144faf88f3 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:37.870462 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13b14f-2c7c-41ab-943f-4b8b7fccecc6 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:37.879882 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:37.898847 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eb311-364a-410f-834b-d74b2d921a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:37.899969 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb311-364a-410f-834b-d74b2d921a2d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:37.900251 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb311-364a-410f-834b-d74b2d921a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:37.900550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6eb311-364a-410f-834b-d74b2d921a2d None admin] PATH_INFO: `/v3/groups/4f22169376e44f31b711e5470ca9f32e` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:37.901965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6eb311-364a-410f-834b-d74b2d921a2d None admin] RBAC: Authorizing `identity:get_group(group_id=4f22169376e44f31b711e5470ca9f32e)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:37.904049 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6eb311-364a-410f-834b-d74b2d921a2d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:37.911357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 208/411] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:37 2022] GET /identity/v3/groups/4f22169376e44f31b711e5470ca9f32e => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:37.912962 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eb311-364a-410f-834b-d74b2d921a2d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:37.917923 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508732b-b16d-43e6-98f1-56974942d5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:37.918823 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508732b-b16d-43e6-98f1-56974942d5ee None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:37.919103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508732b-b16d-43e6-98f1-56974942d5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:37.919380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508732b-b16d-43e6-98f1-56974942d5ee None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:37.920929 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508732b-b16d-43e6-98f1-56974942d5ee None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:37.922511 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508732b-b16d-43e6-98f1-56974942d5ee None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:37.925055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 204/412] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:37 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:37.932365 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508732b-b16d-43e6-98f1-56974942d5ee None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:38.081706 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:38.094552 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6eb311-364a-410f-834b-d74b2d921a2d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:38.102544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508732b-b16d-43e6-98f1-56974942d5ee None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:38.131084 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871da7b-8d69-416a-a6a0-bd13a3738312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:38.131918 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871da7b-8d69-416a-a6a0-bd13a3738312 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:38.132465 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871da7b-8d69-416a-a6a0-bd13a3738312 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:38.132678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871da7b-8d69-416a-a6a0-bd13a3738312 None admin] PATH_INFO: `/v3/projects/c59a6214d8424d94bb73ea602efdca3f` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:38.134318 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871da7b-8d69-416a-a6a0-bd13a3738312 None admin] RBAC: Authorizing `identity:get_project(project_id=c59a6214d8424d94bb73ea602efdca3f)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:38.135992 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871da7b-8d69-416a-a6a0-bd13a3738312 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:38.138279 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 209/413] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:37 2022] GET /identity/v3/projects/c59a6214d8424d94bb73ea602efdca3f => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:38.139666 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26828193-01d1-4406-8bdc-a1ced91639f8 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:38.140082 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26828193-01d1-4406-8bdc-a1ced91639f8 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:38.140449 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26828193-01d1-4406-8bdc-a1ced91639f8 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:38.141957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 210/414] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:38.146472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26828193-01d1-4406-8bdc-a1ced91639f8 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:38.163135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c289d3-915e-4c98-ac09-42e2a09f300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:38.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c289d3-915e-4c98-ac09-42e2a09f300a None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:38.165026 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c289d3-915e-4c98-ac09-42e2a09f300a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:38.165319 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c289d3-915e-4c98-ac09-42e2a09f300a None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/groups/d97e60b0d6d04bff825c3d55f198ffc4/roles/cb116a5614cf4a48ad99840d95f96bb3` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:38.168072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c289d3-915e-4c98-ac09-42e2a09f300a None admin] RBAC: Authorizing `identity:create_grant(project_id=03030ff4e6b54acfb48d93cd4178fd61, group_id=d97e60b0d6d04bff825c3d55f198ffc4, role_id=cb116a5614cf4a48ad99840d95f96bb3)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:38.169596 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c289d3-915e-4c98-ac09-42e2a09f300a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:38.241618 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 205/415] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 19:46:37 2022] PUT /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61/groups/d97e60b0d6d04bff825c3d55f198ffc4/roles/cb116a5614cf4a48ad99840d95f96bb3 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:46:38.242895 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d2743-6a4f-4bc4-9597-a31497820aaa None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:38.243183 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d2743-6a4f-4bc4-9597-a31497820aaa None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:38.243456 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d2743-6a4f-4bc4-9597-a31497820aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:38.277829 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-716d2743-6a4f-4bc4-9597-a31497820aaa None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:38.279089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-716d2743-6a4f-4bc4-9597-a31497820aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:38.297376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26828193-01d1-4406-8bdc-a1ced91639f8 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:38.341185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:38.369792 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1201bd-e895-4a67-a489-ba449961cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:38.370749 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1201bd-e895-4a67-a489-ba449961cb9f None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:38.371035 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1201bd-e895-4a67-a489-ba449961cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:38.371308 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1201bd-e895-4a67-a489-ba449961cb9f None admin] PATH_INFO: `/v3/projects/c59a6214d8424d94bb73ea602efdca3f/groups/4f22169376e44f31b711e5470ca9f32e/roles/815bd48c3df0405babfd5b53430fd3ea` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:38.375485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1201bd-e895-4a67-a489-ba449961cb9f None admin] RBAC: Authorizing `identity:create_grant(project_id=c59a6214d8424d94bb73ea602efdca3f, group_id=4f22169376e44f31b711e5470ca9f32e, role_id=815bd48c3df0405babfd5b53430fd3ea)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:38.376969 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1201bd-e895-4a67-a489-ba449961cb9f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:38.411085 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 206/416] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:38.433022 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d2743-6a4f-4bc4-9597-a31497820aaa None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:38.481872 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 211/417] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 19:46:38 2022] PUT /identity/v3/projects/c59a6214d8424d94bb73ea602efdca3f/groups/4f22169376e44f31b711e5470ca9f32e/roles/815bd48c3df0405babfd5b53430fd3ea => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:46:38.570544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d2743-6a4f-4bc4-9597-a31497820aaa None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:38.614582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a853ca-f11f-4cff-9337-1f824290483b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:38.615783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a853ca-f11f-4cff-9337-1f824290483b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:38.616259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a853ca-f11f-4cff-9337-1f824290483b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:38.616751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a853ca-f11f-4cff-9337-1f824290483b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:38.629390 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a853ca-f11f-4cff-9337-1f824290483b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-57a853ca-f11f-4cff-9337-1f824290483b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:38.638024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:38.639001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:38.639785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:38.647309 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 207/418] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:46:38.649488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1201bd-e895-4a67-a489-ba449961cb9f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:38.685634 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:38.788371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1201bd-e895-4a67-a489-ba449961cb9f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:38.826893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef096ee4-b386-4bd6-b0b9-0a42ccd7c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:38.828184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef096ee4-b386-4bd6-b0b9-0a42ccd7c8eb None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:38.828432 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef096ee4-b386-4bd6-b0b9-0a42ccd7c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:38.828703 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef096ee4-b386-4bd6-b0b9-0a42ccd7c8eb None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:38.829463 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef096ee4-b386-4bd6-b0b9-0a42ccd7c8eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:38.829764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef096ee4-b386-4bd6-b0b9-0a42ccd7c8eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:38.831139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef096ee4-b386-4bd6-b0b9-0a42ccd7c8eb None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:38.840828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 212/419] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:38.849055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a853ca-f11f-4cff-9337-1f824290483b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:38.895845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:38.998044 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a853ca-f11f-4cff-9337-1f824290483b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:39.043931 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27241c-5de5-4417-a65c-996ece55046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:39.044864 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27241c-5de5-4417-a65c-996ece55046e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:39.045322 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27241c-5de5-4417-a65c-996ece55046e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:39.045746 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27241c-5de5-4417-a65c-996ece55046e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:39.138471 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27241c-5de5-4417-a65c-996ece55046e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:39.141885 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a27241c-5de5-4417-a65c-996ece55046e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:39.182531 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:39.307972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 208/420] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:39.314495 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef096ee4-b386-4bd6-b0b9-0a42ccd7c8eb None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:39.400912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aeb2b-172d-4dbb-b21a-b379c70f4d3c None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:39.401228 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aeb2b-172d-4dbb-b21a-b379c70f4d3c None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:39.401479 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aeb2b-172d-4dbb-b21a-b379c70f4d3c None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:39.403201 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 209/421] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:39.409880 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a842b-48df-428a-ba97-1c16154bad36 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:39.410077 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a842b-48df-428a-ba97-1c16154bad36 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:39.410356 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08a842b-48df-428a-ba97-1c16154bad36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:39.444752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:39.465577 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-a08a842b-48df-428a-ba97-1c16154bad36 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:39.466670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-a08a842b-48df-428a-ba97-1c16154bad36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:39.469738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef096ee4-b386-4bd6-b0b9-0a42ccd7c8eb None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:39.510638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045bda6-8630-4187-b6a4-e675192f992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:39.511809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045bda6-8630-4187-b6a4-e675192f992b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:39.512280 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045bda6-8630-4187-b6a4-e675192f992b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:39.513070 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045bda6-8630-4187-b6a4-e675192f992b None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:39.514769 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045bda6-8630-4187-b6a4-e675192f992b None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:39.516458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045bda6-8630-4187-b6a4-e675192f992b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:39.519024 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 213/422] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:39 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:39.529772 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045bda6-8630-4187-b6a4-e675192f992b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:39.664618 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 210/423] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:39.669582 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045bda6-8630-4187-b6a4-e675192f992b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:39.695852 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a842b-48df-428a-ba97-1c16154bad36 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:39.707866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446e7d3-f2bf-460c-a2b0-b916b61b2278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:39.709071 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446e7d3-f2bf-460c-a2b0-b916b61b2278 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:39.709646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446e7d3-f2bf-460c-a2b0-b916b61b2278 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:39.710096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446e7d3-f2bf-460c-a2b0-b916b61b2278 None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:39.714904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446e7d3-f2bf-460c-a2b0-b916b61b2278 None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:39.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446e7d3-f2bf-460c-a2b0-b916b61b2278 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:39.721399 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 214/424] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:39 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:39.726530 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446e7d3-f2bf-460c-a2b0-b916b61b2278 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:39.756712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:39.849121 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446e7d3-f2bf-460c-a2b0-b916b61b2278 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:39.882018 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:39.890776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563764c-0839-4be8-aca3-b5dee94a5379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:39.891736 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563764c-0839-4be8-aca3-b5dee94a5379 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:39.900502 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563764c-0839-4be8-aca3-b5dee94a5379 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:39.900502 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563764c-0839-4be8-aca3-b5dee94a5379 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:39.900502 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563764c-0839-4be8-aca3-b5dee94a5379 None admin] RBAC: Adding query filter params (group.id=d97e60b0d6d04bff825c3d55f198ffc4, role.id=6a41401a07c443f0b5acd2e189145727, scope.project.id=f14d8cdc0ed5452ca04a981594be5974) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:39.900502 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563764c-0839-4be8-aca3-b5dee94a5379 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:39.900502 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563764c-0839-4be8-aca3-b5dee94a5379 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:39.908495 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 215/425] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 19:46:39 2022] GET /identity/v3/role_assignments?group.id=d97e60b0d6d04bff825c3d55f198ffc4&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=6a41401a07c443f0b5acd2e189145727 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:40.035624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08a842b-48df-428a-ba97-1c16154bad36 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:40.086685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f4126-5812-4b3a-a03c-c95b6eaf8a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:40.087505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f4126-5812-4b3a-a03c-c95b6eaf8a70 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:40.087790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f4126-5812-4b3a-a03c-c95b6eaf8a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:40.088064 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9f4126-5812-4b3a-a03c-c95b6eaf8a70 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:40.108775 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9f4126-5812-4b3a-a03c-c95b6eaf8a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-4b9f4126-5812-4b3a-a03c-c95b6eaf8a70 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:40.117067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:40.117965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:40.118758 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 211/426] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:46:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:40.124523 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563764c-0839-4be8-aca3-b5dee94a5379 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:40.235160 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:40.258969 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563764c-0839-4be8-aca3-b5dee94a5379 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:40.304809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1500a11-cd0a-4644-b180-f3c578e0aeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:40.305703 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500a11-cd0a-4644-b180-f3c578e0aeed None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:40.306123 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500a11-cd0a-4644-b180-f3c578e0aeed None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:40.306527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1500a11-cd0a-4644-b180-f3c578e0aeed None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:40.307685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1500a11-cd0a-4644-b180-f3c578e0aeed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:40.308148 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1500a11-cd0a-4644-b180-f3c578e0aeed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:40.310311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1500a11-cd0a-4644-b180-f3c578e0aeed None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:40.324324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 216/427] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:40.331849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f4126-5812-4b3a-a03c-c95b6eaf8a70 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:40.465294 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9f4126-5812-4b3a-a03c-c95b6eaf8a70 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:40.480722 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:40.504394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55399bdf-006e-4cca-b20a-12b46279a295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:40.505063 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55399bdf-006e-4cca-b20a-12b46279a295 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:40.505345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55399bdf-006e-4cca-b20a-12b46279a295 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:40.505621 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55399bdf-006e-4cca-b20a-12b46279a295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:40.569260 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db814a63-8bb5-49f0-9499-535c0476ab4c None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:40.569477 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db814a63-8bb5-49f0-9499-535c0476ab4c None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:40.570308 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db814a63-8bb5-49f0-9499-535c0476ab4c None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:40.572125 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 217/428] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:40.582756 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26921b-3d1f-4df7-9141-b37c359bdca5 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:40.583192 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26921b-3d1f-4df7-9141-b37c359bdca5 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:40.583881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26921b-3d1f-4df7-9141-b37c359bdca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:40.596859 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55399bdf-006e-4cca-b20a-12b46279a295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:40.598360 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55399bdf-006e-4cca-b20a-12b46279a295 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:40.628661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-dd26921b-3d1f-4df7-9141-b37c359bdca5 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:40.630016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-dd26921b-3d1f-4df7-9141-b37c359bdca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:40.719466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 218/429] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:40.743557 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 212/430] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:40.749717 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26921b-3d1f-4df7-9141-b37c359bdca5 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:40.753544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55399bdf-006e-4cca-b20a-12b46279a295 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:40.776699 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:40.831942 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:40.868967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26921b-3d1f-4df7-9141-b37c359bdca5 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:40.892455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55399bdf-006e-4cca-b20a-12b46279a295 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:40.909879 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39f40b-2c01-4e8f-b779-25f19cef4904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:40.910692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39f40b-2c01-4e8f-b779-25f19cef4904 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:40.910979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39f40b-2c01-4e8f-b779-25f19cef4904 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:40.911254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39f40b-2c01-4e8f-b779-25f19cef4904 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:40.925360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39f40b-2c01-4e8f-b779-25f19cef4904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-8a39f40b-2c01-4e8f-b779-25f19cef4904 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:40.934236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:40.935274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:40.936047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 219/431] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:40.942689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39f40b-2c01-4e8f-b779-25f19cef4904 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:40.946827 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce933ea-c13d-480c-8022-2cd4f8873f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:40.948230 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce933ea-c13d-480c-8022-2cd4f8873f23 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:40.948230 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce933ea-c13d-480c-8022-2cd4f8873f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:40.948230 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce933ea-c13d-480c-8022-2cd4f8873f23 None admin] PATH_INFO: `/v3/projects/f14d8cdc0ed5452ca04a981594be5974` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:40.951002 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce933ea-c13d-480c-8022-2cd4f8873f23 None admin] RBAC: Authorizing `identity:get_project(project_id=f14d8cdc0ed5452ca04a981594be5974)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:40.952548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce933ea-c13d-480c-8022-2cd4f8873f23 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:40.955035 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 213/432] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:40 2022] GET /identity/v3/projects/f14d8cdc0ed5452ca04a981594be5974 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:40.961726 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce933ea-c13d-480c-8022-2cd4f8873f23 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:41.071838 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:41.072812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39f40b-2c01-4e8f-b779-25f19cef4904 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:41.096507 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce933ea-c13d-480c-8022-2cd4f8873f23 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:41.117898 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:41.127484 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb49b40-e5b5-4005-b756-554d862f439c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:41.128729 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb49b40-e5b5-4005-b756-554d862f439c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.129585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb49b40-e5b5-4005-b756-554d862f439c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.130350 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb49b40-e5b5-4005-b756-554d862f439c None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.131499 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb49b40-e5b5-4005-b756-554d862f439c None admin] RBAC: Adding query filter params (name=member) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:41.131993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb49b40-e5b5-4005-b756-554d862f439c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:41.134536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb49b40-e5b5-4005-b756-554d862f439c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:41.138660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642391a-b8cf-4d51-90e3-1cd5d2f39aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:41.139539 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642391a-b8cf-4d51-90e3-1cd5d2f39aba None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.139852 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642391a-b8cf-4d51-90e3-1cd5d2f39aba None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.140721 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642391a-b8cf-4d51-90e3-1cd5d2f39aba None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.142182 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642391a-b8cf-4d51-90e3-1cd5d2f39aba None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:41.143660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642391a-b8cf-4d51-90e3-1cd5d2f39aba None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:41.152811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 214/433] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:40 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:41.152811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42b4db-657e-497d-94ec-9e15fc85e3b9 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.152811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42b4db-657e-497d-94ec-9e15fc85e3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.152811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42b4db-657e-497d-94ec-9e15fc85e3b9 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.153345 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 220/435] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:46:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:41.153497 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 215/435] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:41.154887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a92c8-a5ce-41f5-bb76-02970609600d None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.155028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42b4db-657e-497d-94ec-9e15fc85e3b9 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:41.156286 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a92c8-a5ce-41f5-bb76-02970609600d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.156916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a92c8-a5ce-41f5-bb76-02970609600d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.179560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-e69a92c8-a5ce-41f5-bb76-02970609600d None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:41.180793 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-e69a92c8-a5ce-41f5-bb76-02970609600d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:41.254858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 221/436] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:41.255538 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a92c8-a5ce-41f5-bb76-02970609600d None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:41.276701 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42b4db-657e-497d-94ec-9e15fc85e3b9 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:41.318083 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8374e4-2298-4279-88b7-77c03b1e4cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:41.318755 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8374e4-2298-4279-88b7-77c03b1e4cad None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.319037 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8374e4-2298-4279-88b7-77c03b1e4cad None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.319391 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8374e4-2298-4279-88b7-77c03b1e4cad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.347856 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:41.371880 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a92c8-a5ce-41f5-bb76-02970609600d None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:41.397464 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:41.404709 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8374e4-2298-4279-88b7-77c03b1e4cad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:41.406764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8374e4-2298-4279-88b7-77c03b1e4cad None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:41.409625 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02abbe-1e9b-42b5-adb1-29778b25e0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:41.410302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02abbe-1e9b-42b5-adb1-29778b25e0ce None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.410589 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02abbe-1e9b-42b5-adb1-29778b25e0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.410860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02abbe-1e9b-42b5-adb1-29778b25e0ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.411714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02abbe-1e9b-42b5-adb1-29778b25e0ce None admin] RBAC: Adding query filter params (group.id=d97e60b0d6d04bff825c3d55f198ffc4, role.id=cb116a5614cf4a48ad99840d95f96bb3, scope.project.id=f14d8cdc0ed5452ca04a981594be5974) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:41.412023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02abbe-1e9b-42b5-adb1-29778b25e0ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:41.413526 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02abbe-1e9b-42b5-adb1-29778b25e0ce None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:41.419870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 222/437] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 19:46:41 2022] GET /identity/v3/role_assignments?group.id=d97e60b0d6d04bff825c3d55f198ffc4&scope.project.id=f14d8cdc0ed5452ca04a981594be5974&role.id=cb116a5614cf4a48ad99840d95f96bb3 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:41.420776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02abbe-1e9b-42b5-adb1-29778b25e0ce None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:41.544507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02abbe-1e9b-42b5-adb1-29778b25e0ce None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:41.545162 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 216/438] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:41.547535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338728e0-1cf2-4540-8cec-ddcf6830e4bc None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.547912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338728e0-1cf2-4540-8cec-ddcf6830e4bc None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.548222 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338728e0-1cf2-4540-8cec-ddcf6830e4bc None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.549875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 217/439] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:41.553386 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338728e0-1cf2-4540-8cec-ddcf6830e4bc None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:41.591244 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfffa33-0ae7-4855-a209-1e225ec69064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:41.592070 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfffa33-0ae7-4855-a209-1e225ec69064 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.592385 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfffa33-0ae7-4855-a209-1e225ec69064 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.592664 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfffa33-0ae7-4855-a209-1e225ec69064 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.604642 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfffa33-0ae7-4855-a209-1e225ec69064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-ecfffa33-0ae7-4855-a209-1e225ec69064 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:41.613426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:41.614354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:41.615229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 223/440] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:46:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:41.616462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b882d-05ed-424f-9d5d-552fc024ee85 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.616644 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b882d-05ed-424f-9d5d-552fc024ee85 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.616910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b882d-05ed-424f-9d5d-552fc024ee85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.625090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:41.635258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-9c0b882d-05ed-424f-9d5d-552fc024ee85 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:41.636651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-9c0b882d-05ed-424f-9d5d-552fc024ee85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:41.688882 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338728e0-1cf2-4540-8cec-ddcf6830e4bc None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:41.689453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:41.725012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 224/441] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:41.725012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b882d-05ed-424f-9d5d-552fc024ee85 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:41.730055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5cf13-8a1a-4bcd-a83e-16c3df77bad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:41.730929 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5cf13-8a1a-4bcd-a83e-16c3df77bad4 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.731212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5cf13-8a1a-4bcd-a83e-16c3df77bad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.731481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5cf13-8a1a-4bcd-a83e-16c3df77bad4 None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.732690 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5cf13-8a1a-4bcd-a83e-16c3df77bad4 None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:41.734674 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5cf13-8a1a-4bcd-a83e-16c3df77bad4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:41.736685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 218/442] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:41 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:41.741955 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5cf13-8a1a-4bcd-a83e-16c3df77bad4 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:41.848199 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b882d-05ed-424f-9d5d-552fc024ee85 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:41.866287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5cf13-8a1a-4bcd-a83e-16c3df77bad4 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:41.883905 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:41.885089 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9add860-7c44-4b08-96ef-2eb986163690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:41.885756 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9add860-7c44-4b08-96ef-2eb986163690 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.886038 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9add860-7c44-4b08-96ef-2eb986163690 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.886309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9add860-7c44-4b08-96ef-2eb986163690 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.887064 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9add860-7c44-4b08-96ef-2eb986163690 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:41.887365 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9add860-7c44-4b08-96ef-2eb986163690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:41.888764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9add860-7c44-4b08-96ef-2eb986163690 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:41.897476 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 225/443] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:41.901287 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9add860-7c44-4b08-96ef-2eb986163690 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:41.902248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e44b8-1ceb-4bf5-8e51-15a36b7e7160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:41.903112 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e44b8-1ceb-4bf5-8e51-15a36b7e7160 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:41.903398 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e44b8-1ceb-4bf5-8e51-15a36b7e7160 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:41.903677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e44b8-1ceb-4bf5-8e51-15a36b7e7160 None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:41.904980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e44b8-1ceb-4bf5-8e51-15a36b7e7160 None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:41.907011 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e44b8-1ceb-4bf5-8e51-15a36b7e7160 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:41.908669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 219/444] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:41 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:41.909433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e44b8-1ceb-4bf5-8e51-15a36b7e7160 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:41.943360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:42.004212 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9add860-7c44-4b08-96ef-2eb986163690 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:42.012305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e44b8-1ceb-4bf5-8e51-15a36b7e7160 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:42.039076 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbce24-331d-4950-8f35-d448e3543dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:42.039870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbce24-331d-4950-8f35-d448e3543dbc None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:42.040154 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbce24-331d-4950-8f35-d448e3543dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:42.040481 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cbce24-331d-4950-8f35-d448e3543dbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:42.046944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6879d0-3751-48f9-8531-f9d97299f11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:42.047596 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6879d0-3751-48f9-8531-f9d97299f11a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:42.047883 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6879d0-3751-48f9-8531-f9d97299f11a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:42.048160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6879d0-3751-48f9-8531-f9d97299f11a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:42.051490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cbce24-331d-4950-8f35-d448e3543dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-a2cbce24-331d-4950-8f35-d448e3543dbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:42.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:42.060501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:42.061333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 226/445] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:46:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:42.061990 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbce24-331d-4950-8f35-d448e3543dbc None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:42.109354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:42.122483 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6879d0-3751-48f9-8531-f9d97299f11a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:42.123913 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6879d0-3751-48f9-8531-f9d97299f11a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:42.159699 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:42.163556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cbce24-331d-4950-8f35-d448e3543dbc None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:42.197819 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f910a-89e3-4273-ba17-bf7b7b163bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:42.198528 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f910a-89e3-4273-ba17-bf7b7b163bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:42.198762 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f910a-89e3-4273-ba17-bf7b7b163bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:42.199040 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f910a-89e3-4273-ba17-bf7b7b163bf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:42.199884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f910a-89e3-4273-ba17-bf7b7b163bf5 None admin] RBAC: Adding query filter params (group.id=d97e60b0d6d04bff825c3d55f198ffc4, role.id=6a41401a07c443f0b5acd2e189145727, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:42.200201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f910a-89e3-4273-ba17-bf7b7b163bf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:42.201674 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f910a-89e3-4273-ba17-bf7b7b163bf5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:42.208069 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 227/446] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 19:46:42 2022] GET /identity/v3/role_assignments?group.id=d97e60b0d6d04bff825c3d55f198ffc4&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=6a41401a07c443f0b5acd2e189145727 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:42.209053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f910a-89e3-4273-ba17-bf7b7b163bf5 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:42.228166 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 220/447] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:42.234389 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6879d0-3751-48f9-8531-f9d97299f11a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:42.330949 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:42.352568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f910a-89e3-4273-ba17-bf7b7b163bf5 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:42.353057 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6879d0-3751-48f9-8531-f9d97299f11a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:42.393153 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c498ce-af1e-439a-8997-ef634b328f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:42.393369 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c9c0a-b5f1-4d9f-a1d3-f6849cc2702e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:42.393794 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c9c0a-b5f1-4d9f-a1d3-f6849cc2702e None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:42.394057 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c9c0a-b5f1-4d9f-a1d3-f6849cc2702e None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:42.394336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c9c0a-b5f1-4d9f-a1d3-f6849cc2702e None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:42.395103 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c9c0a-b5f1-4d9f-a1d3-f6849cc2702e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:42.395410 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c9c0a-b5f1-4d9f-a1d3-f6849cc2702e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:42.395703 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c498ce-af1e-439a-8997-ef634b328f6b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:42.396166 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c498ce-af1e-439a-8997-ef634b328f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:42.396660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c498ce-af1e-439a-8997-ef634b328f6b None admin] PATH_INFO: `/v3/projects/03030ff4e6b54acfb48d93cd4178fd61` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:42.396837 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c9c0a-b5f1-4d9f-a1d3-f6849cc2702e None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:42.398375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c498ce-af1e-439a-8997-ef634b328f6b None admin] RBAC: Authorizing `identity:get_project(project_id=03030ff4e6b54acfb48d93cd4178fd61)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:42.400616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c498ce-af1e-439a-8997-ef634b328f6b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:42.402817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 221/448] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:42 2022] GET /identity/v3/projects/03030ff4e6b54acfb48d93cd4178fd61 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:42.408362 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c498ce-af1e-439a-8997-ef634b328f6b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:42.411680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 228/449] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:42.417472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c9c0a-b5f1-4d9f-a1d3-f6849cc2702e None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:42.447134 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:42.539364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c9c0a-b5f1-4d9f-a1d3-f6849cc2702e None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:42.555484 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c498ce-af1e-439a-8997-ef634b328f6b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:42.580226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83805b53-a9a8-4f77-80ac-0f594aae5299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:42.580943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83805b53-a9a8-4f77-80ac-0f594aae5299 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:42.581229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83805b53-a9a8-4f77-80ac-0f594aae5299 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:42.581507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83805b53-a9a8-4f77-80ac-0f594aae5299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:42.603262 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7712c0-ca26-4bcd-8a5e-94f92485e3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:42.604124 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7712c0-ca26-4bcd-8a5e-94f92485e3cd None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:42.604490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7712c0-ca26-4bcd-8a5e-94f92485e3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:42.604701 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7712c0-ca26-4bcd-8a5e-94f92485e3cd None admin] PATH_INFO: `/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:42.606947 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7712c0-ca26-4bcd-8a5e-94f92485e3cd None admin] RBAC: Authorizing `identity:get_group(group_id=d97e60b0d6d04bff825c3d55f198ffc4)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:42.608461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7712c0-ca26-4bcd-8a5e-94f92485e3cd None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:42.613382 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 222/450] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:42 2022] GET /identity/v3/groups/d97e60b0d6d04bff825c3d55f198ffc4 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:42.619170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7712c0-ca26-4bcd-8a5e-94f92485e3cd None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:42.648547 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:42.663715 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83805b53-a9a8-4f77-80ac-0f594aae5299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:42.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83805b53-a9a8-4f77-80ac-0f594aae5299 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:42.689535 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:42.724779 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7712c0-ca26-4bcd-8a5e-94f92485e3cd None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:42.763850 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d41eb6-e320-41da-9a6e-077785b2e04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:42.765453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d41eb6-e320-41da-9a6e-077785b2e04f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:42.765453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d41eb6-e320-41da-9a6e-077785b2e04f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:42.765623 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 229/451] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 19:46:42.765727 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d41eb6-e320-41da-9a6e-077785b2e04f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:42.766493 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d41eb6-e320-41da-9a6e-077785b2e04f None admin] RBAC: Adding query filter params (group.id=d97e60b0d6d04bff825c3d55f198ffc4, role.id=cb116a5614cf4a48ad99840d95f96bb3, scope.project.id=03030ff4e6b54acfb48d93cd4178fd61) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:42.766801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d41eb6-e320-41da-9a6e-077785b2e04f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:42.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d41eb6-e320-41da-9a6e-077785b2e04f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:42.771242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83805b53-a9a8-4f77-80ac-0f594aae5299 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:42.774728 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 223/452] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 19:46:42 2022] GET /identity/v3/role_assignments?group.id=d97e60b0d6d04bff825c3d55f198ffc4&scope.project.id=03030ff4e6b54acfb48d93cd4178fd61&role.id=cb116a5614cf4a48ad99840d95f96bb3 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:42.881957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83805b53-a9a8-4f77-80ac-0f594aae5299 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:42.908162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:42.919912 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96eca3e-ca78-4072-8ca3-30af16a96a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:42.920841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96eca3e-ca78-4072-8ca3-30af16a96a1b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:42.921172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96eca3e-ca78-4072-8ca3-30af16a96a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:42.921449 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96eca3e-ca78-4072-8ca3-30af16a96a1b None admin] PATH_INFO: `/v3/projects/c59a6214d8424d94bb73ea602efdca3f` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:42.922721 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96eca3e-ca78-4072-8ca3-30af16a96a1b None admin] RBAC: Authorizing `identity:get_project(project_id=c59a6214d8424d94bb73ea602efdca3f)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:42.924194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96eca3e-ca78-4072-8ca3-30af16a96a1b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:42.926326 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 230/453] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:46:42 2022] GET /identity/v3/projects/c59a6214d8424d94bb73ea602efdca3f => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:42.932274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d41eb6-e320-41da-9a6e-077785b2e04f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:43.022823 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:43.039080 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d41eb6-e320-41da-9a6e-077785b2e04f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:43.075727 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377af7a7-93bb-47f9-a19e-d26d87ad2aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:43.076617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377af7a7-93bb-47f9-a19e-d26d87ad2aef None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:43.076903 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377af7a7-93bb-47f9-a19e-d26d87ad2aef None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:43.077180 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377af7a7-93bb-47f9-a19e-d26d87ad2aef None admin] PATH_INFO: `/v3/groups/4f22169376e44f31b711e5470ca9f32e` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:43.078394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377af7a7-93bb-47f9-a19e-d26d87ad2aef None admin] RBAC: Authorizing `identity:get_group(group_id=4f22169376e44f31b711e5470ca9f32e)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:43.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377af7a7-93bb-47f9-a19e-d26d87ad2aef None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:43.081537 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 224/454] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 19:46:42 2022] GET /identity/v3/groups/4f22169376e44f31b711e5470ca9f32e => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:43.087532 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96eca3e-ca78-4072-8ca3-30af16a96a1b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:43.192871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96eca3e-ca78-4072-8ca3-30af16a96a1b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:43.224866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f2271-6723-485b-a3d3-2ae5b4634913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:43.225515 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f2271-6723-485b-a3d3-2ae5b4634913 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:43.225800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f2271-6723-485b-a3d3-2ae5b4634913 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:43.226076 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f2271-6723-485b-a3d3-2ae5b4634913 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:43.226905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f2271-6723-485b-a3d3-2ae5b4634913 None admin] RBAC: Adding query filter params (group.id=4f22169376e44f31b711e5470ca9f32e, role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=c59a6214d8424d94bb73ea602efdca3f) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:43.227212 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f2271-6723-485b-a3d3-2ae5b4634913 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:43.228667 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f2271-6723-485b-a3d3-2ae5b4634913 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:43.235134 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 231/455] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 19:46:43 2022] GET /identity/v3/role_assignments?group.id=4f22169376e44f31b711e5470ca9f32e&scope.project.id=c59a6214d8424d94bb73ea602efdca3f&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:45.829123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7b984-9d15-4d62-80bd-b527f18c0738 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:45.829648 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7b984-9d15-4d62-80bd-b527f18c0738 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:45.829648 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7b984-9d15-4d62-80bd-b527f18c0738 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:45.831330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 225/456] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:45.837270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dc624-f526-4ab9-9d56-3a88fb866e9f None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:45.837706 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dc624-f526-4ab9-9d56-3a88fb866e9f None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:45.837881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7dc624-f526-4ab9-9d56-3a88fb866e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:45.861345 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-7b7dc624-f526-4ab9-9d56-3a88fb866e9f None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:45.862560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-7b7dc624-f526-4ab9-9d56-3a88fb866e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:45.937099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:45.959352 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 232/457] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:46.000611 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7b984-9d15-4d62-80bd-b527f18c0738 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:46.178605 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7b984-9d15-4d62-80bd-b527f18c0738 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:46.225124 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4403175-a387-4d1e-89bd-baf4156f20e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:46.225985 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4403175-a387-4d1e-89bd-baf4156f20e0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:46.226271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4403175-a387-4d1e-89bd-baf4156f20e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:46.226548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4403175-a387-4d1e-89bd-baf4156f20e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:46.241481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4403175-a387-4d1e-89bd-baf4156f20e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:46.243001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4403175-a387-4d1e-89bd-baf4156f20e0 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-d4403175-a387-4d1e-89bd-baf4156f20e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:46.273637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:46.274685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:46.275613 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:46.275613 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:46.275613 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:46.275613 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:46.278312 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 226/458] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:46:46.284947 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7dc624-f526-4ab9-9d56-3a88fb866e9f None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:46.480578 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7dc624-f526-4ab9-9d56-3a88fb866e9f None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:46.547896 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3a56a-5040-4b11-8933-853f944ef676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:46.548623 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3a56a-5040-4b11-8933-853f944ef676 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:46.548910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3a56a-5040-4b11-8933-853f944ef676 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:46.549188 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3a56a-5040-4b11-8933-853f944ef676 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:46.549954 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3a56a-5040-4b11-8933-853f944ef676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:46.550263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3a56a-5040-4b11-8933-853f944ef676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:46.551949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3a56a-5040-4b11-8933-853f944ef676 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:46.571247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 233/459] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:46.578346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4403175-a387-4d1e-89bd-baf4156f20e0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:46.585058 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d95f9-770c-47c9-97cf-a2ee5d499bb4 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:46.585379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d95f9-770c-47c9-97cf-a2ee5d499bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:46.585665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d95f9-770c-47c9-97cf-a2ee5d499bb4 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:46.587617 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 234/460] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:46.593103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:46.607606 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961b2cc-5bc8-4a99-b3c8-7142a4dd96e6 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:46.609783 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961b2cc-5bc8-4a99-b3c8-7142a4dd96e6 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:46.609783 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961b2cc-5bc8-4a99-b3c8-7142a4dd96e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:46.633866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-9961b2cc-5bc8-4a99-b3c8-7142a4dd96e6 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:46.635094 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-9961b2cc-5bc8-4a99-b3c8-7142a4dd96e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:46.712919 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:46.748093 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 235/461] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:46.791337 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961b2cc-5bc8-4a99-b3c8-7142a4dd96e6 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:46.818755 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4403175-a387-4d1e-89bd-baf4156f20e0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:46.889855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9fa8e-6604-4486-b510-0fb7d2dbbead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:46.890613 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9fa8e-6604-4486-b510-0fb7d2dbbead None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:46.890817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9fa8e-6604-4486-b510-0fb7d2dbbead None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:46.891204 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9fa8e-6604-4486-b510-0fb7d2dbbead None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:46.892028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9fa8e-6604-4486-b510-0fb7d2dbbead None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:46.893588 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9fa8e-6604-4486-b510-0fb7d2dbbead None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:46.939548 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961b2cc-5bc8-4a99-b3c8-7142a4dd96e6 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:46.976454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7325c6-061f-41c8-a1a7-51952b0a3e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:46.977194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7325c6-061f-41c8-a1a7-51952b0a3e23 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:46.977467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7325c6-061f-41c8-a1a7-51952b0a3e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:46.977740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7325c6-061f-41c8-a1a7-51952b0a3e23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:46.988997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7325c6-061f-41c8-a1a7-51952b0a3e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:46.990549 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7325c6-061f-41c8-a1a7-51952b0a3e23 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-bd7325c6-061f-41c8-a1a7-51952b0a3e23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:47.009778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:47.010929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:47.011856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:47.011856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:47.011856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:47.011856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:47.011856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 236/462] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:46:47.012651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6c0c0-bca3-40f8-bddd-0713146a9539 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.012651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6c0c0-bca3-40f8-bddd-0713146a9539 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.012915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6c0c0-bca3-40f8-bddd-0713146a9539 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.018719 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 237/463] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:47.018719 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6c0c0-bca3-40f8-bddd-0713146a9539 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:47.032393 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 227/464] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:46:46 2022] POST /identity/v3/users => generated 246 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:47.032393 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ef2ed-3eb1-4097-a93d-fbe23b6c045d None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.032393 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ef2ed-3eb1-4097-a93d-fbe23b6c045d None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.032393 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ef2ed-3eb1-4097-a93d-fbe23b6c045d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.060395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-456ef2ed-3eb1-4097-a93d-fbe23b6c045d None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:47.062313 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-456ef2ed-3eb1-4097-a93d-fbe23b6c045d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:47.063294 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:47.130864 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:47.143152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 228/465] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:47.144860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43234bf-d814-4f49-ad55-49fa6d297954 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.145259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43234bf-d814-4f49-ad55-49fa6d297954 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.145638 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43234bf-d814-4f49-ad55-49fa6d297954 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.151752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 229/466] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:47.153284 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7176d-1d9c-4fda-8f30-ef2b807e8016 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.153698 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7176d-1d9c-4fda-8f30-ef2b807e8016 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.154080 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7176d-1d9c-4fda-8f30-ef2b807e8016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.158451 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6c0c0-bca3-40f8-bddd-0713146a9539 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:47.182409 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-49f7176d-1d9c-4fda-8f30-ef2b807e8016 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:47.183900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-49f7176d-1d9c-4fda-8f30-ef2b807e8016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:47.199975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d13400-82a8-4d64-b051-d08a13db6f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:47.200849 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d13400-82a8-4d64-b051-d08a13db6f89 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.200970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d13400-82a8-4d64-b051-d08a13db6f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.201213 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d13400-82a8-4d64-b051-d08a13db6f89 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.202159 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d13400-82a8-4d64-b051-d08a13db6f89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:47.202404 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d13400-82a8-4d64-b051-d08a13db6f89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:47.203888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d13400-82a8-4d64-b051-d08a13db6f89 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:47.219548 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 238/467] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:47.220659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d13400-82a8-4d64-b051-d08a13db6f89 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:47.265000 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 230/468] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:47.266192 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7176d-1d9c-4fda-8f30-ef2b807e8016 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:47.325457 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:47.337585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d13400-82a8-4d64-b051-d08a13db6f89 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:47.385370 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bfa54-d24a-4d68-8654-31e3d6b2bbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:47.386169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bfa54-d24a-4d68-8654-31e3d6b2bbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.386452 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bfa54-d24a-4d68-8654-31e3d6b2bbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.386724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bfa54-d24a-4d68-8654-31e3d6b2bbc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.387582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7176d-1d9c-4fda-8f30-ef2b807e8016 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:47.397709 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09bfa54-d24a-4d68-8654-31e3d6b2bbc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:47.399169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09bfa54-d24a-4d68-8654-31e3d6b2bbc9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:47.414531 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-b09bfa54-d24a-4d68-8654-31e3d6b2bbc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:47.420230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:47.426574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:47.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:47.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:47.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:47.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:47.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 239/469] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:47.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bfa54-d24a-4d68-8654-31e3d6b2bbc9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:47.434112 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bc9afe-180c-40ae-9f92-3f46c3f09c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:47.434794 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9afe-180c-40ae-9f92-3f46c3f09c5b None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.435082 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9afe-180c-40ae-9f92-3f46c3f09c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.435718 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bc9afe-180c-40ae-9f92-3f46c3f09c5b None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.436854 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc9afe-180c-40ae-9f92-3f46c3f09c5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:47.438331 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bc9afe-180c-40ae-9f92-3f46c3f09c5b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:47.524171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 231/470] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:46:47 2022] POST /identity/v3/users => generated 248 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:47.525581 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f85f19-4e86-4eee-8d6c-22e80ba31590 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.525948 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f85f19-4e86-4eee-8d6c-22e80ba31590 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.526230 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f85f19-4e86-4eee-8d6c-22e80ba31590 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.535511 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 232/471] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:47.535511 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f85f19-4e86-4eee-8d6c-22e80ba31590 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:47.536779 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bfa54-d24a-4d68-8654-31e3d6b2bbc9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:47.590128 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d136b85-380d-40b3-9b52-a3c18515890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:47.591614 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d136b85-380d-40b3-9b52-a3c18515890c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.591737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d136b85-380d-40b3-9b52-a3c18515890c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.591998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d136b85-380d-40b3-9b52-a3c18515890c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.605180 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:47.608442 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d136b85-380d-40b3-9b52-a3c18515890c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:47.610714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d136b85-380d-40b3-9b52-a3c18515890c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-6d136b85-380d-40b3-9b52-a3c18515890c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:47.634813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:47.635727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:47.636620 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:47.636620 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:47.636620 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:47.636620 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:47.638838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 240/472] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:47.641124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f745-1ac4-4220-80c7-d1c5d2244a0f None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.641733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f745-1ac4-4220-80c7-d1c5d2244a0f None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.642208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44f745-1ac4-4220-80c7-d1c5d2244a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.662566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-4e44f745-1ac4-4220-80c7-d1c5d2244a0f None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:47.664119 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-4e44f745-1ac4-4220-80c7-d1c5d2244a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:47.676121 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f85f19-4e86-4eee-8d6c-22e80ba31590 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:47.728496 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002868e-bef1-4aaf-8d5a-9028cd6df584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:47.729180 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002868e-bef1-4aaf-8d5a-9028cd6df584 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.729466 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002868e-bef1-4aaf-8d5a-9028cd6df584 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.729745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002868e-bef1-4aaf-8d5a-9028cd6df584 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.730490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002868e-bef1-4aaf-8d5a-9028cd6df584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:47.730798 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002868e-bef1-4aaf-8d5a-9028cd6df584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:47.732881 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002868e-bef1-4aaf-8d5a-9028cd6df584 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:47.748187 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 241/473] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:47.749622 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44f745-1ac4-4220-80c7-d1c5d2244a0f None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:47.768483 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 233/474] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:47.772013 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002868e-bef1-4aaf-8d5a-9028cd6df584 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:47.826151 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:47.884016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44f745-1ac4-4220-80c7-d1c5d2244a0f None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:47.908806 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002868e-bef1-4aaf-8d5a-9028cd6df584 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:47.909990 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:47.922171 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06e54c-a3c2-4fc0-a2df-34acd1c5173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:47.922925 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06e54c-a3c2-4fc0-a2df-34acd1c5173c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.923274 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06e54c-a3c2-4fc0-a2df-34acd1c5173c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.923611 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06e54c-a3c2-4fc0-a2df-34acd1c5173c None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.924481 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06e54c-a3c2-4fc0-a2df-34acd1c5173c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:47.924851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06e54c-a3c2-4fc0-a2df-34acd1c5173c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:47.926393 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06e54c-a3c2-4fc0-a2df-34acd1c5173c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:47.943330 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 242/475] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:47.944418 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06e54c-a3c2-4fc0-a2df-34acd1c5173c None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:47.969257 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18389ee9-10d5-45e9-a3fb-297db80303a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:47.970058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18389ee9-10d5-45e9-a3fb-297db80303a8 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:47.970510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18389ee9-10d5-45e9-a3fb-297db80303a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:47.970812 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18389ee9-10d5-45e9-a3fb-297db80303a8 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:47.971874 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18389ee9-10d5-45e9-a3fb-297db80303a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:47.974012 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18389ee9-10d5-45e9-a3fb-297db80303a8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:48.069500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06e54c-a3c2-4fc0-a2df-34acd1c5173c None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:48.116633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 234/476] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:46:47 2022] POST /identity/v3/users => generated 248 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:48.116633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18389ee9-10d5-45e9-a3fb-297db80303a8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:48.116884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ece8c-1f60-4353-864b-56260f9fa65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:48.116884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ece8c-1f60-4353-864b-56260f9fa65f None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:48.116884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ece8c-1f60-4353-864b-56260f9fa65f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:48.116884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ece8c-1f60-4353-864b-56260f9fa65f None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:48.117130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ece8c-1f60-4353-864b-56260f9fa65f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:48.119742 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ece8c-1f60-4353-864b-56260f9fa65f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:48.170994 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.sql.core [None req-641ece8c-1f60-4353-864b-56260f9fa65f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:46:48.170994 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 19:46:48.170994 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [parameters: {'id': 'c9556701d96442f79a00e07981c2f6e1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 19:46:48.170994 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80452) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-641ece8c-1f60-4353-864b-56260f9fa65f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application conn.query(q) May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application result.read() May 19 19:46:48.199672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return ref.to_dict() May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:46:48.200653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application next(self.gen) May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application yield resource May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application next(self.gen) May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise self.value May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application session.commit() May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:46:48.202751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self.session.flush() May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application compat.raise_( May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:46:48.203600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:46:48.205424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application conn.query(q) May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application result.read() May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:46:48.207284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [parameters: {'id': 'c9556701d96442f79a00e07981c2f6e1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:48.208957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 19:46:48.215012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:48.217444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 243/477] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 19:46:47 2022] POST /identity/v3/roles => generated 144 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:48.217444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ece8c-1f60-4353-864b-56260f9fa65f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:48.319975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:48.322869 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:48.336606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18389ee9-10d5-45e9-a3fb-297db80303a8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:48.412211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c53ddd-b306-4b91-b5bc-af428b2fcad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:48.414173 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c53ddd-b306-4b91-b5bc-af428b2fcad3 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:48.414482 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c53ddd-b306-4b91-b5bc-af428b2fcad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:48.414763 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c53ddd-b306-4b91-b5bc-af428b2fcad3 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:48.415593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c53ddd-b306-4b91-b5bc-af428b2fcad3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:48.417081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c53ddd-b306-4b91-b5bc-af428b2fcad3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:48.437909 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ece8c-1f60-4353-864b-56260f9fa65f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:48.482397 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756465f-392b-4bfa-a8f5-f55f6fd834e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:48.483859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756465f-392b-4bfa-a8f5-f55f6fd834e3 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:48.484038 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756465f-392b-4bfa-a8f5-f55f6fd834e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:48.484361 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756465f-392b-4bfa-a8f5-f55f6fd834e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:48.496126 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 235/478] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:46:48 2022] POST /identity/v3/users => generated 249 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:48.499192 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756465f-392b-4bfa-a8f5-f55f6fd834e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-f756465f-392b-4bfa-a8f5-f55f6fd834e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:48.508087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:48.509081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:48.509890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 244/479] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:46:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:48.516866 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c53ddd-b306-4b91-b5bc-af428b2fcad3 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:48.677449 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c53ddd-b306-4b91-b5bc-af428b2fcad3 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:48.729012 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5eff1-68bd-4944-82ba-aed4a77bb3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:48.729703 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5eff1-68bd-4944-82ba-aed4a77bb3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:48.729987 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5eff1-68bd-4944-82ba-aed4a77bb3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:48.730274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5eff1-68bd-4944-82ba-aed4a77bb3d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:48.731041 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5eff1-68bd-4944-82ba-aed4a77bb3d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:48.731349 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5eff1-68bd-4944-82ba-aed4a77bb3d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:48.732808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5eff1-68bd-4944-82ba-aed4a77bb3d3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:48.744447 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 236/480] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:49.790504 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f28bf-e2bc-452b-810c-c4af13573b2a None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:49.791080 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f28bf-e2bc-452b-810c-c4af13573b2a None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:49.791080 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f28bf-e2bc-452b-810c-c4af13573b2a None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:49.792738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 245/481] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:49.800500 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa151c56-cb19-46d2-957e-92683c202cb2 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:49.801423 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa151c56-cb19-46d2-957e-92683c202cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:49.802183 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa151c56-cb19-46d2-957e-92683c202cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:49.821245 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-aa151c56-cb19-46d2-957e-92683c202cb2 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:49.822974 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-aa151c56-cb19-46d2-957e-92683c202cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:49.831793 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:49.914241 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 237/482] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:49.963381 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f28bf-e2bc-452b-810c-c4af13573b2a None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:50.090849 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9f28bf-e2bc-452b-810c-c4af13573b2a None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:50.098545 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:50.128419 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d766e6-8f79-4acf-b532-8262561c0c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:50.129234 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d766e6-8f79-4acf-b532-8262561c0c70 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:50.129520 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d766e6-8f79-4acf-b532-8262561c0c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:50.129787 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d766e6-8f79-4acf-b532-8262561c0c70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:50.142241 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d766e6-8f79-4acf-b532-8262561c0c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-c4d766e6-8f79-4acf-b532-8262561c0c70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:50.157430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:50.158360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:50.159615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 246/483] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:46:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:50.164621 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa151c56-cb19-46d2-957e-92683c202cb2 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:50.204566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10e60-79ec-41e5-baf1-f199c70b5086 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:50.204884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10e60-79ec-41e5-baf1-f199c70b5086 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:50.205204 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10e60-79ec-41e5-baf1-f199c70b5086 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:50.207082 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 247/484] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:50.213467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fb277-d0f8-4884-963c-978d7288ac41 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:50.213765 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fb277-d0f8-4884-963c-978d7288ac41 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:50.214047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fb277-d0f8-4884-963c-978d7288ac41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:50.234339 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-c26fb277-d0f8-4884-963c-978d7288ac41 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:50.235536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-c26fb277-d0f8-4884-963c-978d7288ac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:50.328722 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 248/485] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:50.353011 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa151c56-cb19-46d2-957e-92683c202cb2 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:50.394426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fb277-d0f8-4884-963c-978d7288ac41 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:50.400470 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2ae2a-427f-412c-a7d7-ab1e4dde39c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:50.401149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2ae2a-427f-412c-a7d7-ab1e4dde39c5 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:50.401436 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2ae2a-427f-412c-a7d7-ab1e4dde39c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:50.401716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2ae2a-427f-412c-a7d7-ab1e4dde39c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:50.403346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2ae2a-427f-412c-a7d7-ab1e4dde39c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:50.403665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2ae2a-427f-412c-a7d7-ab1e4dde39c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:50.405656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2ae2a-427f-412c-a7d7-ab1e4dde39c5 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:50.415806 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 238/486] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:46:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:50.421679 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2ae2a-427f-412c-a7d7-ab1e4dde39c5 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:50.444726 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:50.548981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2ae2a-427f-412c-a7d7-ab1e4dde39c5 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:50.591930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e493114-25b2-4a57-b1af-819ac7612191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:50.593407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e493114-25b2-4a57-b1af-819ac7612191 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:50.593738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e493114-25b2-4a57-b1af-819ac7612191 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:50.594010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e493114-25b2-4a57-b1af-819ac7612191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:50.607637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e493114-25b2-4a57-b1af-819ac7612191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:50.609576 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e493114-25b2-4a57-b1af-819ac7612191 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-2e493114-25b2-4a57-b1af-819ac7612191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:50.630216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:50.631141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:50.632191 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:50.632191 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:50.632191 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:50.632191 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:50.632191 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 239/487] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:50.637100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e493114-25b2-4a57-b1af-819ac7612191 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:50.660695 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:50.664687 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fb277-d0f8-4884-963c-978d7288ac41 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:50.736734 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:50.787373 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e493114-25b2-4a57-b1af-819ac7612191 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:50.793762 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439b968-3324-4f9d-8150-41519fe37a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:50.795251 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439b968-3324-4f9d-8150-41519fe37a8a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:50.795768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439b968-3324-4f9d-8150-41519fe37a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:50.796261 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439b968-3324-4f9d-8150-41519fe37a8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:50.827325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367981af-5600-429a-94c1-25793ce4676e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:50.828113 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367981af-5600-429a-94c1-25793ce4676e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:50.828434 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367981af-5600-429a-94c1-25793ce4676e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:50.828716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367981af-5600-429a-94c1-25793ce4676e None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:50.829211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439b968-3324-4f9d-8150-41519fe37a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:50.829448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367981af-5600-429a-94c1-25793ce4676e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:50.829755 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367981af-5600-429a-94c1-25793ce4676e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:50.831169 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367981af-5600-429a-94c1-25793ce4676e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-4439b968-3324-4f9d-8150-41519fe37a8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:50.837950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:50.838860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:50.839653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:50.842934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 249/488] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:46:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:50.848957 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 240/489] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:50.850114 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439b968-3324-4f9d-8150-41519fe37a8a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:50.855854 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367981af-5600-429a-94c1-25793ce4676e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:51.020325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367981af-5600-429a-94c1-25793ce4676e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:51.057806 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690bacf-8729-4fde-bfdc-f40ee0498883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:51.059114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690bacf-8729-4fde-bfdc-f40ee0498883 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.059603 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690bacf-8729-4fde-bfdc-f40ee0498883 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.060110 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690bacf-8729-4fde-bfdc-f40ee0498883 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.070959 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690bacf-8729-4fde-bfdc-f40ee0498883 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:51.073024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690bacf-8729-4fde-bfdc-f40ee0498883 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:51.076585 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-7690bacf-8729-4fde-bfdc-f40ee0498883 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:46:51.092584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:51.093609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:51.094504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:51.094504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:51.094504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:46:51.094504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:46:51.094504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:46:51.094504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:46:51.094504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:46:51.094504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:46:51.094504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:46:51.094504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:51.095030 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 241/490] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 19:46:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:51.100186 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690bacf-8729-4fde-bfdc-f40ee0498883 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:51.135504 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439b968-3324-4f9d-8150-41519fe37a8a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:51.172162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205285b0-fb20-483b-bc3b-39a5a666ff29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:51.172884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205285b0-fb20-483b-bc3b-39a5a666ff29 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.173162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205285b0-fb20-483b-bc3b-39a5a666ff29 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.173454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205285b0-fb20-483b-bc3b-39a5a666ff29 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.174255 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205285b0-fb20-483b-bc3b-39a5a666ff29 None admin] RBAC: Adding query filter params (name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:51.174604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205285b0-fb20-483b-bc3b-39a5a666ff29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:51.176086 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205285b0-fb20-483b-bc3b-39a5a666ff29 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:51.187265 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 250/491] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:46:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:51.188827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b060-4f4f-470e-9a0c-7519fa922480 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.189121 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b060-4f4f-470e-9a0c-7519fa922480 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.189466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03b060-4f4f-470e-9a0c-7519fa922480 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.190899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 251/492] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:51.196312 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03b060-4f4f-470e-9a0c-7519fa922480 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:51.227236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:51.236466 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690bacf-8729-4fde-bfdc-f40ee0498883 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:51.281973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e2495-5c18-4592-b487-6b2e667c84fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:51.282659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e2495-5c18-4592-b487-6b2e667c84fd None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.282963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e2495-5c18-4592-b487-6b2e667c84fd None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.283297 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e2495-5c18-4592-b487-6b2e667c84fd None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.284275 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e2495-5c18-4592-b487-6b2e667c84fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:51.284640 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e2495-5c18-4592-b487-6b2e667c84fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:51.286156 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e2495-5c18-4592-b487-6b2e667c84fd None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:51.327082 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 242/493] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 19:46:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:51.328579 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fdcc7-3dd0-482f-b056-3b3d645addc9 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.328896 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fdcc7-3dd0-482f-b056-3b3d645addc9 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.329230 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fdcc7-3dd0-482f-b056-3b3d645addc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.331476 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03b060-4f4f-470e-9a0c-7519fa922480 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:51.357703 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-320fdcc7-3dd0-482f-b056-3b3d645addc9 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:51.359451 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-320fdcc7-3dd0-482f-b056-3b3d645addc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:51.375572 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a00af-a720-4e0d-99c1-1dfcd7a8a929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:51.377028 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a00af-a720-4e0d-99c1-1dfcd7a8a929 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.378023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a00af-a720-4e0d-99c1-1dfcd7a8a929 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.378555 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a00af-a720-4e0d-99c1-1dfcd7a8a929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.392114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:51.393925 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a00af-a720-4e0d-99c1-1dfcd7a8a929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:51.395591 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a00af-a720-4e0d-99c1-1dfcd7a8a929 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-485a00af-a720-4e0d-99c1-1dfcd7a8a929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:51.413816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:51.414754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:51.415593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:51.415593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:51.415593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:51.415593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:51.419623 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 252/494] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:51.421074 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a00af-a720-4e0d-99c1-1dfcd7a8a929 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:51.440392 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 243/495] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:51.447606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25f8c7-cadb-41c9-8a23-1a10197be73e None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.448119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25f8c7-cadb-41c9-8a23-1a10197be73e None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.448666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25f8c7-cadb-41c9-8a23-1a10197be73e None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.456624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 244/496] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:51 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:51.456624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25f8c7-cadb-41c9-8a23-1a10197be73e None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:51.515875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:51.551374 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a00af-a720-4e0d-99c1-1dfcd7a8a929 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:51.588510 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48b6b7-b856-43de-85d6-b202862ca454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:51.589172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48b6b7-b856-43de-85d6-b202862ca454 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.589455 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48b6b7-b856-43de-85d6-b202862ca454 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.589732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48b6b7-b856-43de-85d6-b202862ca454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.590677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25f8c7-cadb-41c9-8a23-1a10197be73e None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:51.630003 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f6839-6965-482a-afe6-f37a3e025c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:51.630662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f6839-6965-482a-afe6-f37a3e025c22 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.630945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f6839-6965-482a-afe6-f37a3e025c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.631218 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f6839-6965-482a-afe6-f37a3e025c22 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.631942 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f6839-6965-482a-afe6-f37a3e025c22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:51.632245 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f6839-6965-482a-afe6-f37a3e025c22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:51.633668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f6839-6965-482a-afe6-f37a3e025c22 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:51.646484 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 245/497] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:51.649728 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953362f-76e8-4adb-ba13-45c1b35fab43 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.650029 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953362f-76e8-4adb-ba13-45c1b35fab43 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.650296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953362f-76e8-4adb-ba13-45c1b35fab43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.668109 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48b6b7-b856-43de-85d6-b202862ca454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:51.669430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-b953362f-76e8-4adb-ba13-45c1b35fab43 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:51.669765 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48b6b7-b856-43de-85d6-b202862ca454 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:51.670650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-b953362f-76e8-4adb-ba13-45c1b35fab43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:51.691347 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:51.745048 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:51.752819 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 246/498] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:51.753642 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953362f-76e8-4adb-ba13-45c1b35fab43 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:51.782087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 253/499] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:51.783238 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48b6b7-b856-43de-85d6-b202862ca454 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:51.862896 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953362f-76e8-4adb-ba13-45c1b35fab43 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:51.885648 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48b6b7-b856-43de-85d6-b202862ca454 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:51.898699 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15fe09-c0a5-463e-9199-0843f717929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:51.899500 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15fe09-c0a5-463e-9199-0843f717929b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.899784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15fe09-c0a5-463e-9199-0843f717929b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.900058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15fe09-c0a5-463e-9199-0843f717929b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.911128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15fe09-c0a5-463e-9199-0843f717929b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-4c15fe09-c0a5-463e-9199-0843f717929b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:51.919155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:51.920139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:51.921170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 247/500] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:46:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:51.921871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9e523-d7b6-4b16-af98-cd82af300c69 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.922149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9e523-d7b6-4b16-af98-cd82af300c69 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.922454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9e523-d7b6-4b16-af98-cd82af300c69 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.923714 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 248/501] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:51.924734 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9e523-d7b6-4b16-af98-cd82af300c69 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:51.926181 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b2706-3978-4b5c-86c5-a31d1dc215c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:51.927310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b2706-3978-4b5c-86c5-a31d1dc215c8 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:51.927586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b2706-3978-4b5c-86c5-a31d1dc215c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:51.927986 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b2706-3978-4b5c-86c5-a31d1dc215c8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:51.937782 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b2706-3978-4b5c-86c5-a31d1dc215c8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:51.940182 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b2706-3978-4b5c-86c5-a31d1dc215c8 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:51.943543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-b56b2706-3978-4b5c-86c5-a31d1dc215c8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:46:51.958753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:51.965270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 254/502] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:51.966161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b2706-3978-4b5c-86c5-a31d1dc215c8 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:51.994614 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:52.055687 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9e523-d7b6-4b16-af98-cd82af300c69 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:52.069457 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b2706-3978-4b5c-86c5-a31d1dc215c8 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:52.096729 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b92a-99ac-4016-affb-59b94f8c2bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:52.097529 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b92a-99ac-4016-affb-59b94f8c2bde None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:52.097798 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b92a-99ac-4016-affb-59b94f8c2bde None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:52.098069 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53b92a-99ac-4016-affb-59b94f8c2bde None admin] PATH_INFO: `/v3/roles/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:52.105678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a06bfb-46e9-4fce-90ee-e144bff1ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:52.106464 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a06bfb-46e9-4fce-90ee-e144bff1ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:52.106742 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a06bfb-46e9-4fce-90ee-e144bff1ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:52.107012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a06bfb-46e9-4fce-90ee-e144bff1ecf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:52.109581 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53b92a-99ac-4016-affb-59b94f8c2bde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-3a53b92a-99ac-4016-affb-59b94f8c2bde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:52.117548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:52.118662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:52.119559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:52.120253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a06bfb-46e9-4fce-90ee-e144bff1ecf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:52.120372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 249/503] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:46:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:52.120456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a06bfb-46e9-4fce-90ee-e144bff1ecf8 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:52.124808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b92a-99ac-4016-affb-59b94f8c2bde None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-06a06bfb-46e9-4fce-90ee-e144bff1ecf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:52.140571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:52.141566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:52.142456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:52.142456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:52.142456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:52.142456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:52.142456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 255/504] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:52.143193 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fe184-c19c-4b13-b08f-6309b92bc033 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:52.143413 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fe184-c19c-4b13-b08f-6309b92bc033 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:52.143692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fe184-c19c-4b13-b08f-6309b92bc033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:52.161802 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-4e9fe184-c19c-4b13-b08f-6309b92bc033 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:52.163034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-4e9fe184-c19c-4b13-b08f-6309b92bc033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:52.206437 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:52.231460 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 256/505] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:52.231645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53b92a-99ac-4016-affb-59b94f8c2bde None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:52.232307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fe184-c19c-4b13-b08f-6309b92bc033 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:52.246999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:52.268285 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb405f-ef91-4268-bd68-d97bc5f079d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:52.269085 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb405f-ef91-4268-bd68-d97bc5f079d3 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:52.269366 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb405f-ef91-4268-bd68-d97bc5f079d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:52.269636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb405f-ef91-4268-bd68-d97bc5f079d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:52.270375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb405f-ef91-4268-bd68-d97bc5f079d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:52.270677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb405f-ef91-4268-bd68-d97bc5f079d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:52.272102 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb405f-ef91-4268-bd68-d97bc5f079d3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:52.280684 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 250/506] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:46:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:52.281538 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb405f-ef91-4268-bd68-d97bc5f079d3 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:52.350037 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fe184-c19c-4b13-b08f-6309b92bc033 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:52.388391 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835516f-6da8-48e9-87ec-f3dcf9ccf15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:52.389044 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835516f-6da8-48e9-87ec-f3dcf9ccf15c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:52.389380 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835516f-6da8-48e9-87ec-f3dcf9ccf15c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:52.389630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835516f-6da8-48e9-87ec-f3dcf9ccf15c None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:52.390972 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835516f-6da8-48e9-87ec-f3dcf9ccf15c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:52.391201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835516f-6da8-48e9-87ec-f3dcf9ccf15c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:52.392818 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835516f-6da8-48e9-87ec-f3dcf9ccf15c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:52.398436 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb405f-ef91-4268-bd68-d97bc5f079d3 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:52.433902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 257/507] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 19:46:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:52.434876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835516f-6da8-48e9-87ec-f3dcf9ccf15c None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:52.441901 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca0675-079f-4ce8-bfd1-a882c822b987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:52.442588 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca0675-079f-4ce8-bfd1-a882c822b987 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:52.442875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca0675-079f-4ce8-bfd1-a882c822b987 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:52.443158 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca0675-079f-4ce8-bfd1-a882c822b987 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:52.443926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca0675-079f-4ce8-bfd1-a882c822b987 None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:52.444238 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca0675-079f-4ce8-bfd1-a882c822b987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:52.445717 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca0675-079f-4ce8-bfd1-a882c822b987 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:52.448415 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:52.456394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 251/508] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:46:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:52.461960 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca0675-079f-4ce8-bfd1-a882c822b987 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:52.516245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:52.549593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835516f-6da8-48e9-87ec-f3dcf9ccf15c None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:52.562683 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca0675-079f-4ce8-bfd1-a882c822b987 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:52.584392 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c53dd-cbcd-4447-a8fa-5ac8d28623f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:52.585150 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c53dd-cbcd-4447-a8fa-5ac8d28623f5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:52.585445 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c53dd-cbcd-4447-a8fa-5ac8d28623f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:52.585725 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c53dd-cbcd-4447-a8fa-5ac8d28623f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:52.586461 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c53dd-cbcd-4447-a8fa-5ac8d28623f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:52.586776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c53dd-cbcd-4447-a8fa-5ac8d28623f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:52.588313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c53dd-cbcd-4447-a8fa-5ac8d28623f5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:52.601444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 258/509] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:52.601837 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3720126-828b-477c-abd0-c5a5bd1c76a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:52.602288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c53dd-cbcd-4447-a8fa-5ac8d28623f5 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:52.602738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3720126-828b-477c-abd0-c5a5bd1c76a1 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:52.603065 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3720126-828b-477c-abd0-c5a5bd1c76a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:52.603327 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3720126-828b-477c-abd0-c5a5bd1c76a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:52.615082 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3720126-828b-477c-abd0-c5a5bd1c76a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:52.616599 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3720126-828b-477c-abd0-c5a5bd1c76a1 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-a3720126-828b-477c-abd0-c5a5bd1c76a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:52.636853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:52.638015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:52.639194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:52.639194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:52.639194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:52.639194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:52.639194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 252/510] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:52.639194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3720126-828b-477c-abd0-c5a5bd1c76a1 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:52.691803 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:52.706386 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c53dd-cbcd-4447-a8fa-5ac8d28623f5 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:52.736828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:52.742466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a8208-83a3-400f-ae1c-87d090f58067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:52.743264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a8208-83a3-400f-ae1c-87d090f58067 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:52.743546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a8208-83a3-400f-ae1c-87d090f58067 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:52.743821 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a8208-83a3-400f-ae1c-87d090f58067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:52.744573 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3720126-828b-477c-abd0-c5a5bd1c76a1 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:52.753731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a8208-83a3-400f-ae1c-87d090f58067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:52.755196 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a8208-83a3-400f-ae1c-87d090f58067 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-036a8208-83a3-400f-ae1c-87d090f58067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:52.772996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:52.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:52.774721 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:52.774721 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:52.774721 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:52.774721 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:52.774721 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 259/511] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:52.775000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a8208-83a3-400f-ae1c-87d090f58067 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:52.784224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed9786-8d42-459a-9b2a-680e8692af00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:52.784907 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9786-8d42-459a-9b2a-680e8692af00 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:52.785196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9786-8d42-459a-9b2a-680e8692af00 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:52.785474 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed9786-8d42-459a-9b2a-680e8692af00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:52.858586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed9786-8d42-459a-9b2a-680e8692af00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:52.860075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed9786-8d42-459a-9b2a-680e8692af00 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:52.877144 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a8208-83a3-400f-ae1c-87d090f58067 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:52.913837 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b410a-e7df-4e80-ada6-be6ec905971a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:52.914553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b410a-e7df-4e80-ada6-be6ec905971a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:52.914840 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b410a-e7df-4e80-ada6-be6ec905971a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:52.915122 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b410a-e7df-4e80-ada6-be6ec905971a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:52.923677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:52.930719 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b410a-e7df-4e80-ada6-be6ec905971a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:52.932242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b410a-e7df-4e80-ada6-be6ec905971a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-f95b410a-e7df-4e80-ada6-be6ec905971a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:52.950690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:52.952309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:52.953394 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:52.953394 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:52.953394 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:52.953394 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:52.953394 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 260/512] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:52.954036 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b410a-e7df-4e80-ada6-be6ec905971a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:52.981244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 253/513] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:52.982107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed9786-8d42-459a-9b2a-680e8692af00 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:52.990991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:53.057534 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b410a-e7df-4e80-ada6-be6ec905971a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:53.084583 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed9786-8d42-459a-9b2a-680e8692af00 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:53.092013 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8625624-807d-4ce1-8aef-23cec98b8d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:53.093457 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8625624-807d-4ce1-8aef-23cec98b8d9a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:53.093742 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8625624-807d-4ce1-8aef-23cec98b8d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:53.094019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8625624-807d-4ce1-8aef-23cec98b8d9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:53.101021 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8625624-807d-4ce1-8aef-23cec98b8d9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:53.102465 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8625624-807d-4ce1-8aef-23cec98b8d9a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-e8625624-807d-4ce1-8aef-23cec98b8d9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:46:53.118153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:53.119100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:53.120135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:46:53.120135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:46:53.120135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:46:53.120135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:46:53.120135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:46:53.120135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:46:53.120135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:53.120135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:53.120135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 261/514] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:53.120964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8625624-807d-4ce1-8aef-23cec98b8d9a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:53.125294 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbcce89-9f5c-4908-a5ae-bfad7e4e7b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:53.125959 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcce89-9f5c-4908-a5ae-bfad7e4e7b26 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:53.126244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcce89-9f5c-4908-a5ae-bfad7e4e7b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:53.126519 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcce89-9f5c-4908-a5ae-bfad7e4e7b26 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:53.127250 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbcce89-9f5c-4908-a5ae-bfad7e4e7b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:53.127554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbcce89-9f5c-4908-a5ae-bfad7e4e7b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:53.129006 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbcce89-9f5c-4908-a5ae-bfad7e4e7b26 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:53.141133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 254/515] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:53.141972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbcce89-9f5c-4908-a5ae-bfad7e4e7b26 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:53.156533 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:53.223370 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:53.228988 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8625624-807d-4ce1-8aef-23cec98b8d9a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:53.254109 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbcce89-9f5c-4908-a5ae-bfad7e4e7b26 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:53.265401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f15c6-8633-4310-b41b-451a4f029b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:53.266062 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f15c6-8633-4310-b41b-451a4f029b4c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:53.266349 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f15c6-8633-4310-b41b-451a4f029b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:53.266633 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f15c6-8633-4310-b41b-451a4f029b4c None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:53.267353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f15c6-8633-4310-b41b-451a4f029b4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:53.267661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f15c6-8633-4310-b41b-451a4f029b4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:53.269064 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f15c6-8633-4310-b41b-451a4f029b4c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:53.283325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 262/516] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:53.284178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f15c6-8633-4310-b41b-451a4f029b4c None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:53.293556 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27319e1-8272-4004-95c0-683d8ef50f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:53.294216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27319e1-8272-4004-95c0-683d8ef50f0f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:53.294500 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27319e1-8272-4004-95c0-683d8ef50f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:53.294775 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27319e1-8272-4004-95c0-683d8ef50f0f None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:53.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27319e1-8272-4004-95c0-683d8ef50f0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:53.295990 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27319e1-8272-4004-95c0-683d8ef50f0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:53.297399 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27319e1-8272-4004-95c0-683d8ef50f0f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:53.309580 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 255/517] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:53.310306 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27319e1-8272-4004-95c0-683d8ef50f0f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:53.384722 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:53.393490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f15c6-8633-4310-b41b-451a4f029b4c None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:53.417004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27319e1-8272-4004-95c0-683d8ef50f0f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:53.428983 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb30a7-cb0a-4a42-a2d9-74f4eb6eb9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:53.429773 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb30a7-cb0a-4a42-a2d9-74f4eb6eb9ed None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:53.430053 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb30a7-cb0a-4a42-a2d9-74f4eb6eb9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:53.430325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb30a7-cb0a-4a42-a2d9-74f4eb6eb9ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:53.442507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb30a7-cb0a-4a42-a2d9-74f4eb6eb9ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:53.444056 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb30a7-cb0a-4a42-a2d9-74f4eb6eb9ed None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:53.453460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababac50-54f9-4413-a505-6f234b7b3f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:53.454129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababac50-54f9-4413-a505-6f234b7b3f08 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:53.454411 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababac50-54f9-4413-a505-6f234b7b3f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:53.454685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababac50-54f9-4413-a505-6f234b7b3f08 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:53.455480 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababac50-54f9-4413-a505-6f234b7b3f08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:53.455793 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababac50-54f9-4413-a505-6f234b7b3f08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:53.457310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababac50-54f9-4413-a505-6f234b7b3f08 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-44fb30a7-cb0a-4a42-a2d9-74f4eb6eb9ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:53.462685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:53.463525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:53.470342 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:53.470342 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:53.470342 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:53.470342 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:53.470342 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 263/518] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:53.470342 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb30a7-cb0a-4a42-a2d9-74f4eb6eb9ed None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:53.475872 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 256/519] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:46:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:53.477297 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababac50-54f9-4413-a505-6f234b7b3f08 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:53.479282 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:53.571614 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb30a7-cb0a-4a42-a2d9-74f4eb6eb9ed None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:53.580024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababac50-54f9-4413-a505-6f234b7b3f08 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:53.607132 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb342c9-f086-4b13-a615-444d57e0e783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:53.607801 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb342c9-f086-4b13-a615-444d57e0e783 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:53.608093 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb342c9-f086-4b13-a615-444d57e0e783 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:53.608380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:53.608489 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb342c9-f086-4b13-a615-444d57e0e783 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:53.609158 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb342c9-f086-4b13-a615-444d57e0e783 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:53.610513 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb342c9-f086-4b13-a615-444d57e0e783 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:53.614037 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe632045-37f4-40ae-8790-ec28c9fe5ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:53.614857 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe632045-37f4-40ae-8790-ec28c9fe5ded None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:53.615136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe632045-37f4-40ae-8790-ec28c9fe5ded None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:53.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe632045-37f4-40ae-8790-ec28c9fe5ded None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:53.623236 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe632045-37f4-40ae-8790-ec28c9fe5ded None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:53.624726 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe632045-37f4-40ae-8790-ec28c9fe5ded None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-fe632045-37f4-40ae-8790-ec28c9fe5ded None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:46:53.639463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:53.640563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 257/520] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:53.641710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe632045-37f4-40ae-8790-ec28c9fe5ded None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:53.700745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 264/521] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:46:53 2022] POST /identity/v3/users => generated 247 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:46:53.701932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb342c9-f086-4b13-a615-444d57e0e783 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:53.750677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe632045-37f4-40ae-8790-ec28c9fe5ded None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:53.773542 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:53.790832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abfe9d-cb3e-4669-86a9-c005d1d1b8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:53.792378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abfe9d-cb3e-4669-86a9-c005d1d1b8aa None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:53.792676 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abfe9d-cb3e-4669-86a9-c005d1d1b8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:53.793291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abfe9d-cb3e-4669-86a9-c005d1d1b8aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:53.804773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abfe9d-cb3e-4669-86a9-c005d1d1b8aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:53.806704 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abfe9d-cb3e-4669-86a9-c005d1d1b8aa None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:53.815916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb342c9-f086-4b13-a615-444d57e0e783 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-c6abfe9d-cb3e-4669-86a9-c005d1d1b8aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:46:53.828395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:53.831007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:53.831911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:53.831911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:53.831911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:46:53.831911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:46:53.831911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:46:53.831911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:46:53.831911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:46:53.831911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:46:53.831911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:46:53.831911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:53.831911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 258/522] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:46:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:53.834216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abfe9d-cb3e-4669-86a9-c005d1d1b8aa None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:53.860956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56667228-e272-4cf5-a5f2-19cf3a1e27e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:53.862022 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56667228-e272-4cf5-a5f2-19cf3a1e27e6 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:53.862566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56667228-e272-4cf5-a5f2-19cf3a1e27e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:53.863143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56667228-e272-4cf5-a5f2-19cf3a1e27e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:53.863898 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56667228-e272-4cf5-a5f2-19cf3a1e27e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:53.864224 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56667228-e272-4cf5-a5f2-19cf3a1e27e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:53.865775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56667228-e272-4cf5-a5f2-19cf3a1e27e6 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:53.884748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 265/523] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:53.886666 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56667228-e272-4cf5-a5f2-19cf3a1e27e6 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:53.891044 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:53.951112 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abfe9d-cb3e-4669-86a9-c005d1d1b8aa None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:53.992651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca609198-d0ac-48b6-8842-49dcf0ab8db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:53.993307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca609198-d0ac-48b6-8842-49dcf0ab8db6 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:53.993589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca609198-d0ac-48b6-8842-49dcf0ab8db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:53.993859 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca609198-d0ac-48b6-8842-49dcf0ab8db6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:53.994691 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca609198-d0ac-48b6-8842-49dcf0ab8db6 None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=0c0e71329c4a4470935593636c025257) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:53.995003 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca609198-d0ac-48b6-8842-49dcf0ab8db6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:53.996581 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca609198-d0ac-48b6-8842-49dcf0ab8db6 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:54.002944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 259/524] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:53 2022] GET /identity/v3/role_assignments?user.id=0c0e71329c4a4470935593636c025257&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:54.008539 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca609198-d0ac-48b6-8842-49dcf0ab8db6 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:54.008819 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56667228-e272-4cf5-a5f2-19cf3a1e27e6 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:54.034746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:54.048854 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb4f26-4569-43b7-a7b2-972fbc0e7f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:54.049811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb4f26-4569-43b7-a7b2-972fbc0e7f5b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:54.050113 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb4f26-4569-43b7-a7b2-972fbc0e7f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:54.050380 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb4f26-4569-43b7-a7b2-972fbc0e7f5b None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:54.051247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb4f26-4569-43b7-a7b2-972fbc0e7f5b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:54.051877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb4f26-4569-43b7-a7b2-972fbc0e7f5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:54.053393 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb4f26-4569-43b7-a7b2-972fbc0e7f5b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:54.078385 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 266/525] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 19:46:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:54.079375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb4f26-4569-43b7-a7b2-972fbc0e7f5b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:54.148070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca609198-d0ac-48b6-8842-49dcf0ab8db6 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:54.172768 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:54.191216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e964aeb-d9ad-4e9b-9216-5e2d9bf3f422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:54.192217 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e964aeb-d9ad-4e9b-9216-5e2d9bf3f422 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:54.192690 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e964aeb-d9ad-4e9b-9216-5e2d9bf3f422 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:54.193248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e964aeb-d9ad-4e9b-9216-5e2d9bf3f422 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:54.194335 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e964aeb-d9ad-4e9b-9216-5e2d9bf3f422 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:54.194837 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e964aeb-d9ad-4e9b-9216-5e2d9bf3f422 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:54.196510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e964aeb-d9ad-4e9b-9216-5e2d9bf3f422 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:54.217501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb4f26-4569-43b7-a7b2-972fbc0e7f5b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:54.224370 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 260/526] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 19:46:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:54.224370 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e964aeb-d9ad-4e9b-9216-5e2d9bf3f422 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:54.273534 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e6207-7d25-4e2d-8e6f-582d149ae324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:54.274754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e6207-7d25-4e2d-8e6f-582d149ae324 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:54.275050 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e6207-7d25-4e2d-8e6f-582d149ae324 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:54.275339 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e6207-7d25-4e2d-8e6f-582d149ae324 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:54.286124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e6207-7d25-4e2d-8e6f-582d149ae324 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:54.287921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e6207-7d25-4e2d-8e6f-582d149ae324 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-e70e6207-7d25-4e2d-8e6f-582d149ae324 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:46:54.307431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:54.308527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:54.309550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:46:54.309550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:46:54.309550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:46:54.309550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:46:54.309550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:46:54.309550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:46:54.309550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:54.309550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:54.309550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 267/527] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:54.310080 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e6207-7d25-4e2d-8e6f-582d149ae324 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:54.337905 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e964aeb-d9ad-4e9b-9216-5e2d9bf3f422 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:54.377322 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b147877-49c0-41a8-8b46-b9ad5cd23750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:54.378154 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147877-49c0-41a8-8b46-b9ad5cd23750 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:54.378536 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147877-49c0-41a8-8b46-b9ad5cd23750 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:54.378911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b147877-49c0-41a8-8b46-b9ad5cd23750 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:54.384859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:54.431481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:54.450510 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e6207-7d25-4e2d-8e6f-582d149ae324 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:54.471412 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b147877-49c0-41a8-8b46-b9ad5cd23750 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:54.473988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b147877-49c0-41a8-8b46-b9ad5cd23750 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:54.493353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c45e8c-4325-4c44-a2ac-59f0f9ebfbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:54.494112 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c45e8c-4325-4c44-a2ac-59f0f9ebfbfa None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:54.494396 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c45e8c-4325-4c44-a2ac-59f0f9ebfbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:54.494675 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c45e8c-4325-4c44-a2ac-59f0f9ebfbfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:54.575534 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c45e8c-4325-4c44-a2ac-59f0f9ebfbfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:54.577739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c45e8c-4325-4c44-a2ac-59f0f9ebfbfa None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:54.625980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 261/528] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:54.626942 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b147877-49c0-41a8-8b46-b9ad5cd23750 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:54.628747 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:54.699081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 268/529] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:54.700201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c45e8c-4325-4c44-a2ac-59f0f9ebfbfa None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:54.704984 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:54.735394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b147877-49c0-41a8-8b46-b9ad5cd23750 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:54.772551 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf4ac7-f23d-4da9-b2d5-d9f1bf87bfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:54.773306 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf4ac7-f23d-4da9-b2d5-d9f1bf87bfac None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:54.773658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf4ac7-f23d-4da9-b2d5-d9f1bf87bfac None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:54.774001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf4ac7-f23d-4da9-b2d5-d9f1bf87bfac None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:54.774857 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf4ac7-f23d-4da9-b2d5-d9f1bf87bfac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:54.775235 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf4ac7-f23d-4da9-b2d5-d9f1bf87bfac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:54.776809 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf4ac7-f23d-4da9-b2d5-d9f1bf87bfac None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:54.789736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 262/530] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:46:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:54.790816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf4ac7-f23d-4da9-b2d5-d9f1bf87bfac None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:54.810284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c45e8c-4325-4c44-a2ac-59f0f9ebfbfa None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:54.851313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991051b3-f815-460f-b6ac-8deadfb15e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:54.852239 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991051b3-f815-460f-b6ac-8deadfb15e9c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:54.853000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991051b3-f815-460f-b6ac-8deadfb15e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:54.853283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991051b3-f815-460f-b6ac-8deadfb15e9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:54.865048 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:54.868300 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991051b3-f815-460f-b6ac-8deadfb15e9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:54.869931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991051b3-f815-460f-b6ac-8deadfb15e9c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-991051b3-f815-460f-b6ac-8deadfb15e9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:54.890876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:54.893124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:54.895283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:54.895283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:54.895283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:54.895283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:54.895283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 269/531] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:54.895283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991051b3-f815-460f-b6ac-8deadfb15e9c None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:54.903130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf4ac7-f23d-4da9-b2d5-d9f1bf87bfac None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:54.937273 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:54.942209 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837eb25a-8057-4249-8b8d-a090c8177d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:54.943341 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837eb25a-8057-4249-8b8d-a090c8177d42 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:54.943341 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837eb25a-8057-4249-8b8d-a090c8177d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:54.943583 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837eb25a-8057-4249-8b8d-a090c8177d42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:54.956672 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837eb25a-8057-4249-8b8d-a090c8177d42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:54.958450 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837eb25a-8057-4249-8b8d-a090c8177d42 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-837eb25a-8057-4249-8b8d-a090c8177d42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:54.980944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:54.982058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:54.983635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:54.983635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:54.983635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:54.983635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:54.986551 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 263/532] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:54.987161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837eb25a-8057-4249-8b8d-a090c8177d42 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:55.020952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991051b3-f815-460f-b6ac-8deadfb15e9c None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:55.061012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcd2c8-9696-49b4-add9-13cb532c3495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:55.062026 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd2c8-9696-49b4-add9-13cb532c3495 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:55.062452 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd2c8-9696-49b4-add9-13cb532c3495 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:55.062897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcd2c8-9696-49b4-add9-13cb532c3495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:55.063932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcd2c8-9696-49b4-add9-13cb532c3495 None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=abcddf66e05c418cb148b321f2423aab) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:55.064442 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcd2c8-9696-49b4-add9-13cb532c3495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:55.066117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fcd2c8-9696-49b4-add9-13cb532c3495 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:55.072976 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 270/533] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:54 2022] GET /identity/v3/role_assignments?user.id=abcddf66e05c418cb148b321f2423aab&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:55.074070 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcd2c8-9696-49b4-add9-13cb532c3495 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:55.101664 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837eb25a-8057-4249-8b8d-a090c8177d42 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:55.141204 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d22bf-b3e4-448d-86fc-abdeb9a24f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:55.141997 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d22bf-b3e4-448d-86fc-abdeb9a24f67 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:55.142407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d22bf-b3e4-448d-86fc-abdeb9a24f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:55.142705 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d22bf-b3e4-448d-86fc-abdeb9a24f67 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:55.143506 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d22bf-b3e4-448d-86fc-abdeb9a24f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:55.143885 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d22bf-b3e4-448d-86fc-abdeb9a24f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:55.144131 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:55.145587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d22bf-b3e4-448d-86fc-abdeb9a24f67 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:55.160617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 264/534] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:55.173437 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d22bf-b3e4-448d-86fc-abdeb9a24f67 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:55.198637 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcd2c8-9696-49b4-add9-13cb532c3495 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:55.240985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d1246-2d98-4e73-b1fc-5836ce484ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:55.242081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d1246-2d98-4e73-b1fc-5836ce484ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:55.242613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d1246-2d98-4e73-b1fc-5836ce484ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:55.243091 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d1246-2d98-4e73-b1fc-5836ce484ba6 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:55.244202 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d1246-2d98-4e73-b1fc-5836ce484ba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:55.244743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d1246-2d98-4e73-b1fc-5836ce484ba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:55.246364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d1246-2d98-4e73-b1fc-5836ce484ba6 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:55.261801 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 271/535] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:55.269796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d1246-2d98-4e73-b1fc-5836ce484ba6 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:55.301955 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:55.438198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d22bf-b3e4-448d-86fc-abdeb9a24f67 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:55.465475 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d1246-2d98-4e73-b1fc-5836ce484ba6 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:55.478155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70d9c1-fecd-487b-943c-bb03e0d24c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:55.486138 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d9c1-fecd-487b-943c-bb03e0d24c4a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:55.486138 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d9c1-fecd-487b-943c-bb03e0d24c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:55.486138 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d9c1-fecd-487b-943c-bb03e0d24c4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:55.486479 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:55.488193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70d9c1-fecd-487b-943c-bb03e0d24c4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:55.489797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70d9c1-fecd-487b-943c-bb03e0d24c4a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-cb70d9c1-fecd-487b-943c-bb03e0d24c4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:46:55.507481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:55.508435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:55.509380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:46:55.509380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:46:55.509380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:46:55.509380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:46:55.509380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:46:55.509380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:46:55.509380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:55.509380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:55.509380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 265/536] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:55.515847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70d9c1-fecd-487b-943c-bb03e0d24c4a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:55.535964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d79f79-1902-4df1-9503-37e1325e68c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:55.537236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d79f79-1902-4df1-9503-37e1325e68c3 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:55.537741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d79f79-1902-4df1-9503-37e1325e68c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:55.538255 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d79f79-1902-4df1-9503-37e1325e68c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:55.554367 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d79f79-1902-4df1-9503-37e1325e68c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:55.558933 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d79f79-1902-4df1-9503-37e1325e68c3 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-23d79f79-1902-4df1-9503-37e1325e68c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:46:55.581454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:55.582497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:55.583325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:46:55.583325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:46:55.583325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:46:55.583325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:46:55.583325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:46:55.583325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:46:55.583325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:55.583325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:55.597508 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 272/537] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:46:55.603469 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d79f79-1902-4df1-9503-37e1325e68c3 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:55.781643 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70d9c1-fecd-487b-943c-bb03e0d24c4a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:55.788672 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:55.843340 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec111a-14fa-4b10-88b8-31d3a414baf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:55.844019 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec111a-14fa-4b10-88b8-31d3a414baf7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:55.844391 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec111a-14fa-4b10-88b8-31d3a414baf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:55.844685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec111a-14fa-4b10-88b8-31d3a414baf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:55.845543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec111a-14fa-4b10-88b8-31d3a414baf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:55.845865 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec111a-14fa-4b10-88b8-31d3a414baf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:55.847458 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec111a-14fa-4b10-88b8-31d3a414baf7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:55.860617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 266/538] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:55.869392 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec111a-14fa-4b10-88b8-31d3a414baf7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:55.875647 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d79f79-1902-4df1-9503-37e1325e68c3 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:55.953645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089be69-6ae2-4719-80b3-d81fe204a8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:55.954385 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089be69-6ae2-4719-80b3-d81fe204a8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:55.954671 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089be69-6ae2-4719-80b3-d81fe204a8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:55.954951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089be69-6ae2-4719-80b3-d81fe204a8e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:55.955887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089be69-6ae2-4719-80b3-d81fe204a8e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:55.956201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089be69-6ae2-4719-80b3-d81fe204a8e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:55.957669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089be69-6ae2-4719-80b3-d81fe204a8e9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:55.995932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 273/539] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:46:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:56.003908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089be69-6ae2-4719-80b3-d81fe204a8e9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:56.058296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec111a-14fa-4b10-88b8-31d3a414baf7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:56.073928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:56.138283 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6100ab-9d78-4ed3-95e0-a3ee222207f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:56.139497 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6100ab-9d78-4ed3-95e0-a3ee222207f2 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:56.139784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6100ab-9d78-4ed3-95e0-a3ee222207f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:56.140064 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6100ab-9d78-4ed3-95e0-a3ee222207f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:56.140943 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6100ab-9d78-4ed3-95e0-a3ee222207f2 None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=86eeca6ef9374305b98ac8f8b03f2fc5) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:56.141259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6100ab-9d78-4ed3-95e0-a3ee222207f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:56.142768 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6100ab-9d78-4ed3-95e0-a3ee222207f2 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:56.164456 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 267/540] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:55 2022] GET /identity/v3/role_assignments?user.id=86eeca6ef9374305b98ac8f8b03f2fc5&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 259 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:56.215259 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089be69-6ae2-4719-80b3-d81fe204a8e9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:56.251049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca03890-4683-41b0-b25e-4476f5a7ab68 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:56.251678 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca03890-4683-41b0-b25e-4476f5a7ab68 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:56.252223 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca03890-4683-41b0-b25e-4476f5a7ab68 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:56.254098 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 268/541] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:56.259954 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18487b-b017-4dd4-a481-0733ecf5b263 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:56.260567 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18487b-b017-4dd4-a481-0733ecf5b263 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:56.261059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18487b-b017-4dd4-a481-0733ecf5b263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:56.262252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f544225-357b-4a50-8fc6-b4e351c6216d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:56.262927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f544225-357b-4a50-8fc6-b4e351c6216d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:56.263214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f544225-357b-4a50-8fc6-b4e351c6216d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:56.263498 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f544225-357b-4a50-8fc6-b4e351c6216d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:56.264676 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f544225-357b-4a50-8fc6-b4e351c6216d None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=6d5a01edfe0444f8a6fef9d7ee9645f3) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:56.264998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f544225-357b-4a50-8fc6-b4e351c6216d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:56.266551 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f544225-357b-4a50-8fc6-b4e351c6216d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:56.282978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 274/542] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:46:56 2022] GET /identity/v3/role_assignments?user.id=6d5a01edfe0444f8a6fef9d7ee9645f3&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:56.297593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-9c18487b-b017-4dd4-a481-0733ecf5b263 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:56.299205 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-9c18487b-b017-4dd4-a481-0733ecf5b263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:56.348600 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:56.442999 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 269/543] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:56.487577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f544225-357b-4a50-8fc6-b4e351c6216d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:56.620849 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:56.623914 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f544225-357b-4a50-8fc6-b4e351c6216d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:56.671895 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252443ea-04f4-4d62-a439-4c7d253f8b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:56.672724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252443ea-04f4-4d62-a439-4c7d253f8b98 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:56.673011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252443ea-04f4-4d62-a439-4c7d253f8b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:56.673285 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252443ea-04f4-4d62-a439-4c7d253f8b98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:56.688623 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252443ea-04f4-4d62-a439-4c7d253f8b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-252443ea-04f4-4d62-a439-4c7d253f8b98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:56.696500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:56.697415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:56.698629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 275/544] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:46:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:56.704161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18487b-b017-4dd4-a481-0733ecf5b263 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:56.710197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f658-6150-443f-b74d-10c772349c08 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:56.710488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f658-6150-443f-b74d-10c772349c08 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:56.710771 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87f658-6150-443f-b74d-10c772349c08 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:56.712391 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 276/545] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:46:56.722204 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24399b14-f527-4491-af9d-833b8f7f7120 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:56.722494 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24399b14-f527-4491-af9d-833b8f7f7120 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:56.722756 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24399b14-f527-4491-af9d-833b8f7f7120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:56.753440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-24399b14-f527-4491-af9d-833b8f7f7120 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:56.755126 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-24399b14-f527-4491-af9d-833b8f7f7120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:56.867202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c18487b-b017-4dd4-a481-0733ecf5b263 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:56.883180 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 277/546] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:46:56.907304 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2dd1e8-ce84-4509-97e6-5d52d047b633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:56.907975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dd1e8-ce84-4509-97e6-5d52d047b633 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:56.908257 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dd1e8-ce84-4509-97e6-5d52d047b633 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:56.908556 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2dd1e8-ce84-4509-97e6-5d52d047b633 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:56.909300 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2dd1e8-ce84-4509-97e6-5d52d047b633 None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:56.909601 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2dd1e8-ce84-4509-97e6-5d52d047b633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:56.911048 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2dd1e8-ce84-4509-97e6-5d52d047b633 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:56.917387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24399b14-f527-4491-af9d-833b8f7f7120 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:56.921220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 270/547] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:46:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:56.928235 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2dd1e8-ce84-4509-97e6-5d52d047b633 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:56.953348 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:57.032845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:57.045781 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24399b14-f527-4491-af9d-833b8f7f7120 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:57.084061 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb7380-a852-42c1-9c4b-803bafb26fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:57.084898 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb7380-a852-42c1-9c4b-803bafb26fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:57.085182 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb7380-a852-42c1-9c4b-803bafb26fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:57.085453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb7380-a852-42c1-9c4b-803bafb26fa4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:57.086017 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2dd1e8-ce84-4509-97e6-5d52d047b633 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:57.098442 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb7380-a852-42c1-9c4b-803bafb26fa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-10cb7380-a852-42c1-9c4b-803bafb26fa4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:57.106271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:57.107155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:57.109613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 278/548] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:46:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:57.113482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb7380-a852-42c1-9c4b-803bafb26fa4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:57.147288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a07f98-7408-44db-98cb-2ee484e2e904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:57.148015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a07f98-7408-44db-98cb-2ee484e2e904 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:57.148300 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a07f98-7408-44db-98cb-2ee484e2e904 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:57.148606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a07f98-7408-44db-98cb-2ee484e2e904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:57.172671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a07f98-7408-44db-98cb-2ee484e2e904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:57.174163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a07f98-7408-44db-98cb-2ee484e2e904 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-a0a07f98-7408-44db-98cb-2ee484e2e904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:57.216715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:57.218822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:57.220912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:57.220912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:57.220912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:57.220912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:57.223827 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 271/549] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:46:57.228378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a07f98-7408-44db-98cb-2ee484e2e904 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:57.283539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb7380-a852-42c1-9c4b-803bafb26fa4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:57.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa733b-3c3e-4328-b14e-9b7b57ba912e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:57.333978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa733b-3c3e-4328-b14e-9b7b57ba912e None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:57.334267 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa733b-3c3e-4328-b14e-9b7b57ba912e None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:57.334542 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa733b-3c3e-4328-b14e-9b7b57ba912e None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:57.335309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa733b-3c3e-4328-b14e-9b7b57ba912e None admin] RBAC: Adding query filter params (name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:57.335614 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa733b-3c3e-4328-b14e-9b7b57ba912e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:57.339057 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa733b-3c3e-4328-b14e-9b7b57ba912e None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:57.343674 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:57.350838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 279/550] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:46:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:57.361402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa733b-3c3e-4328-b14e-9b7b57ba912e None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:57.377998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a07f98-7408-44db-98cb-2ee484e2e904 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:57.391910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:57.450383 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243179-cc53-4523-90d6-f3427c212ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:57.451321 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243179-cc53-4523-90d6-f3427c212ebc None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:57.451754 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243179-cc53-4523-90d6-f3427c212ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:57.452172 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243179-cc53-4523-90d6-f3427c212ebc None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:57.453156 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e243179-cc53-4523-90d6-f3427c212ebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:57.453627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e243179-cc53-4523-90d6-f3427c212ebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:57.455179 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e243179-cc53-4523-90d6-f3427c212ebc None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:57.475810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 272/551] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:46:57.485416 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243179-cc53-4523-90d6-f3427c212ebc None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:57.565226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa733b-3c3e-4328-b14e-9b7b57ba912e None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:57.635006 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef161f-e85f-46a5-8ccd-6c96046b12df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:57.636128 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef161f-e85f-46a5-8ccd-6c96046b12df None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:57.636602 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef161f-e85f-46a5-8ccd-6c96046b12df None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:57.637032 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef161f-e85f-46a5-8ccd-6c96046b12df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:57.666959 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef161f-e85f-46a5-8ccd-6c96046b12df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:57.668440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef161f-e85f-46a5-8ccd-6c96046b12df None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:57.690102 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243179-cc53-4523-90d6-f3427c212ebc None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-aeef161f-e85f-46a5-8ccd-6c96046b12df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:57.702313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:57.703225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:57.704089 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:57.704089 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:57.704089 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:57.704089 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:57.704089 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 280/552] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:57.709276 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef161f-e85f-46a5-8ccd-6c96046b12df None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:57.745161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:57.750768 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1e03a-5b14-4d9a-a956-f58ff80b911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:57.751923 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1e03a-5b14-4d9a-a956-f58ff80b911a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:57.752295 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1e03a-5b14-4d9a-a956-f58ff80b911a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:57.752618 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1e03a-5b14-4d9a-a956-f58ff80b911a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:57.764753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1e03a-5b14-4d9a-a956-f58ff80b911a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:57.767282 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1e03a-5b14-4d9a-a956-f58ff80b911a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-5ae1e03a-5b14-4d9a-a956-f58ff80b911a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:46:57.788287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:57.789343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:57.790271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:57.790271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:57.790271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:46:57.790271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:46:57.790271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:46:57.790271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:46:57.790271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:46:57.790271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:46:57.790271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:46:57.790271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:57.790271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 273/553] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:46:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:57.791169 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a18d4-37f6-4f73-b039-eab99ddbc773 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:57.791392 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a18d4-37f6-4f73-b039-eab99ddbc773 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:57.793115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a18d4-37f6-4f73-b039-eab99ddbc773 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:57.799575 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 274/554] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:57.802328 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a18d4-37f6-4f73-b039-eab99ddbc773 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:57.811785 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:57.866983 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef161f-e85f-46a5-8ccd-6c96046b12df None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:57.912428 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0100e0-59f1-4332-a448-18e09e48d2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:57.913815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0100e0-59f1-4332-a448-18e09e48d2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:57.914525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0100e0-59f1-4332-a448-18e09e48d2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:57.914954 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0100e0-59f1-4332-a448-18e09e48d2b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:57.915997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0100e0-59f1-4332-a448-18e09e48d2b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:57.916629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0100e0-59f1-4332-a448-18e09e48d2b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:57.918014 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0100e0-59f1-4332-a448-18e09e48d2b4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:57.928943 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a18d4-37f6-4f73-b039-eab99ddbc773 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:57.933670 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 281/555] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:57.935038 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d131e-b629-4577-951c-1c04471132e5 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:57.935343 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d131e-b629-4577-951c-1c04471132e5 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:57.936157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d131e-b629-4577-951c-1c04471132e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:57.956185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-509d131e-b629-4577-951c-1c04471132e5 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:57.957697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-509d131e-b629-4577-951c-1c04471132e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:57.975661 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cb172-5f33-45be-bf0b-3bb372ce1910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:57.977073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cb172-5f33-45be-bf0b-3bb372ce1910 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:57.977794 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cb172-5f33-45be-bf0b-3bb372ce1910 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:57.978212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cb172-5f33-45be-bf0b-3bb372ce1910 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:57.979537 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cb172-5f33-45be-bf0b-3bb372ce1910 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:57.980386 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cb172-5f33-45be-bf0b-3bb372ce1910 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:57.981991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cb172-5f33-45be-bf0b-3bb372ce1910 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:58.016642 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 275/556] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 19:46:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:58.016642 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cb172-5f33-45be-bf0b-3bb372ce1910 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:58.052474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 282/557] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:58.053683 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d131e-b629-4577-951c-1c04471132e5 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:58.074165 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:58.131148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cb172-5f33-45be-bf0b-3bb372ce1910 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:58.158777 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:58.172088 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79603b79-9530-4e14-a255-a0e30c31bfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:58.173358 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79603b79-9530-4e14-a255-a0e30c31bfef None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:58.173808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79603b79-9530-4e14-a255-a0e30c31bfef None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:58.174244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79603b79-9530-4e14-a255-a0e30c31bfef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:58.183256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79603b79-9530-4e14-a255-a0e30c31bfef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:58.185215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79603b79-9530-4e14-a255-a0e30c31bfef None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-79603b79-9530-4e14-a255-a0e30c31bfef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:46:58.202801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:58.204293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:58.205252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:58.205252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:58.205252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:46:58.205252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:46:58.205252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:46:58.205252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:46:58.205252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:46:58.205252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:46:58.205252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:46:58.205252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:58.208010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 276/558] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 19:46:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:58.209199 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79603b79-9530-4e14-a255-a0e30c31bfef None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:58.211073 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d131e-b629-4577-951c-1c04471132e5 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:58.250577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc0ae-17bb-418c-ba10-51a27d0abced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:58.251037 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc0ae-17bb-418c-ba10-51a27d0abced None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:58.251324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc0ae-17bb-418c-ba10-51a27d0abced None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:58.251601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dc0ae-17bb-418c-ba10-51a27d0abced None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:58.324050 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79603b79-9530-4e14-a255-a0e30c31bfef None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:58.330478 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dc0ae-17bb-418c-ba10-51a27d0abced None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:58.331966 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dc0ae-17bb-418c-ba10-51a27d0abced None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:58.337657 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:58.364111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0392a-4f2d-488e-8e62-19126c44d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:58.365446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0392a-4f2d-488e-8e62-19126c44d481 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:58.365932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0392a-4f2d-488e-8e62-19126c44d481 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:58.366434 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0392a-4f2d-488e-8e62-19126c44d481 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:58.380433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0392a-4f2d-488e-8e62-19126c44d481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-ffe0392a-4f2d-488e-8e62-19126c44d481 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:58.389627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:58.390656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:58.391504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:58.393088 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:58.395353 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 277/559] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:46:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:58.396431 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0392a-4f2d-488e-8e62-19126c44d481 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:58.452775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 283/560] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:46:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:46:58.452775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc0ae-17bb-418c-ba10-51a27d0abced None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:58.603106 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0392a-4f2d-488e-8e62-19126c44d481 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:58.645047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c466-0b6b-4c78-b8ab-6a12b991bd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:58.646090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c466-0b6b-4c78-b8ab-6a12b991bd20 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:58.646576 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c466-0b6b-4c78-b8ab-6a12b991bd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:58.647146 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0c466-0b6b-4c78-b8ab-6a12b991bd20 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:58.648282 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0c466-0b6b-4c78-b8ab-6a12b991bd20 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:58.648862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0c466-0b6b-4c78-b8ab-6a12b991bd20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:58.650541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0c466-0b6b-4c78-b8ab-6a12b991bd20 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:58.658349 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dc0ae-17bb-418c-ba10-51a27d0abced None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:58.672057 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 278/561] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 19:46:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:58.673397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c466-0b6b-4c78-b8ab-6a12b991bd20 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:58.705069 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:58.722286 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc7b80-2503-4521-bcf1-8b1f64047e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:58.723637 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc7b80-2503-4521-bcf1-8b1f64047e70 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:58.724388 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc7b80-2503-4521-bcf1-8b1f64047e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:58.724552 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc7b80-2503-4521-bcf1-8b1f64047e70 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:58.725917 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc7b80-2503-4521-bcf1-8b1f64047e70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:58.726435 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc7b80-2503-4521-bcf1-8b1f64047e70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:58.729186 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc7b80-2503-4521-bcf1-8b1f64047e70 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:58.741036 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 284/562] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:46:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:58.746710 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc7b80-2503-4521-bcf1-8b1f64047e70 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:58.766711 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:58.792871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0c466-0b6b-4c78-b8ab-6a12b991bd20 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:58.836515 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304e2fb-1268-4ca8-9f5d-487aca72db60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:58.837375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304e2fb-1268-4ca8-9f5d-487aca72db60 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:58.837654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304e2fb-1268-4ca8-9f5d-487aca72db60 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:58.837931 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304e2fb-1268-4ca8-9f5d-487aca72db60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:58.851388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304e2fb-1268-4ca8-9f5d-487aca72db60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:58.853092 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304e2fb-1268-4ca8-9f5d-487aca72db60 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:58.863021 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc7b80-2503-4521-bcf1-8b1f64047e70 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-c304e2fb-1268-4ca8-9f5d-487aca72db60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:58.874095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:58.875075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:58.877532 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:58.877532 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:58.877532 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:58.877532 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:58.877532 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 279/563] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:58.880974 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304e2fb-1268-4ca8-9f5d-487aca72db60 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:58.904860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da3158-8a7f-400a-8223-aa6563102139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:58.905807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da3158-8a7f-400a-8223-aa6563102139 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:58.906223 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da3158-8a7f-400a-8223-aa6563102139 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:58.906505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da3158-8a7f-400a-8223-aa6563102139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:58.919460 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da3158-8a7f-400a-8223-aa6563102139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:58.921065 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da3158-8a7f-400a-8223-aa6563102139 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-85da3158-8a7f-400a-8223-aa6563102139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:58.941417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:58.942579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:58.943832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:58.943832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:58.943832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:58.943832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:58.943832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 285/564] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:58.943832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da3158-8a7f-400a-8223-aa6563102139 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:58.985826 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:59.008038 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304e2fb-1268-4ca8-9f5d-487aca72db60 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:59.022817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:59.045534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5af713-09f9-4108-93ff-bdf6bb48744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:59.046328 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af713-09f9-4108-93ff-bdf6bb48744e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.046607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af713-09f9-4108-93ff-bdf6bb48744e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.046875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5af713-09f9-4108-93ff-bdf6bb48744e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.057890 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5af713-09f9-4108-93ff-bdf6bb48744e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:59.059368 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5af713-09f9-4108-93ff-bdf6bb48744e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:59.061540 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da3158-8a7f-400a-8223-aa6563102139 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-ee5af713-09f9-4108-93ff-bdf6bb48744e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:46:59.078448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:59.079460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:59.080362 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:46:59.080362 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:46:59.080362 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:46:59.080362 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:59.080362 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 280/565] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:46:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:59.080973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56fbd1-4388-47c3-811a-a48400090885 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.081264 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56fbd1-4388-47c3-811a-a48400090885 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.081529 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56fbd1-4388-47c3-811a-a48400090885 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.082808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 281/566] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:59.083512 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56fbd1-4388-47c3-811a-a48400090885 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:59.104431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f123625-f030-4b88-b0b9-c45394458391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:59.105224 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f123625-f030-4b88-b0b9-c45394458391 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.105514 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f123625-f030-4b88-b0b9-c45394458391 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.105805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f123625-f030-4b88-b0b9-c45394458391 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.106571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f123625-f030-4b88-b0b9-c45394458391 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:59.106881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f123625-f030-4b88-b0b9-c45394458391 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:59.108824 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f123625-f030-4b88-b0b9-c45394458391 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:59.120750 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 286/567] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:59.121951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0325259-ed6e-4d86-ade2-4da54d87c12e None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.122238 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0325259-ed6e-4d86-ade2-4da54d87c12e None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.122514 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0325259-ed6e-4d86-ade2-4da54d87c12e None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.124551 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 287/568] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:46:59.128953 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0325259-ed6e-4d86-ade2-4da54d87c12e None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:59.192300 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56fbd1-4388-47c3-811a-a48400090885 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:59.222274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:59.226942 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773459b-3c8a-4c74-9e02-603e9bce9bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:59.227598 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773459b-3c8a-4c74-9e02-603e9bce9bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.227883 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773459b-3c8a-4c74-9e02-603e9bce9bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.228157 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773459b-3c8a-4c74-9e02-603e9bce9bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773459b-3c8a-4c74-9e02-603e9bce9bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:59.229268 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773459b-3c8a-4c74-9e02-603e9bce9bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:59.230612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773459b-3c8a-4c74-9e02-603e9bce9bf7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:59.230936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0325259-ed6e-4d86-ade2-4da54d87c12e None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:59.242412 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 282/569] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:59.243623 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3cf4-c6d4-4131-8dc3-c92033b934c1 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.243913 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3cf4-c6d4-4131-8dc3-c92033b934c1 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.244192 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3cf4-c6d4-4131-8dc3-c92033b934c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.257522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:59.260081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-11fb3cf4-c6d4-4131-8dc3-c92033b934c1 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:59.261308 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-11fb3cf4-c6d4-4131-8dc3-c92033b934c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:59.264931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d89fb1-e03f-40a7-8828-4c0dc02950d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:59.265589 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d89fb1-e03f-40a7-8828-4c0dc02950d0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.265869 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d89fb1-e03f-40a7-8828-4c0dc02950d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.266171 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d89fb1-e03f-40a7-8828-4c0dc02950d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.268119 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d89fb1-e03f-40a7-8828-4c0dc02950d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:59.268119 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d89fb1-e03f-40a7-8828-4c0dc02950d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:59.268566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d89fb1-e03f-40a7-8828-4c0dc02950d0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:59.280428 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 288/570] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:59.281273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d89fb1-e03f-40a7-8828-4c0dc02950d0 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:59.331691 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 283/571] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:59.333120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cf3d-d443-4cac-be86-d95516d2dcc4 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.333410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cf3d-d443-4cac-be86-d95516d2dcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.333684 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cf3d-d443-4cac-be86-d95516d2dcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.351160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-26c7cf3d-d443-4cac-be86-d95516d2dcc4 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:46:59.352329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-26c7cf3d-d443-4cac-be86-d95516d2dcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:46:59.392862 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d89fb1-e03f-40a7-8828-4c0dc02950d0 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:59.418712 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 284/572] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:46:59.419549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7cf3d-d443-4cac-be86-d95516d2dcc4 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:59.427440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38efb98-22fb-48ea-8583-064a687152d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:59.428222 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38efb98-22fb-48ea-8583-064a687152d8 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38efb98-22fb-48ea-8583-064a687152d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.428831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38efb98-22fb-48ea-8583-064a687152d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.435408 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38efb98-22fb-48ea-8583-064a687152d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:59.436874 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38efb98-22fb-48ea-8583-064a687152d8 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-b38efb98-22fb-48ea-8583-064a687152d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:46:59.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:59.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:59.454209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:46:59.454209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:46:59.454209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:46:59.454209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:46:59.454209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:46:59.454209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:46:59.454209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:59.454209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:59.454209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 289/573] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:59.454714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38efb98-22fb-48ea-8583-064a687152d8 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:59.481168 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:59.503663 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:59.530182 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7cf3d-d443-4cac-be86-d95516d2dcc4 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:59.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38efb98-22fb-48ea-8583-064a687152d8 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:59.564544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7ae1d-0fa8-4915-b48d-3315c068b2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:59.565337 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7ae1d-0fa8-4915-b48d-3315c068b2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.565617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7ae1d-0fa8-4915-b48d-3315c068b2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.565894 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7ae1d-0fa8-4915-b48d-3315c068b2f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.572926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7ae1d-0fa8-4915-b48d-3315c068b2f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:59.574395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a7ae1d-0fa8-4915-b48d-3315c068b2f3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:59.587617 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c7063-0691-4e64-b135-52c8c27023ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:59.588424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c7063-0691-4e64-b135-52c8c27023ad None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.588706 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c7063-0691-4e64-b135-52c8c27023ad None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.588981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c7063-0691-4e64-b135-52c8c27023ad None admin] PATH_INFO: `/v3/users/glance` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-12a7ae1d-0fa8-4915-b48d-3315c068b2f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:46:59.589073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:59.589938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:59.590846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:46:59.590846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:46:59.590846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:46:59.590846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:46:59.590846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:46:59.590846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:46:59.590846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:46:59.590846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:59.590846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 285/574] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:59.591321 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7ae1d-0fa8-4915-b48d-3315c068b2f3 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:59.597895 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c7063-0691-4e64-b135-52c8c27023ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:59.599557 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c7063-0691-4e64-b135-52c8c27023ad None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-1f3c7063-0691-4e64-b135-52c8c27023ad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:46:59.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:59.615409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:59.616387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:59.616387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:59.616387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:46:59.616387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:46:59.616387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:46:59.616387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:46:59.616387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:46:59.616387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:46:59.616387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:46:59.616387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:59.616387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 290/575] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:46:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:59.617014 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c7063-0691-4e64-b135-52c8c27023ad None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:59.692154 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a7ae1d-0fa8-4915-b48d-3315c068b2f3 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:59.705208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:59.714813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c7063-0691-4e64-b135-52c8c27023ad None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:59.725364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:59.726119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc70df-1416-45a4-bdfb-f897c10c9523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:59.726908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc70df-1416-45a4-bdfb-f897c10c9523 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.727193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc70df-1416-45a4-bdfb-f897c10c9523 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.727467 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc70df-1416-45a4-bdfb-f897c10c9523 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.737896 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc70df-1416-45a4-bdfb-f897c10c9523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-b8bc70df-1416-45a4-bdfb-f897c10c9523 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:59.745952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:59.746980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:59.747790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:46:59.752629 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 286/576] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:46:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:59.752932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642e414-ade5-407b-aae6-ec0c6087b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:59.753485 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc70df-1416-45a4-bdfb-f897c10c9523 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:59.753791 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642e414-ade5-407b-aae6-ec0c6087b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.754070 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642e414-ade5-407b-aae6-ec0c6087b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.754343 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642e414-ade5-407b-aae6-ec0c6087b4d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.766843 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642e414-ade5-407b-aae6-ec0c6087b4d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-d642e414-ade5-407b-aae6-ec0c6087b4d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:46:59.774645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:46:59.775692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:46:59.776547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 291/577] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:46:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:46:59.777235 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642e414-ade5-407b-aae6-ec0c6087b4d2 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:59.859055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc70df-1416-45a4-bdfb-f897c10c9523 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:59.875784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642e414-ade5-407b-aae6-ec0c6087b4d2 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:46:59.892499 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff727245-6647-4f1e-8994-c77e890e7f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:59.893154 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727245-6647-4f1e-8994-c77e890e7f6f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.893432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727245-6647-4f1e-8994-c77e890e7f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.893768 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff727245-6647-4f1e-8994-c77e890e7f6f None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.894552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff727245-6647-4f1e-8994-c77e890e7f6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:59.894857 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff727245-6647-4f1e-8994-c77e890e7f6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:59.896296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff727245-6647-4f1e-8994-c77e890e7f6f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:59.908615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 287/578] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:59.909268 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff727245-6647-4f1e-8994-c77e890e7f6f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:59.914499 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5973131-7645-4131-8a39-a5149206546a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:46:59.915166 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5973131-7645-4131-8a39-a5149206546a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:46:59.915455 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5973131-7645-4131-8a39-a5149206546a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:46:59.915732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5973131-7645-4131-8a39-a5149206546a None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:46:59.916550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5973131-7645-4131-8a39-a5149206546a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:46:59.916862 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5973131-7645-4131-8a39-a5149206546a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:46:59.918283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5973131-7645-4131-8a39-a5149206546a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:46:59.930754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 292/579] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:46:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:46:59.931940 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5973131-7645-4131-8a39-a5149206546a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:46:59.941387 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:46:59.960280 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:00.011577 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff727245-6647-4f1e-8994-c77e890e7f6f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:00.029787 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5973131-7645-4131-8a39-a5149206546a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:00.044776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f790645-0aca-450b-acd3-2693770a9911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:00.045439 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f790645-0aca-450b-acd3-2693770a9911 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:00.045719 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f790645-0aca-450b-acd3-2693770a9911 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:00.045990 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f790645-0aca-450b-acd3-2693770a9911 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:00.046817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f790645-0aca-450b-acd3-2693770a9911 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:00.047125 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f790645-0aca-450b-acd3-2693770a9911 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:00.048586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f790645-0aca-450b-acd3-2693770a9911 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:00.064537 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793570f-f974-4432-b219-6cf9e47bbfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:00.065201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793570f-f974-4432-b219-6cf9e47bbfa8 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:00.065488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793570f-f974-4432-b219-6cf9e47bbfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:00.065765 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793570f-f974-4432-b219-6cf9e47bbfa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:00.066512 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793570f-f974-4432-b219-6cf9e47bbfa8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:00.066815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793570f-f974-4432-b219-6cf9e47bbfa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:00.067153 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 288/580] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 19:46:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:00.067948 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f790645-0aca-450b-acd3-2693770a9911 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:00.068188 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793570f-f974-4432-b219-6cf9e47bbfa8 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:00.081657 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 293/581] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:46:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:00.081657 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793570f-f974-4432-b219-6cf9e47bbfa8 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:00.162726 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:00.172450 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f790645-0aca-450b-acd3-2693770a9911 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:00.175714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:00.185107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793570f-f974-4432-b219-6cf9e47bbfa8 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:00.207457 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5f514-d282-4dea-88cd-dd0f38a99fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:00.217070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5f514-d282-4dea-88cd-dd0f38a99fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:00.217070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5f514-d282-4dea-88cd-dd0f38a99fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:00.217070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5f514-d282-4dea-88cd-dd0f38a99fc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:00.217070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5f514-d282-4dea-88cd-dd0f38a99fc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:00.217070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5f514-d282-4dea-88cd-dd0f38a99fc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:00.217070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5f514-d282-4dea-88cd-dd0f38a99fc0 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:00.219088 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 289/582] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:47:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:00.219854 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5f514-d282-4dea-88cd-dd0f38a99fc0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:00.220039 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b003b-eb2b-4a6f-b64a-c75b19bf63c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:00.220851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b003b-eb2b-4a6f-b64a-c75b19bf63c4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:00.221137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b003b-eb2b-4a6f-b64a-c75b19bf63c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:00.221408 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b003b-eb2b-4a6f-b64a-c75b19bf63c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:00.222239 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b003b-eb2b-4a6f-b64a-c75b19bf63c4 None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=c8a36bb1e26f4faaa44fabff8b509fe7) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:00.222550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b003b-eb2b-4a6f-b64a-c75b19bf63c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:00.224014 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b003b-eb2b-4a6f-b64a-c75b19bf63c4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:00.233784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 294/583] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:00 2022] GET /identity/v3/role_assignments?user.id=c8a36bb1e26f4faaa44fabff8b509fe7&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:00.233784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b003b-eb2b-4a6f-b64a-c75b19bf63c4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:00.348951 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5f514-d282-4dea-88cd-dd0f38a99fc0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:00.349388 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b003b-eb2b-4a6f-b64a-c75b19bf63c4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:00.391128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05faafe-d545-45de-abb2-90dfc115a7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:00.392041 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05faafe-d545-45de-abb2-90dfc115a7df None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:00.392325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05faafe-d545-45de-abb2-90dfc115a7df None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:00.392678 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05faafe-d545-45de-abb2-90dfc115a7df None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/0c0e71329c4a4470935593636c025257/roles/a615c7d83a0c46b1b26642bcc6e6ecef` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:00.403891 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b5d58-d051-41b2-b78b-51ff1e4a8b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:00.404766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b5d58-d051-41b2-b78b-51ff1e4a8b30 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:00.405050 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b5d58-d051-41b2-b78b-51ff1e4a8b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:00.405323 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b5d58-d051-41b2-b78b-51ff1e4a8b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:00.409875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05faafe-d545-45de-abb2-90dfc115a7df None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=0c0e71329c4a4470935593636c025257, role_id=a615c7d83a0c46b1b26642bcc6e6ecef)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:00.411307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05faafe-d545-45de-abb2-90dfc115a7df None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:00.421475 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b5d58-d051-41b2-b78b-51ff1e4a8b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:00.422950 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b5d58-d051-41b2-b78b-51ff1e4a8b30 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-8b0b5d58-d051-41b2-b78b-51ff1e4a8b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:00.441894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:00.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:00.444016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:00.444016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:00.444016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:00.444016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:00.444016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 295/584] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:00.444304 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b5d58-d051-41b2-b78b-51ff1e4a8b30 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:00.453373 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:00.485563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 290/585] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:47:00 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/0c0e71329c4a4470935593636c025257/roles/a615c7d83a0c46b1b26642bcc6e6ecef => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:47:00.486480 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05faafe-d545-45de-abb2-90dfc115a7df None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:00.489313 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:00.578572 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b5d58-d051-41b2-b78b-51ff1e4a8b30 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:00.625442 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f40e9-4000-4d87-ae21-0285f6408d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:00.626661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f40e9-4000-4d87-ae21-0285f6408d0b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:00.627113 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f40e9-4000-4d87-ae21-0285f6408d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:00.627548 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f40e9-4000-4d87-ae21-0285f6408d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:00.636894 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05faafe-d545-45de-abb2-90dfc115a7df None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:00.642556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f40e9-4000-4d87-ae21-0285f6408d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:00.644292 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f40e9-4000-4d87-ae21-0285f6408d0b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-b65f40e9-4000-4d87-ae21-0285f6408d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:00.665305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:00.666338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:00.667453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:00.667453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:00.667453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:00.667453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:00.669756 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 296/586] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:00.671025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f40e9-4000-4d87-ae21-0285f6408d0b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:00.690291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672ccc2-3234-4117-93c0-3211035f7f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:00.691690 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672ccc2-3234-4117-93c0-3211035f7f10 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:00.692063 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672ccc2-3234-4117-93c0-3211035f7f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:00.692551 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6672ccc2-3234-4117-93c0-3211035f7f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:00.706537 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672ccc2-3234-4117-93c0-3211035f7f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:00.708590 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6672ccc2-3234-4117-93c0-3211035f7f10 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-6672ccc2-3234-4117-93c0-3211035f7f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:00.740608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:00.741549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:00.742440 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:00.742440 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:00.742440 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:00.742440 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:00.742440 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 291/587] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:00.743183 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672ccc2-3234-4117-93c0-3211035f7f10 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:00.758814 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:00.804708 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f40e9-4000-4d87-ae21-0285f6408d0b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:00.813085 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:00.845230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df47c8-1e1a-46c2-ada0-2ae720ff3875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:00.846643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df47c8-1e1a-46c2-ada0-2ae720ff3875 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:00.847265 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df47c8-1e1a-46c2-ada0-2ae720ff3875 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:00.847735 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df47c8-1e1a-46c2-ada0-2ae720ff3875 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:00.849231 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df47c8-1e1a-46c2-ada0-2ae720ff3875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:00.849716 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df47c8-1e1a-46c2-ada0-2ae720ff3875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:00.852147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df47c8-1e1a-46c2-ada0-2ae720ff3875 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:00.853832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6672ccc2-3234-4117-93c0-3211035f7f10 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:00.869607 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 297/588] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:00.870720 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df47c8-1e1a-46c2-ada0-2ae720ff3875 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:00.895673 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93cc7e-a538-4751-a2b0-c72a2e525320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:00.896707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93cc7e-a538-4751-a2b0-c72a2e525320 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:00.897218 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93cc7e-a538-4751-a2b0-c72a2e525320 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:00.897673 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93cc7e-a538-4751-a2b0-c72a2e525320 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:00.898618 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93cc7e-a538-4751-a2b0-c72a2e525320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:00.899103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93cc7e-a538-4751-a2b0-c72a2e525320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:00.900756 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb93cc7e-a538-4751-a2b0-c72a2e525320 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:00.914666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 292/589] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:00.915893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93cc7e-a538-4751-a2b0-c72a2e525320 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:00.987028 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df47c8-1e1a-46c2-ada0-2ae720ff3875 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:01.006486 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:01.025247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf36223-e515-4712-bc01-c13e0909317a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:01.026152 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf36223-e515-4712-bc01-c13e0909317a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:01.026601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf36223-e515-4712-bc01-c13e0909317a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:01.026972 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf36223-e515-4712-bc01-c13e0909317a None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:01.027885 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf36223-e515-4712-bc01-c13e0909317a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:01.028303 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf36223-e515-4712-bc01-c13e0909317a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:01.029799 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf36223-e515-4712-bc01-c13e0909317a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:01.031037 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93cc7e-a538-4751-a2b0-c72a2e525320 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:01.043343 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 298/590] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:01.044557 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf36223-e515-4712-bc01-c13e0909317a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:01.048679 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:01.072520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb293c5-e71f-45ce-93de-fde603c9a4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:01.073596 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb293c5-e71f-45ce-93de-fde603c9a4db None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:01.073890 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb293c5-e71f-45ce-93de-fde603c9a4db None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:01.074167 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb293c5-e71f-45ce-93de-fde603c9a4db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:01.083061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb293c5-e71f-45ce-93de-fde603c9a4db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:01.084597 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb293c5-e71f-45ce-93de-fde603c9a4db None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-8eb293c5-e71f-45ce-93de-fde603c9a4db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:01.100310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:01.101269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:01.102161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:01.102161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:01.102161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:01.102161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:01.102161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:01.102161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:01.102161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:01.102161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:01.102161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 293/591] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:01.102641 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb293c5-e71f-45ce-93de-fde603c9a4db None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:01.167533 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf36223-e515-4712-bc01-c13e0909317a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:01.205809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a8c1d-3815-4115-a641-a45980a41cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:01.207016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8c1d-3815-4115-a641-a45980a41cba None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:01.207635 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8c1d-3815-4115-a641-a45980a41cba None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:01.208158 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a8c1d-3815-4115-a641-a45980a41cba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:01.216177 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb293c5-e71f-45ce-93de-fde603c9a4db None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:01.217794 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a8c1d-3815-4115-a641-a45980a41cba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:01.219749 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a8c1d-3815-4115-a641-a45980a41cba None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-653a8c1d-3815-4115-a641-a45980a41cba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:01.235447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:01.237043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:01.237997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:01.237997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:01.237997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:01.237997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:01.237997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:01.237997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:01.237997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:01.237997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:01.237997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:47:01.237997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:01.240644 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 299/592] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:47:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:01.241921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a8c1d-3815-4115-a641-a45980a41cba None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:01.253624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f9bf0-58f8-49ef-9437-012239359a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:01.254664 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f9bf0-58f8-49ef-9437-012239359a23 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:01.254867 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f9bf0-58f8-49ef-9437-012239359a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:01.255117 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f9bf0-58f8-49ef-9437-012239359a23 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:01.263241 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f9bf0-58f8-49ef-9437-012239359a23 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:01.265497 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f9bf0-58f8-49ef-9437-012239359a23 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-a05f9bf0-58f8-49ef-9437-012239359a23 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:01.283079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:01.284081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 294/593] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:47:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:01.285686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f9bf0-58f8-49ef-9437-012239359a23 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:01.290844 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:01.323156 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:01.364454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a8c1d-3815-4115-a641-a45980a41cba None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:01.401555 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76b708-e330-4c79-ae25-b8831a08c12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:01.402399 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76b708-e330-4c79-ae25-b8831a08c12b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:01.402787 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76b708-e330-4c79-ae25-b8831a08c12b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:01.403165 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76b708-e330-4c79-ae25-b8831a08c12b None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:01.403847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f9bf0-58f8-49ef-9437-012239359a23 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:01.404745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76b708-e330-4c79-ae25-b8831a08c12b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:01.405159 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76b708-e330-4c79-ae25-b8831a08c12b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:01.406809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76b708-e330-4c79-ae25-b8831a08c12b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:01.426171 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 300/594] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:01.427626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76b708-e330-4c79-ae25-b8831a08c12b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:01.445280 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318684c-f400-43a1-a464-9892556ed8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:01.446332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318684c-f400-43a1-a464-9892556ed8ad None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:01.446829 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318684c-f400-43a1-a464-9892556ed8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:01.447286 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318684c-f400-43a1-a464-9892556ed8ad None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:01.448450 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318684c-f400-43a1-a464-9892556ed8ad None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:01.449082 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318684c-f400-43a1-a464-9892556ed8ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:01.450707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318684c-f400-43a1-a464-9892556ed8ad None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:01.471343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 295/595] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 19:47:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:01.472461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318684c-f400-43a1-a464-9892556ed8ad None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:01.574619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:01.579908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76b708-e330-4c79-ae25-b8831a08c12b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:01.620719 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:01.629186 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186fcfb-473f-431f-870e-37ac0e0c00db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:01.630177 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186fcfb-473f-431f-870e-37ac0e0c00db None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:01.630465 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186fcfb-473f-431f-870e-37ac0e0c00db None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:01.630744 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186fcfb-473f-431f-870e-37ac0e0c00db None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:01.631595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186fcfb-473f-431f-870e-37ac0e0c00db None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:01.631905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186fcfb-473f-431f-870e-37ac0e0c00db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:01.632564 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318684c-f400-43a1-a464-9892556ed8ad None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:01.633380 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186fcfb-473f-431f-870e-37ac0e0c00db None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:01.656506 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 301/596] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 19:47:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:01.657438 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186fcfb-473f-431f-870e-37ac0e0c00db None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:01.680623 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71837542-b8dd-4d87-ae74-e34744bcae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:01.681780 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71837542-b8dd-4d87-ae74-e34744bcae60 None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:01.682066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71837542-b8dd-4d87-ae74-e34744bcae60 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:01.682339 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71837542-b8dd-4d87-ae74-e34744bcae60 None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/abcddf66e05c418cb148b321f2423aab/roles/a615c7d83a0c46b1b26642bcc6e6ecef` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:01.704877 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71837542-b8dd-4d87-ae74-e34744bcae60 None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=abcddf66e05c418cb148b321f2423aab, role_id=a615c7d83a0c46b1b26642bcc6e6ecef)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:01.707570 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71837542-b8dd-4d87-ae74-e34744bcae60 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:01.778119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 296/597] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:47:01 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/abcddf66e05c418cb148b321f2423aab/roles/a615c7d83a0c46b1b26642bcc6e6ecef => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:47:01.779204 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71837542-b8dd-4d87-ae74-e34744bcae60 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:01.788060 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186fcfb-473f-431f-870e-37ac0e0c00db None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:01.860115 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a8893-a5a0-42af-abc7-63eec04484ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:01.863326 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a8893-a5a0-42af-abc7-63eec04484ea None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:01.864081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a8893-a5a0-42af-abc7-63eec04484ea None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:01.864389 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a8893-a5a0-42af-abc7-63eec04484ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:01.887179 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a8893-a5a0-42af-abc7-63eec04484ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:01.890356 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a8893-a5a0-42af-abc7-63eec04484ea None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-9a7a8893-a5a0-42af-abc7-63eec04484ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:01.927338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:01.928302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:01.929244 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:01.929244 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:01.929244 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:01.929244 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:01.929921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 302/598] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:01.939522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a8893-a5a0-42af-abc7-63eec04484ea None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:01.958570 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:02.002195 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71837542-b8dd-4d87-ae74-e34744bcae60 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:02.051058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cb3a7-1308-43ae-a544-74a20355ca4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:02.052622 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cb3a7-1308-43ae-a544-74a20355ca4a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:02.052920 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cb3a7-1308-43ae-a544-74a20355ca4a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:02.053483 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cb3a7-1308-43ae-a544-74a20355ca4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:02.067778 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93cb3a7-1308-43ae-a544-74a20355ca4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:02.069735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93cb3a7-1308-43ae-a544-74a20355ca4a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:02.071710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-b93cb3a7-1308-43ae-a544-74a20355ca4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:02.093616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:02.095697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:02.097860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:02.097860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:02.097860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:02.097860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:02.097860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 297/599] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:02.098582 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a8893-a5a0-42af-abc7-63eec04484ea None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:02.104102 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cb3a7-1308-43ae-a544-74a20355ca4a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:02.145975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58fcb4-b06b-4b3d-b111-c2a6ec5c0905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:02.147367 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58fcb4-b06b-4b3d-b111-c2a6ec5c0905 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:02.147669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58fcb4-b06b-4b3d-b111-c2a6ec5c0905 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:02.147954 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58fcb4-b06b-4b3d-b111-c2a6ec5c0905 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:02.148727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58fcb4-b06b-4b3d-b111-c2a6ec5c0905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:02.149034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58fcb4-b06b-4b3d-b111-c2a6ec5c0905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:02.150408 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58fcb4-b06b-4b3d-b111-c2a6ec5c0905 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:02.165939 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 303/600] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:02.173340 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58fcb4-b06b-4b3d-b111-c2a6ec5c0905 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:02.241186 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93cb3a7-1308-43ae-a544-74a20355ca4a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:02.278731 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f240b91-2320-4856-b2ca-9e67ef8c0131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:02.280033 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f240b91-2320-4856-b2ca-9e67ef8c0131 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:02.280313 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f240b91-2320-4856-b2ca-9e67ef8c0131 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:02.280609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f240b91-2320-4856-b2ca-9e67ef8c0131 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:02.281388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f240b91-2320-4856-b2ca-9e67ef8c0131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:02.281694 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f240b91-2320-4856-b2ca-9e67ef8c0131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:02.283100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f240b91-2320-4856-b2ca-9e67ef8c0131 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:02.297025 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 298/601] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:02.302711 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f240b91-2320-4856-b2ca-9e67ef8c0131 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:02.363769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:02.372278 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:02.411885 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f240b91-2320-4856-b2ca-9e67ef8c0131 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:02.448636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2149c5-3c19-4d3b-99b5-32ab693beea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:02.449434 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2149c5-3c19-4d3b-99b5-32ab693beea2 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:02.452356 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2149c5-3c19-4d3b-99b5-32ab693beea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:02.452356 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2149c5-3c19-4d3b-99b5-32ab693beea2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:02.455715 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58fcb4-b06b-4b3d-b111-c2a6ec5c0905 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:02.457262 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2149c5-3c19-4d3b-99b5-32ab693beea2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:02.460523 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2149c5-3c19-4d3b-99b5-32ab693beea2 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-8e2149c5-3c19-4d3b-99b5-32ab693beea2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:02.498272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:02.499426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:02.500305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:02.500305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:02.500305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:02.500305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:02.500305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:02.500305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:02.500305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:02.500305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:02.500305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 299/602] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:02.509271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2149c5-3c19-4d3b-99b5-32ab693beea2 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:02.557156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a75a59-3972-43e3-94a8-b4b9e100e974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:02.558017 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a75a59-3972-43e3-94a8-b4b9e100e974 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:02.558299 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a75a59-3972-43e3-94a8-b4b9e100e974 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:02.558575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a75a59-3972-43e3-94a8-b4b9e100e974 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:02.589213 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a75a59-3972-43e3-94a8-b4b9e100e974 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:02.590742 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a75a59-3972-43e3-94a8-b4b9e100e974 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-d6a75a59-3972-43e3-94a8-b4b9e100e974 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:02.624807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:02.625784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:02.626697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:02.626697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:02.626697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:02.626697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:02.626697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:02.626697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:02.626697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:02.626697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:02.630873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2149c5-3c19-4d3b-99b5-32ab693beea2 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:02.636687 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 304/603] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 464 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:02.637651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a75a59-3972-43e3-94a8-b4b9e100e974 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:02.669037 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:02.681515 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f4b1e-6756-4bfa-8627-a18f0ad5c42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:02.682606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4b1e-6756-4bfa-8627-a18f0ad5c42d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:02.684247 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4b1e-6756-4bfa-8627-a18f0ad5c42d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:02.684706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4b1e-6756-4bfa-8627-a18f0ad5c42d None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:02.688260 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f4b1e-6756-4bfa-8627-a18f0ad5c42d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:02.688855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f4b1e-6756-4bfa-8627-a18f0ad5c42d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:02.690711 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f4b1e-6756-4bfa-8627-a18f0ad5c42d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:02.729020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 300/604] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:02.731078 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f161a6-297b-4178-8b7a-5b13ddde72cb None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:02.731475 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f161a6-297b-4178-8b7a-5b13ddde72cb None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:02.731856 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f161a6-297b-4178-8b7a-5b13ddde72cb None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:02.733648 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 301/605] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:02.739458 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42b750-42ff-4907-8338-4dac666b478f None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:02.739622 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42b750-42ff-4907-8338-4dac666b478f None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:02.740154 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42b750-42ff-4907-8338-4dac666b478f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:02.757752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-6d42b750-42ff-4907-8338-4dac666b478f None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:02.758988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-6d42b750-42ff-4907-8338-4dac666b478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:02.814727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a75a59-3972-43e3-94a8-b4b9e100e974 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:02.840602 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 302/606] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:47:02.842278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42b750-42ff-4907-8338-4dac666b478f None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:02.848934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:02.857598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3c885-9059-4b41-a06b-054a5d21705d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:02.858604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3c885-9059-4b41-a06b-054a5d21705d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:02.859102 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3c885-9059-4b41-a06b-054a5d21705d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:02.859572 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3c885-9059-4b41-a06b-054a5d21705d None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:02.860594 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3c885-9059-4b41-a06b-054a5d21705d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:02.868266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3c885-9059-4b41-a06b-054a5d21705d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:02.868266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3c885-9059-4b41-a06b-054a5d21705d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:02.878067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 305/607] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:02.880311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d8263-0d2c-455a-ab95-53d8866a4d73 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:02.880831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d8263-0d2c-455a-ab95-53d8866a4d73 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:02.881305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d8263-0d2c-455a-ab95-53d8866a4d73 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:02.883504 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 306/608] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:02.884901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d8263-0d2c-455a-ab95-53d8866a4d73 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:02.966471 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42b750-42ff-4907-8338-4dac666b478f None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:02.969938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:02.993816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d8263-0d2c-455a-ab95-53d8866a4d73 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:03.002989 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def20296-9b4d-477f-8523-24c95c6c6f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:03.003872 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def20296-9b4d-477f-8523-24c95c6c6f1e None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:03.004235 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def20296-9b4d-477f-8523-24c95c6c6f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:03.004543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def20296-9b4d-477f-8523-24c95c6c6f1e None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/6d5a01edfe0444f8a6fef9d7ee9645f3/roles/a615c7d83a0c46b1b26642bcc6e6ecef` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:03.021428 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def20296-9b4d-477f-8523-24c95c6c6f1e None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=6d5a01edfe0444f8a6fef9d7ee9645f3, role_id=a615c7d83a0c46b1b26642bcc6e6ecef)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:03.022716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def20296-9b4d-477f-8523-24c95c6c6f1e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:03.029749 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a204681-89e3-4188-986c-013883a8ca3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:03.030538 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a204681-89e3-4188-986c-013883a8ca3e None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:03.030821 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a204681-89e3-4188-986c-013883a8ca3e None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:03.031095 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a204681-89e3-4188-986c-013883a8ca3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:03.041463 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a204681-89e3-4188-986c-013883a8ca3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-6a204681-89e3-4188-986c-013883a8ca3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:03.049453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:03.050516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 307/609] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:47:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:03.051636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a204681-89e3-4188-986c-013883a8ca3e None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:03.101029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:03.142464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 303/610] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:47:02 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/6d5a01edfe0444f8a6fef9d7ee9645f3/roles/a615c7d83a0c46b1b26642bcc6e6ecef => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:47:03.151657 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a204681-89e3-4188-986c-013883a8ca3e None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:03.153974 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc15dde-d82a-4382-b9b9-b13dab49b8c7 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:03.154266 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc15dde-d82a-4382-b9b9-b13dab49b8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:03.154544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc15dde-d82a-4382-b9b9-b13dab49b8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:03.193631 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-2dc15dde-d82a-4382-b9b9-b13dab49b8c7 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:03.195311 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-2dc15dde-d82a-4382-b9b9-b13dab49b8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:03.198554 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcfc8b-97a7-43c5-9513-88deed9dbb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:03.200407 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcfc8b-97a7-43c5-9513-88deed9dbb29 None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:03.200701 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcfc8b-97a7-43c5-9513-88deed9dbb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:03.201538 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcfc8b-97a7-43c5-9513-88deed9dbb29 None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/86eeca6ef9374305b98ac8f8b03f2fc5/roles/a615c7d83a0c46b1b26642bcc6e6ecef` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:03.222617 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dcfc8b-97a7-43c5-9513-88deed9dbb29 None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=86eeca6ef9374305b98ac8f8b03f2fc5, role_id=a615c7d83a0c46b1b26642bcc6e6ecef)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:03.224007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dcfc8b-97a7-43c5-9513-88deed9dbb29 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:03.276828 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 304/611] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:47:03.277772 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc15dde-d82a-4382-b9b9-b13dab49b8c7 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:03.304577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 308/612] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:47:03 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/86eeca6ef9374305b98ac8f8b03f2fc5/roles/a615c7d83a0c46b1b26642bcc6e6ecef => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:47:03.323248 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcfc8b-97a7-43c5-9513-88deed9dbb29 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:03.417829 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:03.438055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc15dde-d82a-4382-b9b9-b13dab49b8c7 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:03.477178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:03.485342 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dcfc8b-97a7-43c5-9513-88deed9dbb29 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:03.498373 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e13b3f-34c0-4f8e-9749-82da45ec20e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:03.499052 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e13b3f-34c0-4f8e-9749-82da45ec20e5 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:03.500249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e13b3f-34c0-4f8e-9749-82da45ec20e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:03.500249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e13b3f-34c0-4f8e-9749-82da45ec20e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:03.500457 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e13b3f-34c0-4f8e-9749-82da45ec20e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:03.500772 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e13b3f-34c0-4f8e-9749-82da45ec20e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:03.502896 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e13b3f-34c0-4f8e-9749-82da45ec20e5 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:03.520078 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 305/613] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:47:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:03.528506 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e13b3f-34c0-4f8e-9749-82da45ec20e5 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:03.565091 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9230d-7513-4dcd-9152-b49beeb8b775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:03.565428 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9230d-7513-4dcd-9152-b49beeb8b775 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:03.565716 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9230d-7513-4dcd-9152-b49beeb8b775 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:03.565996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9230d-7513-4dcd-9152-b49beeb8b775 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:03.580861 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9230d-7513-4dcd-9152-b49beeb8b775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-38b9230d-7513-4dcd-9152-b49beeb8b775 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:03.590985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:03.593220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:03.595007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 309/614] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:47:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:03.600099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9230d-7513-4dcd-9152-b49beeb8b775 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:03.662924 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e13b3f-34c0-4f8e-9749-82da45ec20e5 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:03.701567 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba393e-7781-4de0-83ed-684397c39bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:03.702423 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba393e-7781-4de0-83ed-684397c39bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:03.702703 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba393e-7781-4de0-83ed-684397c39bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:03.703158 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba393e-7781-4de0-83ed-684397c39bf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:03.715354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba393e-7781-4de0-83ed-684397c39bf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:03.717399 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba393e-7781-4de0-83ed-684397c39bf8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:03.724724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-32ba393e-7781-4de0-83ed-684397c39bf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:03.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:03.745971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:03.748114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:03.748114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:03.748114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:03.748114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:03.748114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 306/615] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:03.748779 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9230d-7513-4dcd-9152-b49beeb8b775 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:03.748934 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba393e-7781-4de0-83ed-684397c39bf8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:03.788425 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d03be-ac43-4e84-a205-2310db94b563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:03.789103 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d03be-ac43-4e84-a205-2310db94b563 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:03.789388 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d03be-ac43-4e84-a205-2310db94b563 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:03.789666 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241d03be-ac43-4e84-a205-2310db94b563 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:03.790421 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d03be-ac43-4e84-a205-2310db94b563 None admin] RBAC: Adding query filter params (name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:03.790728 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d03be-ac43-4e84-a205-2310db94b563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:03.792107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241d03be-ac43-4e84-a205-2310db94b563 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:03.818872 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 310/616] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:47:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:03.824169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d03be-ac43-4e84-a205-2310db94b563 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:03.864935 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba393e-7781-4de0-83ed-684397c39bf8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:03.876006 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:03.909719 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b048c7c-5708-4c80-8a57-cb2cafb375e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:03.910391 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b048c7c-5708-4c80-8a57-cb2cafb375e8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:03.910731 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b048c7c-5708-4c80-8a57-cb2cafb375e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:03.911006 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b048c7c-5708-4c80-8a57-cb2cafb375e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:03.911746 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b048c7c-5708-4c80-8a57-cb2cafb375e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:03.912094 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b048c7c-5708-4c80-8a57-cb2cafb375e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:03.913697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b048c7c-5708-4c80-8a57-cb2cafb375e8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:03.930917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 307/617] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:03.936870 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b048c7c-5708-4c80-8a57-cb2cafb375e8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:04.009723 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:04.027442 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241d03be-ac43-4e84-a205-2310db94b563 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:04.066612 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e85c32-6202-4d7a-8245-7d54a9d693e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:04.067414 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e85c32-6202-4d7a-8245-7d54a9d693e4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:04.067698 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e85c32-6202-4d7a-8245-7d54a9d693e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:04.067970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e85c32-6202-4d7a-8245-7d54a9d693e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:04.085744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b048c7c-5708-4c80-8a57-cb2cafb375e8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:04.087211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e85c32-6202-4d7a-8245-7d54a9d693e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:04.088913 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e85c32-6202-4d7a-8245-7d54a9d693e4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-e3e85c32-6202-4d7a-8245-7d54a9d693e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:04.118584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:04.119558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:04.120459 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:04.120459 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:04.120459 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:04.120459 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:04.122781 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 311/618] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:04.133358 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e85c32-6202-4d7a-8245-7d54a9d693e4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:04.150284 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74089975-1008-428a-aa60-aa4805360c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:04.151120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74089975-1008-428a-aa60-aa4805360c65 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:04.151353 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74089975-1008-428a-aa60-aa4805360c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:04.151625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74089975-1008-428a-aa60-aa4805360c65 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:04.164499 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74089975-1008-428a-aa60-aa4805360c65 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:04.164499 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74089975-1008-428a-aa60-aa4805360c65 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-74089975-1008-428a-aa60-aa4805360c65 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:04.183660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:04.184742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:04.185738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:04.185738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:04.185738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:04.185738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:04.185738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:04.185738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:04.185738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:04.185738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:04.185738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:47:04.185738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:04.188528 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 308/619] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:04.197667 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74089975-1008-428a-aa60-aa4805360c65 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:04.237137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:04.269980 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e85c32-6202-4d7a-8245-7d54a9d693e4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:04.309025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af66851-55de-4932-b289-2e90eb016580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:04.309694 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66851-55de-4932-b289-2e90eb016580 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:04.310003 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66851-55de-4932-b289-2e90eb016580 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:04.310288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af66851-55de-4932-b289-2e90eb016580 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:04.311025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af66851-55de-4932-b289-2e90eb016580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:04.311331 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af66851-55de-4932-b289-2e90eb016580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:04.312870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af66851-55de-4932-b289-2e90eb016580 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:04.326452 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 312/620] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:04.331797 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af66851-55de-4932-b289-2e90eb016580 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:04.407975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74089975-1008-428a-aa60-aa4805360c65 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:04.425533 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:04.443406 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50d545-edfc-43f3-800d-d161b466b7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:04.444301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50d545-edfc-43f3-800d-d161b466b7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:04.444759 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50d545-edfc-43f3-800d-d161b466b7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:04.445675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50d545-edfc-43f3-800d-d161b466b7f7 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:04.446289 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50d545-edfc-43f3-800d-d161b466b7f7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:04.446753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50d545-edfc-43f3-800d-d161b466b7f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:04.448355 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50d545-edfc-43f3-800d-d161b466b7f7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:04.451816 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af66851-55de-4932-b289-2e90eb016580 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:04.467162 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 309/621] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 19:47:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:04.468846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0919f3-8346-4411-8862-c30249c52ae3 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:04.469358 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0919f3-8346-4411-8862-c30249c52ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:04.469828 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0919f3-8346-4411-8862-c30249c52ae3 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:04.475312 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 310/622] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:04.476402 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0919f3-8346-4411-8862-c30249c52ae3 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:04.493662 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:04.500692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a1ec0-b47b-4a16-89a1-97db3ff29c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:04.501745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a1ec0-b47b-4a16-89a1-97db3ff29c63 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:04.502277 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a1ec0-b47b-4a16-89a1-97db3ff29c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:04.502853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a1ec0-b47b-4a16-89a1-97db3ff29c63 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:04.512087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a1ec0-b47b-4a16-89a1-97db3ff29c63 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:04.515113 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a1ec0-b47b-4a16-89a1-97db3ff29c63 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-ff0a1ec0-b47b-4a16-89a1-97db3ff29c63 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:04.532928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:04.533952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:04.534928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:04.534928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:04.534928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:04.534928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:04.534928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:04.534928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:04.534928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:04.534928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:04.534928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:47:04.534928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:04.542112 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 313/623] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 19:47:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:04.543804 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48998386-7e73-4fc7-a69e-65de7c8e8c03 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:04.544292 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48998386-7e73-4fc7-a69e-65de7c8e8c03 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:04.544896 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48998386-7e73-4fc7-a69e-65de7c8e8c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:04.564528 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-48998386-7e73-4fc7-a69e-65de7c8e8c03 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:04.566113 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-48998386-7e73-4fc7-a69e-65de7c8e8c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:04.585918 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0919f3-8346-4411-8862-c30249c52ae3 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:04.621043 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd1d09-9ec7-4158-9c3e-e12dc5ede90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:04.621858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1d09-9ec7-4158-9c3e-e12dc5ede90c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:04.622137 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1d09-9ec7-4158-9c3e-e12dc5ede90c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:04.622415 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd1d09-9ec7-4158-9c3e-e12dc5ede90c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:04.633800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd1d09-9ec7-4158-9c3e-e12dc5ede90c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:04.635265 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd1d09-9ec7-4158-9c3e-e12dc5ede90c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:04.645505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 314/624] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:47:04.646888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48998386-7e73-4fc7-a69e-65de7c8e8c03 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-1ecd1d09-9ec7-4158-9c3e-e12dc5ede90c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:04.658148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:04.659095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:04.659980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:04.659980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:04.659980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:04.659980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:04.659980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 311/625] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:04.665396 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd1d09-9ec7-4158-9c3e-e12dc5ede90c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:04.707134 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:04.766473 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48998386-7e73-4fc7-a69e-65de7c8e8c03 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:04.791030 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:04.804376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21522e8-8d85-404c-bec5-f7babe5e8b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:04.805390 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21522e8-8d85-404c-bec5-f7babe5e8b30 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:04.805899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21522e8-8d85-404c-bec5-f7babe5e8b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:04.806382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21522e8-8d85-404c-bec5-f7babe5e8b30 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:04.807428 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21522e8-8d85-404c-bec5-f7babe5e8b30 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:04.807952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21522e8-8d85-404c-bec5-f7babe5e8b30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:04.809039 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd1d09-9ec7-4158-9c3e-e12dc5ede90c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:04.809905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21522e8-8d85-404c-bec5-f7babe5e8b30 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:04.838482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 315/626] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 19:47:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:04.838482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21522e8-8d85-404c-bec5-f7babe5e8b30 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:04.852001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb3b25-1215-438d-8f74-82fd04b2c7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:04.853139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb3b25-1215-438d-8f74-82fd04b2c7fd None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:04.853628 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb3b25-1215-438d-8f74-82fd04b2c7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:04.854102 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb3b25-1215-438d-8f74-82fd04b2c7fd None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:04.855036 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb3b25-1215-438d-8f74-82fd04b2c7fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:04.855618 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb3b25-1215-438d-8f74-82fd04b2c7fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:04.857849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb3b25-1215-438d-8f74-82fd04b2c7fd None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:04.871507 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 312/627] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:04.873297 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb3b25-1215-438d-8f74-82fd04b2c7fd None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:04.971730 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:04.974029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21522e8-8d85-404c-bec5-f7babe5e8b30 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:04.986504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb3b25-1215-438d-8f74-82fd04b2c7fd None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:05.017793 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb709415-eaa9-4459-922d-d7f9851787cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:05.018742 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb709415-eaa9-4459-922d-d7f9851787cb None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.019028 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb709415-eaa9-4459-922d-d7f9851787cb None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.019304 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb709415-eaa9-4459-922d-d7f9851787cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:05.028646 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8872e-aebe-4a47-b860-2aeec9419a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:05.029717 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8872e-aebe-4a47-b860-2aeec9419a7c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.030228 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8872e-aebe-4a47-b860-2aeec9419a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.030706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8872e-aebe-4a47-b860-2aeec9419a7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:05.031417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb709415-eaa9-4459-922d-d7f9851787cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-fb709415-eaa9-4459-922d-d7f9851787cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:05.040733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:05.041732 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:05.042660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 316/628] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:47:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:05.043359 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb709415-eaa9-4459-922d-d7f9851787cb None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:05.097656 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:05.111664 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8872e-aebe-4a47-b860-2aeec9419a7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:05.113667 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8872e-aebe-4a47-b860-2aeec9419a7c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:05.167046 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb709415-eaa9-4459-922d-d7f9851787cb None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:05.202370 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858238b2-a2be-4bb1-b7d8-7318cef0bc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:05.203171 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858238b2-a2be-4bb1-b7d8-7318cef0bc83 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.203453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858238b2-a2be-4bb1-b7d8-7318cef0bc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.203728 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858238b2-a2be-4bb1-b7d8-7318cef0bc83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:05.211374 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858238b2-a2be-4bb1-b7d8-7318cef0bc83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:05.213043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858238b2-a2be-4bb1-b7d8-7318cef0bc83 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:05.220207 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-858238b2-a2be-4bb1-b7d8-7318cef0bc83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:05.227908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:05.228915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:05.229817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:05.229817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:05.229817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:05.229817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:05.229817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:05.229817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:05.229817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:05.229817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:05.234543 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 317/629] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:05.237112 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858238b2-a2be-4bb1-b7d8-7318cef0bc83 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:05.238615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 313/630] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:47:05.239740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8872e-aebe-4a47-b860-2aeec9419a7c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:05.335498 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:05.342820 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858238b2-a2be-4bb1-b7d8-7318cef0bc83 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:05.362747 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8872e-aebe-4a47-b860-2aeec9419a7c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:05.379058 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e319c1-4145-4683-bfc1-3aaaaea46449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:05.379728 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e319c1-4145-4683-bfc1-3aaaaea46449 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.380013 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e319c1-4145-4683-bfc1-3aaaaea46449 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.380292 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e319c1-4145-4683-bfc1-3aaaaea46449 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:05.381199 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e319c1-4145-4683-bfc1-3aaaaea46449 None admin] RBAC: Adding query filter params (name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:05.381381 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e319c1-4145-4683-bfc1-3aaaaea46449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:05.382735 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e319c1-4145-4683-bfc1-3aaaaea46449 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:05.391582 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 318/631] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:47:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:05.392356 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e319c1-4145-4683-bfc1-3aaaaea46449 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:05.405578 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78b738-76b0-4428-bb6f-bd9d35dc2615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:05.406602 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78b738-76b0-4428-bb6f-bd9d35dc2615 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.407113 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78b738-76b0-4428-bb6f-bd9d35dc2615 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.407592 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78b738-76b0-4428-bb6f-bd9d35dc2615 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:05.408796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78b738-76b0-4428-bb6f-bd9d35dc2615 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:05.409771 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78b738-76b0-4428-bb6f-bd9d35dc2615 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:05.411557 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78b738-76b0-4428-bb6f-bd9d35dc2615 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:05.426404 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 314/632] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:05.427525 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78b738-76b0-4428-bb6f-bd9d35dc2615 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:05.486703 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:05.504522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e319c1-4145-4683-bfc1-3aaaaea46449 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:05.541028 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655579a-868a-424e-a485-747206c92863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:05.541819 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655579a-868a-424e-a485-747206c92863 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.542106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655579a-868a-424e-a485-747206c92863 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.542375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655579a-868a-424e-a485-747206c92863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:05.550531 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78b738-76b0-4428-bb6f-bd9d35dc2615 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:05.553688 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655579a-868a-424e-a485-747206c92863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:05.555163 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655579a-868a-424e-a485-747206c92863 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-6655579a-868a-424e-a485-747206c92863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:05.573015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:05.574012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:05.574892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:05.574892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:05.574892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:05.574892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:05.574892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 319/633] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:05.576135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655579a-868a-424e-a485-747206c92863 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:05.594182 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da36bc8-746a-44f3-b9fd-ba4d49d7b08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:05.595870 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36bc8-746a-44f3-b9fd-ba4d49d7b08a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.596422 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36bc8-746a-44f3-b9fd-ba4d49d7b08a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.596903 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36bc8-746a-44f3-b9fd-ba4d49d7b08a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:05.602645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:05.610543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da36bc8-746a-44f3-b9fd-ba4d49d7b08a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:05.614049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da36bc8-746a-44f3-b9fd-ba4d49d7b08a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-7da36bc8-746a-44f3-b9fd-ba4d49d7b08a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:05.631724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:05.632797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:05.633681 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:05.633681 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:05.633681 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:05.633681 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:05.636253 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 315/634] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:05.637310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da36bc8-746a-44f3-b9fd-ba4d49d7b08a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:05.702275 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655579a-868a-424e-a485-747206c92863 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:05.742335 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e392c6a-1e40-4377-a9fc-dfc7c82d5492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:05.743553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e392c6a-1e40-4377-a9fc-dfc7c82d5492 None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.743840 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e392c6a-1e40-4377-a9fc-dfc7c82d5492 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.744118 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e392c6a-1e40-4377-a9fc-dfc7c82d5492 None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/c8a36bb1e26f4faaa44fabff8b509fe7/roles/a615c7d83a0c46b1b26642bcc6e6ecef` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:05.746326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:05.749828 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da36bc8-746a-44f3-b9fd-ba4d49d7b08a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:05.759626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e392c6a-1e40-4377-a9fc-dfc7c82d5492 None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=c8a36bb1e26f4faaa44fabff8b509fe7, role_id=a615c7d83a0c46b1b26642bcc6e6ecef)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:05.772809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e392c6a-1e40-4377-a9fc-dfc7c82d5492 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:05.784645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29504ce1-b7cd-4eb5-ab63-89042841ea0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:05.785313 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29504ce1-b7cd-4eb5-ab63-89042841ea0a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.785595 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29504ce1-b7cd-4eb5-ab63-89042841ea0a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.785870 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29504ce1-b7cd-4eb5-ab63-89042841ea0a None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:05.786747 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29504ce1-b7cd-4eb5-ab63-89042841ea0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:05.787049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29504ce1-b7cd-4eb5-ab63-89042841ea0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:05.788452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29504ce1-b7cd-4eb5-ab63-89042841ea0a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:05.801526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 316/635] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:05.802117 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29504ce1-b7cd-4eb5-ab63-89042841ea0a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:05.850607 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 320/636] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:47:05 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/c8a36bb1e26f4faaa44fabff8b509fe7/roles/a615c7d83a0c46b1b26642bcc6e6ecef => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:47:05.852348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e57431-4ae6-4a03-a65b-ebbce3bca0ad None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.852971 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e57431-4ae6-4a03-a65b-ebbce3bca0ad None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.853910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e57431-4ae6-4a03-a65b-ebbce3bca0ad None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:05.856148 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 321/637] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:05.857802 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e57431-4ae6-4a03-a65b-ebbce3bca0ad None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:05.931402 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29504ce1-b7cd-4eb5-ab63-89042841ea0a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:05.971902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a411ee35-f8be-4c2d-8efd-e222e5514776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:05.972618 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411ee35-f8be-4c2d-8efd-e222e5514776 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.973435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411ee35-f8be-4c2d-8efd-e222e5514776 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.973435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a411ee35-f8be-4c2d-8efd-e222e5514776 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:05.974005 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411ee35-f8be-4c2d-8efd-e222e5514776 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:05.974318 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411ee35-f8be-4c2d-8efd-e222e5514776 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:05.975749 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a411ee35-f8be-4c2d-8efd-e222e5514776 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:05.976770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:05.990746 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 317/638] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:05.992473 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cd0a0-a95b-45bc-a284-632774cdd9b4 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:05.993173 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cd0a0-a95b-45bc-a284-632774cdd9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:05.993173 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cd0a0-a95b-45bc-a284-632774cdd9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.010174 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e57431-4ae6-4a03-a65b-ebbce3bca0ad None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:06.016799 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-190cd0a0-a95b-45bc-a284-632774cdd9b4 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:06.018028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-190cd0a0-a95b-45bc-a284-632774cdd9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:06.040134 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:06.061079 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d51a3-0928-4705-b69d-dac52eb082a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:06.062530 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d51a3-0928-4705-b69d-dac52eb082a6 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:06.063054 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d51a3-0928-4705-b69d-dac52eb082a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:06.063556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d51a3-0928-4705-b69d-dac52eb082a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.072894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d51a3-0928-4705-b69d-dac52eb082a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:06.074552 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d51a3-0928-4705-b69d-dac52eb082a6 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-a73d51a3-0928-4705-b69d-dac52eb082a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:06.093099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:06.094043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:06.094902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:06.094902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:06.094902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:06.094902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:06.094902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:06.094902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:06.094902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:06.094902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:06.094902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 322/639] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:06.094902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d51a3-0928-4705-b69d-dac52eb082a6 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:06.116530 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 318/640] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:47:06.120356 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebc458-61b2-47cc-8c25-5d0701365717 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:06.120356 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebc458-61b2-47cc-8c25-5d0701365717 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:06.120356 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebc458-61b2-47cc-8c25-5d0701365717 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.120356 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 319/641] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:06.120356 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebc458-61b2-47cc-8c25-5d0701365717 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:06.215159 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d51a3-0928-4705-b69d-dac52eb082a6 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:06.231361 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebc458-61b2-47cc-8c25-5d0701365717 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:06.239120 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:06.250127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80459ef-c649-44dc-a061-735b0ea02984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:06.254528 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80459ef-c649-44dc-a061-735b0ea02984 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:06.254528 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80459ef-c649-44dc-a061-735b0ea02984 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:06.254528 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80459ef-c649-44dc-a061-735b0ea02984 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.259810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80459ef-c649-44dc-a061-735b0ea02984 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:06.261334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80459ef-c649-44dc-a061-735b0ea02984 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:06.272449 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52734157-640a-4fc6-a456-fc65dabbf34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:06.273378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52734157-640a-4fc6-a456-fc65dabbf34f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:06.273846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52734157-640a-4fc6-a456-fc65dabbf34f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:06.274290 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52734157-640a-4fc6-a456-fc65dabbf34f None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.275246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52734157-640a-4fc6-a456-fc65dabbf34f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:06.275741 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52734157-640a-4fc6-a456-fc65dabbf34f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-e80459ef-c649-44dc-a061-735b0ea02984 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:06.276955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:06.278156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:06.279905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:06.279905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:06.279905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:06.279905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:06.279905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:06.279905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:06.279905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:06.279905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:06.279905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 19:47:06.279905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:06.281155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52734157-640a-4fc6-a456-fc65dabbf34f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:06.284082 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 323/642] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:47:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:06.285611 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc89be0-a256-4ca0-9776-fb66e4c5af75 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:06.285899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc89be0-a256-4ca0-9776-fb66e4c5af75 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:06.286208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc89be0-a256-4ca0-9776-fb66e4c5af75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.294894 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 320/643] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:06.295980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52734157-640a-4fc6-a456-fc65dabbf34f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:06.306907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-0cc89be0-a256-4ca0-9776-fb66e4c5af75 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:06.308099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-0cc89be0-a256-4ca0-9776-fb66e4c5af75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:06.312728 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:06.390137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 324/644] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:47:06.390930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc89be0-a256-4ca0-9776-fb66e4c5af75 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:06.425739 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52734157-640a-4fc6-a456-fc65dabbf34f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:06.461325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389deb0e-0c55-45bc-a3c5-9f9dbfa29caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:06.462120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389deb0e-0c55-45bc-a3c5-9f9dbfa29caa None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:06.462403 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389deb0e-0c55-45bc-a3c5-9f9dbfa29caa None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:06.462682 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389deb0e-0c55-45bc-a3c5-9f9dbfa29caa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.476490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389deb0e-0c55-45bc-a3c5-9f9dbfa29caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-389deb0e-0c55-45bc-a3c5-9f9dbfa29caa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:06.486945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:06.487916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 321/645] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:47:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:06.489862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389deb0e-0c55-45bc-a3c5-9f9dbfa29caa None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:06.503390 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc89be0-a256-4ca0-9776-fb66e4c5af75 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:06.518739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:06.540867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aface0-499a-403b-95f4-04a33868bfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:06.541755 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aface0-499a-403b-95f4-04a33868bfaf None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:06.542136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aface0-499a-403b-95f4-04a33868bfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:06.542476 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aface0-499a-403b-95f4-04a33868bfaf None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.543383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aface0-499a-403b-95f4-04a33868bfaf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:06.543766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aface0-499a-403b-95f4-04a33868bfaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:06.545339 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aface0-499a-403b-95f4-04a33868bfaf None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:06.564267 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 325/646] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 19:47:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:06.565236 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aface0-499a-403b-95f4-04a33868bfaf None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:06.570443 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:06.602288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389deb0e-0c55-45bc-a3c5-9f9dbfa29caa None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:06.639316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2c62f-658c-40a8-97f7-096106f77269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:06.639973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2c62f-658c-40a8-97f7-096106f77269 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:06.640309 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2c62f-658c-40a8-97f7-096106f77269 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:06.640566 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e2c62f-658c-40a8-97f7-096106f77269 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.641410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2c62f-658c-40a8-97f7-096106f77269 None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=0c0e71329c4a4470935593636c025257) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:06.641719 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2c62f-658c-40a8-97f7-096106f77269 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:06.643190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e2c62f-658c-40a8-97f7-096106f77269 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:06.660384 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 322/647] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:06 2022] GET /identity/v3/role_assignments?user.id=0c0e71329c4a4470935593636c025257&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:06.660384 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2c62f-658c-40a8-97f7-096106f77269 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:06.683563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aface0-499a-403b-95f4-04a33868bfaf None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:06.730686 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c5519-2509-48b3-8382-e05f4ec6d4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:06.731640 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c5519-2509-48b3-8382-e05f4ec6d4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:06.732608 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c5519-2509-48b3-8382-e05f4ec6d4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:06.732884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c5519-2509-48b3-8382-e05f4ec6d4e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.745441 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c5519-2509-48b3-8382-e05f4ec6d4e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-877c5519-2509-48b3-8382-e05f4ec6d4e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:06.754850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:06.756211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 326/648] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:47:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:06.757284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c5519-2509-48b3-8382-e05f4ec6d4e4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:06.794304 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:06.795351 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e2c62f-658c-40a8-97f7-096106f77269 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:06.835444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28afca-82e2-48ec-a219-94ee07b6df14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:06.836697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28afca-82e2-48ec-a219-94ee07b6df14 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:06.837192 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28afca-82e2-48ec-a219-94ee07b6df14 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:06.837667 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28afca-82e2-48ec-a219-94ee07b6df14 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.838661 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28afca-82e2-48ec-a219-94ee07b6df14 None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:06.839186 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28afca-82e2-48ec-a219-94ee07b6df14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:06.841042 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28afca-82e2-48ec-a219-94ee07b6df14 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:06.843530 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:06.852909 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 323/649] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:47:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:06.854195 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28afca-82e2-48ec-a219-94ee07b6df14 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:06.875442 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c5519-2509-48b3-8382-e05f4ec6d4e4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:06.927588 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc7431-cd06-4acd-a29b-f419f3ba437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:06.928172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc7431-cd06-4acd-a29b-f419f3ba437d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:06.928488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc7431-cd06-4acd-a29b-f419f3ba437d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:06.928774 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc7431-cd06-4acd-a29b-f419f3ba437d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:06.991628 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28afca-82e2-48ec-a219-94ee07b6df14 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:07.010354 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc7431-cd06-4acd-a29b-f419f3ba437d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:07.012106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafc7431-cd06-4acd-a29b-f419f3ba437d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:07.037014 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e06d-8892-40db-a956-5444f7b2a869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:07.037960 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e06d-8892-40db-a956-5444f7b2a869 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:07.038748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e06d-8892-40db-a956-5444f7b2a869 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:07.039184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127e06d-8892-40db-a956-5444f7b2a869 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:07.040752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127e06d-8892-40db-a956-5444f7b2a869 None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:07.041514 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127e06d-8892-40db-a956-5444f7b2a869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:07.044044 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127e06d-8892-40db-a956-5444f7b2a869 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:07.047870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:07.057552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 324/650] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:47:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:07.058634 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e06d-8892-40db-a956-5444f7b2a869 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:07.132537 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 327/651] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:47:07.133926 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc7431-cd06-4acd-a29b-f419f3ba437d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:07.142499 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:07.205642 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127e06d-8892-40db-a956-5444f7b2a869 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:07.250086 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8aee6-45b0-4b98-93e5-fc2f1bdf9b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:07.251122 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8aee6-45b0-4b98-93e5-fc2f1bdf9b82 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:07.251546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8aee6-45b0-4b98-93e5-fc2f1bdf9b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:07.251761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc7431-cd06-4acd-a29b-f419f3ba437d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:07.252149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8aee6-45b0-4b98-93e5-fc2f1bdf9b82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:07.266441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8aee6-45b0-4b98-93e5-fc2f1bdf9b82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:07.268367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f8aee6-45b0-4b98-93e5-fc2f1bdf9b82 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:07.287865 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-c0f8aee6-45b0-4b98-93e5-fc2f1bdf9b82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:07.292801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:07.293669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:07.294683 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:07.294683 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:07.294683 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:07.294683 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:07.294905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e1fa7-8af7-44af-9c6b-842b52b1cd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:07.294905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e1fa7-8af7-44af-9c6b-842b52b1cd42 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:07.294905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e1fa7-8af7-44af-9c6b-842b52b1cd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:07.294905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e1fa7-8af7-44af-9c6b-842b52b1cd42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:07.297286 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 325/652] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:07.298252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8aee6-45b0-4b98-93e5-fc2f1bdf9b82 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:07.313801 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e1fa7-8af7-44af-9c6b-842b52b1cd42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:07.315635 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e1fa7-8af7-44af-9c6b-842b52b1cd42 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-615e1fa7-8af7-44af-9c6b-842b52b1cd42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:07.343485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:07.344353 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:07.345245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:07.345245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:07.345245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:07.345245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:07.349208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 328/653] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:07.350064 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e1fa7-8af7-44af-9c6b-842b52b1cd42 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:07.422073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f8aee6-45b0-4b98-93e5-fc2f1bdf9b82 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:07.433035 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:07.462212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368a618-b7cb-409f-997e-7eb04e6e7806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:07.463365 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368a618-b7cb-409f-997e-7eb04e6e7806 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:07.463680 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368a618-b7cb-409f-997e-7eb04e6e7806 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:07.464380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4368a618-b7cb-409f-997e-7eb04e6e7806 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:07.472099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e1fa7-8af7-44af-9c6b-842b52b1cd42 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:07.476078 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368a618-b7cb-409f-997e-7eb04e6e7806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:07.477700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4368a618-b7cb-409f-997e-7eb04e6e7806 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-4368a618-b7cb-409f-997e-7eb04e6e7806 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:07.495654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:07.496675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:07.498181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:07.498181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:07.498181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:07.498181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:07.502065 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 326/654] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:07.502942 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368a618-b7cb-409f-997e-7eb04e6e7806 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:07.513730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ce414-b3f6-44b3-bea5-21cf05dbe825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:07.514745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ce414-b3f6-44b3-bea5-21cf05dbe825 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:07.515226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ce414-b3f6-44b3-bea5-21cf05dbe825 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:07.515748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ce414-b3f6-44b3-bea5-21cf05dbe825 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:07.516697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ce414-b3f6-44b3-bea5-21cf05dbe825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:07.517302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ce414-b3f6-44b3-bea5-21cf05dbe825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:07.518877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ce414-b3f6-44b3-bea5-21cf05dbe825 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:07.533415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 329/655] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:07.534701 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ce414-b3f6-44b3-bea5-21cf05dbe825 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:07.585978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:07.613355 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4368a618-b7cb-409f-997e-7eb04e6e7806 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:07.647294 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ce414-b3f6-44b3-bea5-21cf05dbe825 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:07.649454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd665e-c965-4347-be9f-2e0b341bce80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:07.650123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd665e-c965-4347-be9f-2e0b341bce80 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:07.650406 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd665e-c965-4347-be9f-2e0b341bce80 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:07.650678 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd665e-c965-4347-be9f-2e0b341bce80 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:07.651809 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd665e-c965-4347-be9f-2e0b341bce80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:07.651809 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd665e-c965-4347-be9f-2e0b341bce80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:07.653474 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd665e-c965-4347-be9f-2e0b341bce80 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:07.670484 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 327/656] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:07.671824 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd665e-c965-4347-be9f-2e0b341bce80 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:07.687507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f8a75-3b94-41d5-b177-92354a94d504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:07.688531 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f8a75-3b94-41d5-b177-92354a94d504 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:07.689017 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f8a75-3b94-41d5-b177-92354a94d504 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:07.689437 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f8a75-3b94-41d5-b177-92354a94d504 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:07.690341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f8a75-3b94-41d5-b177-92354a94d504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:07.690852 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f8a75-3b94-41d5-b177-92354a94d504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:07.692620 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f8a75-3b94-41d5-b177-92354a94d504 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:07.698148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:07.707564 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 330/657] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:07.708647 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f8a75-3b94-41d5-b177-92354a94d504 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:07.783103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd665e-c965-4347-be9f-2e0b341bce80 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:07.820255 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f8a75-3b94-41d5-b177-92354a94d504 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:07.823520 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:07.862568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44d3e7-6c97-4efa-a673-9ef849e51397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:07.863683 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44d3e7-6c97-4efa-a673-9ef849e51397 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:07.864175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44d3e7-6c97-4efa-a673-9ef849e51397 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:07.864666 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44d3e7-6c97-4efa-a673-9ef849e51397 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:07.874239 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44d3e7-6c97-4efa-a673-9ef849e51397 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:07.876022 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44d3e7-6c97-4efa-a673-9ef849e51397 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-fc44d3e7-6c97-4efa-a673-9ef849e51397 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:07.892291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:07.893344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:07.894288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:07.894288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:07.894288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:07.894288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:07.894288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:07.894288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:07.894288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:07.894288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:07.894288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:47:07.894288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:07.897165 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 331/658] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:47:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:07.898215 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44d3e7-6c97-4efa-a673-9ef849e51397 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:07.966641 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cb6c1-c317-4382-95b2-bdcd2c885107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:07.967835 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cb6c1-c317-4382-95b2-bdcd2c885107 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:07.968125 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cb6c1-c317-4382-95b2-bdcd2c885107 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:07.968423 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224cb6c1-c317-4382-95b2-bdcd2c885107 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:07.979855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224cb6c1-c317-4382-95b2-bdcd2c885107 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:07.982286 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224cb6c1-c317-4382-95b2-bdcd2c885107 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-224cb6c1-c317-4382-95b2-bdcd2c885107 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:08.003202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:08.004261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:08.005463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:08.005463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:08.005463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:08.005463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:08.005463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:08.005463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:08.005463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:08.005463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:08.005463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 19:47:08.005463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:08.005463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 328/659] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:47:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:08.006069 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cb6c1-c317-4382-95b2-bdcd2c885107 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:08.028380 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44d3e7-6c97-4efa-a673-9ef849e51397 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:08.073907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f84639-487c-46c2-9840-3d8a134c183f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:08.075183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f84639-487c-46c2-9840-3d8a134c183f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:08.075876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f84639-487c-46c2-9840-3d8a134c183f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:08.076305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f84639-487c-46c2-9840-3d8a134c183f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:08.086259 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f84639-487c-46c2-9840-3d8a134c183f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:08.087834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f84639-487c-46c2-9840-3d8a134c183f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-b9f84639-487c-46c2-9840-3d8a134c183f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:08.102886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:08.103826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:08.104770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:08.104770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:08.104770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:08.104770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:08.104770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:08.104770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:08.104770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:08.104770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:08.104770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 332/660] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:08.105863 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f84639-487c-46c2-9840-3d8a134c183f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:08.121384 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:08.134089 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:08.141867 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224cb6c1-c317-4382-95b2-bdcd2c885107 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:08.179973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2e03d-7348-4425-9dfc-ea30f8548553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:08.181029 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2e03d-7348-4425-9dfc-ea30f8548553 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:08.181566 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2e03d-7348-4425-9dfc-ea30f8548553 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:08.182044 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2e03d-7348-4425-9dfc-ea30f8548553 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:08.183197 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2e03d-7348-4425-9dfc-ea30f8548553 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:08.183712 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2e03d-7348-4425-9dfc-ea30f8548553 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:08.185423 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2e03d-7348-4425-9dfc-ea30f8548553 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:08.203767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 329/661] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 19:47:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:08.204986 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2e03d-7348-4425-9dfc-ea30f8548553 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:08.215738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f84639-487c-46c2-9840-3d8a134c183f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:08.252421 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86079f01-5128-4654-b487-b0493ff031d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:08.253208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86079f01-5128-4654-b487-b0493ff031d8 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:08.253488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86079f01-5128-4654-b487-b0493ff031d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:08.253763 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86079f01-5128-4654-b487-b0493ff031d8 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:08.254666 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86079f01-5128-4654-b487-b0493ff031d8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:08.254975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86079f01-5128-4654-b487-b0493ff031d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:08.256417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86079f01-5128-4654-b487-b0493ff031d8 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:08.273239 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 333/662] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 19:47:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:08.274309 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc19dd-0404-4ef2-87b2-4327087ec08d None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:08.274596 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc19dd-0404-4ef2-87b2-4327087ec08d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:08.274869 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc19dd-0404-4ef2-87b2-4327087ec08d None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:08.280511 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 334/663] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:08.281715 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc19dd-0404-4ef2-87b2-4327087ec08d None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:08.324274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2e03d-7348-4425-9dfc-ea30f8548553 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:08.363202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba385a-2b21-4b30-a5a4-6b5638491193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:08.363864 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba385a-2b21-4b30-a5a4-6b5638491193 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:08.364153 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba385a-2b21-4b30-a5a4-6b5638491193 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:08.364461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba385a-2b21-4b30-a5a4-6b5638491193 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:08.365257 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba385a-2b21-4b30-a5a4-6b5638491193 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:08.365567 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba385a-2b21-4b30-a5a4-6b5638491193 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:08.367024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ba385a-2b21-4b30-a5a4-6b5638491193 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:08.376443 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:08.384534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 330/664] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:08.384534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba385a-2b21-4b30-a5a4-6b5638491193 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:08.386903 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:08.406086 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc19dd-0404-4ef2-87b2-4327087ec08d None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:08.441055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af14f0-025c-454c-9f0e-a397f230ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:08.442015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af14f0-025c-454c-9f0e-a397f230ecb2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:08.442433 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af14f0-025c-454c-9f0e-a397f230ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:08.442878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af14f0-025c-454c-9f0e-a397f230ecb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:08.486292 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba385a-2b21-4b30-a5a4-6b5638491193 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:08.518486 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af14f0-025c-454c-9f0e-a397f230ecb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:08.519960 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af14f0-025c-454c-9f0e-a397f230ecb2 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:08.525580 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aa9a9-b749-4f9a-817a-6aea5f8320b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:08.526637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aa9a9-b749-4f9a-817a-6aea5f8320b4 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:08.526637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aa9a9-b749-4f9a-817a-6aea5f8320b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:08.526935 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aa9a9-b749-4f9a-817a-6aea5f8320b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:08.602208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:08.604851 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017aa9a9-b749-4f9a-817a-6aea5f8320b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:08.606046 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017aa9a9-b749-4f9a-817a-6aea5f8320b4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:08.612756 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:08.636253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 335/665] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:47:08.636253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae019ac-f54c-4368-98e3-ccc7dcfd172e None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:08.636253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae019ac-f54c-4368-98e3-ccc7dcfd172e None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:08.636253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae019ac-f54c-4368-98e3-ccc7dcfd172e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:08.651711 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-bae019ac-f54c-4368-98e3-ccc7dcfd172e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:08.653052 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-bae019ac-f54c-4368-98e3-ccc7dcfd172e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:08.721058 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 336/666] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:47:08.721904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae019ac-f54c-4368-98e3-ccc7dcfd172e None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:08.740536 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 331/667] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:47:08.741941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aa9a9-b749-4f9a-817a-6aea5f8320b4 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:08.844253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae019ac-f54c-4368-98e3-ccc7dcfd172e None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:08.852304 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:08.856214 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aa9a9-b749-4f9a-817a-6aea5f8320b4 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:08.857996 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:08.879710 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3e58d-6087-46b7-afc1-d00b93fb4e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:08.880707 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3e58d-6087-46b7-afc1-d00b93fb4e79 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:08.881178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3e58d-6087-46b7-afc1-d00b93fb4e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:08.881666 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3e58d-6087-46b7-afc1-d00b93fb4e79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:08.882506 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3e58d-6087-46b7-afc1-d00b93fb4e79 None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=abcddf66e05c418cb148b321f2423aab) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:08.882820 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3e58d-6087-46b7-afc1-d00b93fb4e79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:08.884344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a3e58d-6087-46b7-afc1-d00b93fb4e79 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:08.895008 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 337/668] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:08 2022] GET /identity/v3/role_assignments?user.id=abcddf66e05c418cb148b321f2423aab&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:08.895008 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3e58d-6087-46b7-afc1-d00b93fb4e79 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:08.902732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a92ed-c4b2-4eea-a3c0-78d3b48daf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:08.904215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a92ed-c4b2-4eea-a3c0-78d3b48daf8e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:08.904533 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a92ed-c4b2-4eea-a3c0-78d3b48daf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:08.905423 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a92ed-c4b2-4eea-a3c0-78d3b48daf8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:08.924380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a92ed-c4b2-4eea-a3c0-78d3b48daf8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:08.926499 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a92ed-c4b2-4eea-a3c0-78d3b48daf8e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-096a92ed-c4b2-4eea-a3c0-78d3b48daf8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:08.952784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:08.953845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:08.954991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:08.954991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:08.954991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:08.954991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:08.954991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 332/669] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:08.955576 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a92ed-c4b2-4eea-a3c0-78d3b48daf8e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:09.018520 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3e58d-6087-46b7-afc1-d00b93fb4e79 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:09.059811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07d89b-a690-46d1-b30a-a5738f14b67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:09.060766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07d89b-a690-46d1-b30a-a5738f14b67b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.061113 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07d89b-a690-46d1-b30a-a5738f14b67b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.061471 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07d89b-a690-46d1-b30a-a5738f14b67b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.074619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07d89b-a690-46d1-b30a-a5738f14b67b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:09.076392 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07d89b-a690-46d1-b30a-a5738f14b67b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:09.082568 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a92ed-c4b2-4eea-a3c0-78d3b48daf8e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-0e07d89b-a690-46d1-b30a-a5738f14b67b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:09.096776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:09.097943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:09.099009 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:09.099009 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:09.099009 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:09.099009 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:09.099411 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 338/670] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:09.100262 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07d89b-a690-46d1-b30a-a5738f14b67b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:09.133986 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:09.135806 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c222a-30e1-4d9f-916d-ecbc4a8ee3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:09.136876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c222a-30e1-4d9f-916d-ecbc4a8ee3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.137161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c222a-30e1-4d9f-916d-ecbc4a8ee3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.137428 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c222a-30e1-4d9f-916d-ecbc4a8ee3e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.149915 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c222a-30e1-4d9f-916d-ecbc4a8ee3e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-b03c222a-30e1-4d9f-916d-ecbc4a8ee3e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:09.158612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:09.159607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:09.160587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:09.161245 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 333/671] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:47:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:09.162122 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8085d-aebb-47fc-8097-bbdc376800fd None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.162414 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8085d-aebb-47fc-8097-bbdc376800fd None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.162754 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8085d-aebb-47fc-8097-bbdc376800fd None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.163971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 334/672] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:09.165360 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8085d-aebb-47fc-8097-bbdc376800fd None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:09.185272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:09.237548 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07d89b-a690-46d1-b30a-a5738f14b67b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:09.278149 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1553412-48ee-409e-b8c7-93e8224c2359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:09.279406 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1553412-48ee-409e-b8c7-93e8224c2359 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.279766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1553412-48ee-409e-b8c7-93e8224c2359 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.279955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1553412-48ee-409e-b8c7-93e8224c2359 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.280722 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1553412-48ee-409e-b8c7-93e8224c2359 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:09.281026 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1553412-48ee-409e-b8c7-93e8224c2359 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:09.282399 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1553412-48ee-409e-b8c7-93e8224c2359 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:09.283873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8085d-aebb-47fc-8097-bbdc376800fd None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:09.294531 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 339/673] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:09.295288 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1553412-48ee-409e-b8c7-93e8224c2359 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:09.321876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1089f8f-c794-4e92-b329-1c05b6b0bfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:09.322639 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1089f8f-c794-4e92-b329-1c05b6b0bfef None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.322916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1089f8f-c794-4e92-b329-1c05b6b0bfef None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.323375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1089f8f-c794-4e92-b329-1c05b6b0bfef None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.324232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1089f8f-c794-4e92-b329-1c05b6b0bfef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:09.324561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1089f8f-c794-4e92-b329-1c05b6b0bfef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:09.326547 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1089f8f-c794-4e92-b329-1c05b6b0bfef None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:09.350181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 335/674] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:09.350181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41661d-20b0-40e4-80ee-fdcae54e686c None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.350181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41661d-20b0-40e4-80ee-fdcae54e686c None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.350181 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41661d-20b0-40e4-80ee-fdcae54e686c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.365399 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-5c41661d-20b0-40e4-80ee-fdcae54e686c None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:09.366613 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-5c41661d-20b0-40e4-80ee-fdcae54e686c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:09.405127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:09.418014 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1553412-48ee-409e-b8c7-93e8224c2359 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:09.445032 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 336/675] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:47:09.445906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41661d-20b0-40e4-80ee-fdcae54e686c None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:09.449776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:09.473293 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27578d82-419a-4267-a94c-8f98cfdb69df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:09.474097 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27578d82-419a-4267-a94c-8f98cfdb69df None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.474385 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27578d82-419a-4267-a94c-8f98cfdb69df None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.474676 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27578d82-419a-4267-a94c-8f98cfdb69df None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.475431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27578d82-419a-4267-a94c-8f98cfdb69df None admin] RBAC: Adding query filter params (name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:09.475738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27578d82-419a-4267-a94c-8f98cfdb69df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:09.477237 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27578d82-419a-4267-a94c-8f98cfdb69df None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:09.489396 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 340/676] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:47:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:09.490182 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27578d82-419a-4267-a94c-8f98cfdb69df None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:09.561739 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41661d-20b0-40e4-80ee-fdcae54e686c None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:09.601341 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ad287-9c13-4745-a2af-cacfec531aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:09.602432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ad287-9c13-4745-a2af-cacfec531aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.602894 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ad287-9c13-4745-a2af-cacfec531aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.603317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5ad287-9c13-4745-a2af-cacfec531aa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.611018 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ad287-9c13-4745-a2af-cacfec531aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:09.612889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5ad287-9c13-4745-a2af-cacfec531aa7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:09.621641 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27578d82-419a-4267-a94c-8f98cfdb69df None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-8e5ad287-9c13-4745-a2af-cacfec531aa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:09.627212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:09.628103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:09.629020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:09.629020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:09.629020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:09.629020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:09.629020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:09.629020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:09.629020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:09.629020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:09.629020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 337/677] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:09.630036 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ad287-9c13-4745-a2af-cacfec531aa7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:09.668209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb57bb7-62d0-4fe4-90e6-c8d317cb4024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:09.668965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb57bb7-62d0-4fe4-90e6-c8d317cb4024 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.669249 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb57bb7-62d0-4fe4-90e6-c8d317cb4024 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.669579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb57bb7-62d0-4fe4-90e6-c8d317cb4024 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.676928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb57bb7-62d0-4fe4-90e6-c8d317cb4024 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:09.678277 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb57bb7-62d0-4fe4-90e6-c8d317cb4024 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-4eb57bb7-62d0-4fe4-90e6-c8d317cb4024 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:09.693817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:09.694745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:09.695682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:09.695682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:09.695682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:09.695682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:09.695682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:09.695682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:09.695682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:09.695682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:09.695682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 341/678] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:09.696992 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb57bb7-62d0-4fe4-90e6-c8d317cb4024 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:09.706537 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:09.710500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:09.751607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5ad287-9c13-4745-a2af-cacfec531aa7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:09.793941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03e210-b468-48fe-935a-465b43d05720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:09.798404 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03e210-b468-48fe-935a-465b43d05720 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.798404 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03e210-b468-48fe-935a-465b43d05720 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.798404 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03e210-b468-48fe-935a-465b43d05720 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.808615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03e210-b468-48fe-935a-465b43d05720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:09.811610 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb57bb7-62d0-4fe4-90e6-c8d317cb4024 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-0f03e210-b468-48fe-935a-465b43d05720 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:09.817190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:09.818326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 338/679] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:09.824625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03e210-b468-48fe-935a-465b43d05720 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:09.856997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a4d56-ca90-4b37-9bf9-ca0c8f012445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:09.858388 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4d56-ca90-4b37-9bf9-ca0c8f012445 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.858852 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4d56-ca90-4b37-9bf9-ca0c8f012445 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.859316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4d56-ca90-4b37-9bf9-ca0c8f012445 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.870350 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a4d56-ca90-4b37-9bf9-ca0c8f012445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:09.871827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a4d56-ca90-4b37-9bf9-ca0c8f012445 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-272a4d56-ca90-4b37-9bf9-ca0c8f012445 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:09.892859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:09.893957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:09.894839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:09.894839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:09.894839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:09.894839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:09.894839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 342/680] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:09.895112 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a4d56-ca90-4b37-9bf9-ca0c8f012445 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:09.938110 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03e210-b468-48fe-935a-465b43d05720 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:09.962288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:09.969286 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:09.976408 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b694b-83aa-4e58-86d3-d84895f445e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:09.977066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b694b-83aa-4e58-86d3-d84895f445e2 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:09.977347 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b694b-83aa-4e58-86d3-d84895f445e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:09.977620 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b694b-83aa-4e58-86d3-d84895f445e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:09.978417 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b694b-83aa-4e58-86d3-d84895f445e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:09.978723 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b694b-83aa-4e58-86d3-d84895f445e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:09.980165 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b694b-83aa-4e58-86d3-d84895f445e2 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:09.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 339/681] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:09.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b694b-83aa-4e58-86d3-d84895f445e2 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:10.016214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a4d56-ca90-4b37-9bf9-ca0c8f012445 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:10.058604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a687cfc-28e1-419b-a047-8f75bb6552f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:10.059266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687cfc-28e1-419b-a047-8f75bb6552f0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:10.059729 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687cfc-28e1-419b-a047-8f75bb6552f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:10.060704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687cfc-28e1-419b-a047-8f75bb6552f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:10.061624 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a687cfc-28e1-419b-a047-8f75bb6552f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:10.061932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a687cfc-28e1-419b-a047-8f75bb6552f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:10.063362 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a687cfc-28e1-419b-a047-8f75bb6552f0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:10.080507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 343/682] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:10.080507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a687cfc-28e1-419b-a047-8f75bb6552f0 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:10.116322 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b694b-83aa-4e58-86d3-d84895f445e2 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:10.152185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2703c-b84c-4e47-a8a9-52712d52f0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:10.152783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2703c-b84c-4e47-a8a9-52712d52f0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:10.153063 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2703c-b84c-4e47-a8a9-52712d52f0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:10.153332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a2703c-b84c-4e47-a8a9-52712d52f0a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:10.154201 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2703c-b84c-4e47-a8a9-52712d52f0a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:10.154508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2703c-b84c-4e47-a8a9-52712d52f0a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:10.155952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a2703c-b84c-4e47-a8a9-52712d52f0a3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:10.165598 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 340/683] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:47:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:10.166465 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2703c-b84c-4e47-a8a9-52712d52f0a3 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:10.195656 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a687cfc-28e1-419b-a047-8f75bb6552f0 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:10.214864 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:10.221425 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:10.234037 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f60e11-e758-43cb-812a-8a2fbfd6c969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:10.234904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f60e11-e758-43cb-812a-8a2fbfd6c969 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:10.235264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f60e11-e758-43cb-812a-8a2fbfd6c969 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:10.235544 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f60e11-e758-43cb-812a-8a2fbfd6c969 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:10.236280 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f60e11-e758-43cb-812a-8a2fbfd6c969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:10.236644 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f60e11-e758-43cb-812a-8a2fbfd6c969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:10.238012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f60e11-e758-43cb-812a-8a2fbfd6c969 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:10.251034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 344/684] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:10.251886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f60e11-e758-43cb-812a-8a2fbfd6c969 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:10.275026 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a2703c-b84c-4e47-a8a9-52712d52f0a3 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:10.314689 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc9346-949a-4237-9d96-541cd9f00521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:10.315340 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc9346-949a-4237-9d96-541cd9f00521 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:10.315623 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc9346-949a-4237-9d96-541cd9f00521 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:10.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc9346-949a-4237-9d96-541cd9f00521 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:10.316765 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc9346-949a-4237-9d96-541cd9f00521 None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=86eeca6ef9374305b98ac8f8b03f2fc5) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:10.317073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc9346-949a-4237-9d96-541cd9f00521 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:10.318532 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc9346-949a-4237-9d96-541cd9f00521 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:10.325334 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 341/685] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:10 2022] GET /identity/v3/role_assignments?user.id=86eeca6ef9374305b98ac8f8b03f2fc5&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:10.326046 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc9346-949a-4237-9d96-541cd9f00521 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:10.385735 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f60e11-e758-43cb-812a-8a2fbfd6c969 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:10.429441 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24d9e4-7f50-464c-9cb4-1d8d288398be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:10.430616 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24d9e4-7f50-464c-9cb4-1d8d288398be None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:10.431135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24d9e4-7f50-464c-9cb4-1d8d288398be None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:10.431640 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24d9e4-7f50-464c-9cb4-1d8d288398be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:10.432766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24d9e4-7f50-464c-9cb4-1d8d288398be None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=6d5a01edfe0444f8a6fef9d7ee9645f3) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:10.433315 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24d9e4-7f50-464c-9cb4-1d8d288398be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:10.435019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24d9e4-7f50-464c-9cb4-1d8d288398be None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:10.442669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 345/686] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:10 2022] GET /identity/v3/role_assignments?user.id=6d5a01edfe0444f8a6fef9d7ee9645f3&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:10.444022 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24d9e4-7f50-464c-9cb4-1d8d288398be None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:10.466972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc9346-949a-4237-9d96-541cd9f00521 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:10.477986 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:10.510483 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:10.541215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a280d3c-1781-42d7-9332-e71dee2b2611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:10.542037 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280d3c-1781-42d7-9332-e71dee2b2611 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:10.542317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280d3c-1781-42d7-9332-e71dee2b2611 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:10.542594 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280d3c-1781-42d7-9332-e71dee2b2611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:10.561224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a280d3c-1781-42d7-9332-e71dee2b2611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:10.562721 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a280d3c-1781-42d7-9332-e71dee2b2611 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:10.571001 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24d9e4-7f50-464c-9cb4-1d8d288398be None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-6a280d3c-1781-42d7-9332-e71dee2b2611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:10.606061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:10.607658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:10.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:10.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:10.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:10.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:10.609208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 342/687] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:10.613559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a280d3c-1781-42d7-9332-e71dee2b2611 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:10.624089 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9736da0-5391-404f-801b-19460d0969da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:10.625676 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9736da0-5391-404f-801b-19460d0969da None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:10.626186 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9736da0-5391-404f-801b-19460d0969da None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:10.626696 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9736da0-5391-404f-801b-19460d0969da None admin] PATH_INFO: `/v3/users/swift` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:10.642259 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9736da0-5391-404f-801b-19460d0969da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:10.644006 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9736da0-5391-404f-801b-19460d0969da None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-c9736da0-5391-404f-801b-19460d0969da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:10.665012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:10.666020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:10.667000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:10.667000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:10.667000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:10.667000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:10.667000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:10.667000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:10.667000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:10.667000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:10.667000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:47:10.667000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:10.674836 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 346/688] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:10.676765 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9736da0-5391-404f-801b-19460d0969da None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:10.751941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a280d3c-1781-42d7-9332-e71dee2b2611 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:10.793802 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b240bb3-9cc8-44b3-a493-9790fd524ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:10.794865 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b240bb3-9cc8-44b3-a493-9790fd524ebb None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:10.795386 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b240bb3-9cc8-44b3-a493-9790fd524ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:10.795863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b240bb3-9cc8-44b3-a493-9790fd524ebb None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:10.796821 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b240bb3-9cc8-44b3-a493-9790fd524ebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:10.797330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b240bb3-9cc8-44b3-a493-9790fd524ebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:10.798938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b240bb3-9cc8-44b3-a493-9790fd524ebb None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:10.812507 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 343/689] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:10.813951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9736da0-5391-404f-801b-19460d0969da None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:10.822198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b240bb3-9cc8-44b3-a493-9790fd524ebb None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:10.830207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:10.863978 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:10.877541 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da97679-ec38-4914-8b7a-8983d28fd8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:10.878240 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97679-ec38-4914-8b7a-8983d28fd8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:10.878529 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97679-ec38-4914-8b7a-8983d28fd8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:10.878811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da97679-ec38-4914-8b7a-8983d28fd8f4 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:10.879674 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da97679-ec38-4914-8b7a-8983d28fd8f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:10.879984 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da97679-ec38-4914-8b7a-8983d28fd8f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:10.881462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da97679-ec38-4914-8b7a-8983d28fd8f4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:10.923654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 347/690] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 19:47:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:10.930003 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da97679-ec38-4914-8b7a-8983d28fd8f4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:10.985528 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b240bb3-9cc8-44b3-a493-9790fd524ebb None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:11.025294 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d7edf-e030-43ad-8591-8833e1739363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:11.026070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d7edf-e030-43ad-8591-8833e1739363 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:11.026355 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d7edf-e030-43ad-8591-8833e1739363 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:11.026632 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d7edf-e030-43ad-8591-8833e1739363 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:11.035090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d7edf-e030-43ad-8591-8833e1739363 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:11.036751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d7edf-e030-43ad-8591-8833e1739363 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-b21d7edf-e030-43ad-8591-8833e1739363 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:11.054246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:11.055315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:11.056839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:11.056839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:11.056839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:11.056839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:11.056839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:11.056839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:11.056839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:11.056839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:11.056839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:47:11.056839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:11.056839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 344/691] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 19:47:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:11.065683 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d7edf-e030-43ad-8591-8833e1739363 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:11.115374 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da97679-ec38-4914-8b7a-8983d28fd8f4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:11.162296 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ce667-dfae-4084-b09c-1850db70c0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:11.162966 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ce667-dfae-4084-b09c-1850db70c0be None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:11.163248 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ce667-dfae-4084-b09c-1850db70c0be None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:11.163521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ce667-dfae-4084-b09c-1850db70c0be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:11.242069 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:11.252744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:11.269485 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d7edf-e030-43ad-8591-8833e1739363 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:11.319272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67f212-06a3-458d-b67f-ffcb6532bad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:11.320270 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67f212-06a3-458d-b67f-ffcb6532bad2 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:11.320737 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67f212-06a3-458d-b67f-ffcb6532bad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:11.321161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67f212-06a3-458d-b67f-ffcb6532bad2 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:11.322191 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67f212-06a3-458d-b67f-ffcb6532bad2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:11.322647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67f212-06a3-458d-b67f-ffcb6532bad2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:11.325575 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67f212-06a3-458d-b67f-ffcb6532bad2 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:11.329171 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ce667-dfae-4084-b09c-1850db70c0be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:11.330894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ce667-dfae-4084-b09c-1850db70c0be None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:11.358395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 345/692] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 19:47:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:11.360107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d464fbc-8cad-4786-a58d-2117e6b231d7 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:11.360672 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d464fbc-8cad-4786-a58d-2117e6b231d7 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:11.361101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d464fbc-8cad-4786-a58d-2117e6b231d7 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:11.362815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 346/693] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:11.363633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d464fbc-8cad-4786-a58d-2117e6b231d7 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:11.457130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 348/694] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:47:11.458528 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050cdbb-c68c-43a9-978d-324a9cd46189 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:11.458818 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050cdbb-c68c-43a9-978d-324a9cd46189 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:11.459457 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050cdbb-c68c-43a9-978d-324a9cd46189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:11.481120 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-9050cdbb-c68c-43a9-978d-324a9cd46189 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:11.483483 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-9050cdbb-c68c-43a9-978d-324a9cd46189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:11.486292 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d464fbc-8cad-4786-a58d-2117e6b231d7 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:11.531482 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf72b92-5d9c-485e-be9a-286b085fcb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:11.532300 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf72b92-5d9c-485e-be9a-286b085fcb6c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:11.532726 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf72b92-5d9c-485e-be9a-286b085fcb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:11.533576 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf72b92-5d9c-485e-be9a-286b085fcb6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:11.564797 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:11.565471 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:11.568489 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 349/695] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:47:11.569557 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050cdbb-c68c-43a9-978d-324a9cd46189 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:11.625185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf72b92-5d9c-485e-be9a-286b085fcb6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:11.626978 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf72b92-5d9c-485e-be9a-286b085fcb6c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:11.709673 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050cdbb-c68c-43a9-978d-324a9cd46189 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:11.746131 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 347/696] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:47:11.747840 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf72b92-5d9c-485e-be9a-286b085fcb6c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:11.752066 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743656d-df8c-4ff6-be47-c1e6ec74cf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:11.752208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743656d-df8c-4ff6-be47-c1e6ec74cf1a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:11.752499 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743656d-df8c-4ff6-be47-c1e6ec74cf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:11.752774 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743656d-df8c-4ff6-be47-c1e6ec74cf1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:11.764445 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743656d-df8c-4ff6-be47-c1e6ec74cf1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:11.765999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743656d-df8c-4ff6-be47-c1e6ec74cf1a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-e743656d-df8c-4ff6-be47-c1e6ec74cf1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:11.786258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:11.787194 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:11.788109 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:11.788109 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:11.788109 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:11.788109 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:11.791976 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 350/697] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:11.792863 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743656d-df8c-4ff6-be47-c1e6ec74cf1a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:11.794391 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:11.850477 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:11.860599 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf72b92-5d9c-485e-be9a-286b085fcb6c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:11.896531 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945754e0-0012-408f-973a-68269152462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:11.897553 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945754e0-0012-408f-973a-68269152462c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:11.897925 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945754e0-0012-408f-973a-68269152462c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:11.898295 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945754e0-0012-408f-973a-68269152462c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:11.911342 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945754e0-0012-408f-973a-68269152462c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:11.913016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945754e0-0012-408f-973a-68269152462c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:11.917528 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743656d-df8c-4ff6-be47-c1e6ec74cf1a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-945754e0-0012-408f-973a-68269152462c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:11.932490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:11.933441 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:11.934325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:11.934325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:11.934325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:11.934325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:11.939132 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 348/698] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:11.939855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945754e0-0012-408f-973a-68269152462c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:11.966459 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e38cc5-7b55-411e-b489-e6462ada023b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:11.967262 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e38cc5-7b55-411e-b489-e6462ada023b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:11.967653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e38cc5-7b55-411e-b489-e6462ada023b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:11.967935 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e38cc5-7b55-411e-b489-e6462ada023b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:11.979537 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e38cc5-7b55-411e-b489-e6462ada023b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:11.981034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e38cc5-7b55-411e-b489-e6462ada023b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-c5e38cc5-7b55-411e-b489-e6462ada023b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:12.000956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:12.001938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:12.002856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:12.002856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:12.002856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:12.002856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:12.006780 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 351/699] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:12.007669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e38cc5-7b55-411e-b489-e6462ada023b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:12.048624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:12.058996 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945754e0-0012-408f-973a-68269152462c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:12.095335 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b9966-92e7-4a8a-a283-851e987db373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:12.095995 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9966-92e7-4a8a-a283-851e987db373 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:12.096274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9966-92e7-4a8a-a283-851e987db373 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:12.096586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7b9966-92e7-4a8a-a283-851e987db373 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:12.097306 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b9966-92e7-4a8a-a283-851e987db373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:12.097606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b9966-92e7-4a8a-a283-851e987db373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:12.099138 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7b9966-92e7-4a8a-a283-851e987db373 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:12.115505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 349/700] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:12.115505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b9966-92e7-4a8a-a283-851e987db373 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:12.127314 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:12.132102 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e38cc5-7b55-411e-b489-e6462ada023b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:12.168363 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6708b8-1212-4d3b-921b-9c993801fc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:12.169377 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6708b8-1212-4d3b-921b-9c993801fc36 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:12.169879 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6708b8-1212-4d3b-921b-9c993801fc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:12.170367 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6708b8-1212-4d3b-921b-9c993801fc36 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:12.171312 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6708b8-1212-4d3b-921b-9c993801fc36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:12.171804 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6708b8-1212-4d3b-921b-9c993801fc36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:12.173376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6708b8-1212-4d3b-921b-9c993801fc36 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:12.188139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 352/701] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:12.189226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6708b8-1212-4d3b-921b-9c993801fc36 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:12.223427 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7b9966-92e7-4a8a-a283-851e987db373 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:12.258469 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144825a-d3f4-4ac1-ad52-fdf8d9178063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:12.259190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144825a-d3f4-4ac1-ad52-fdf8d9178063 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:12.259411 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144825a-d3f4-4ac1-ad52-fdf8d9178063 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:12.259682 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144825a-d3f4-4ac1-ad52-fdf8d9178063 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:12.260437 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144825a-d3f4-4ac1-ad52-fdf8d9178063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:12.260736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144825a-d3f4-4ac1-ad52-fdf8d9178063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:12.262108 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144825a-d3f4-4ac1-ad52-fdf8d9178063 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:12.279598 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 350/702] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:12.280565 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144825a-d3f4-4ac1-ad52-fdf8d9178063 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:12.292909 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:12.302525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6708b8-1212-4d3b-921b-9c993801fc36 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:12.343943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9d182-bc57-4611-9003-6bc81e561187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:12.345190 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9d182-bc57-4611-9003-6bc81e561187 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:12.345655 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9d182-bc57-4611-9003-6bc81e561187 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:12.346163 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9d182-bc57-4611-9003-6bc81e561187 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:12.353812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9d182-bc57-4611-9003-6bc81e561187 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:12.355542 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9d182-bc57-4611-9003-6bc81e561187 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-0ed9d182-bc57-4611-9003-6bc81e561187 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:12.372743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:12.373714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:12.374587 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:12.374587 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:12.374587 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:12.374587 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:12.374587 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:12.374587 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:12.374587 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:12.374587 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:12.381550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 353/703] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:12.382668 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9d182-bc57-4611-9003-6bc81e561187 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:12.389201 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144825a-d3f4-4ac1-ad52-fdf8d9178063 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:12.403237 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:12.425643 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482489a0-3257-4352-8799-c2db876181da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:12.426578 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482489a0-3257-4352-8799-c2db876181da None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:12.426858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482489a0-3257-4352-8799-c2db876181da None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:12.427129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482489a0-3257-4352-8799-c2db876181da None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:12.427840 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482489a0-3257-4352-8799-c2db876181da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:12.429639 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482489a0-3257-4352-8799-c2db876181da None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:12.492147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9d182-bc57-4611-9003-6bc81e561187 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:12.524152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 351/704] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:47:12 2022] POST /identity/v3/users => generated 254 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:12.524152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482489a0-3257-4352-8799-c2db876181da None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:12.556881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9856f-5a73-4f30-824b-94a92ac6594d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:12.557697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9856f-5a73-4f30-824b-94a92ac6594d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:12.557981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9856f-5a73-4f30-824b-94a92ac6594d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:12.558256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9856f-5a73-4f30-824b-94a92ac6594d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:12.573583 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9856f-5a73-4f30-824b-94a92ac6594d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:12.575559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9856f-5a73-4f30-824b-94a92ac6594d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-07c9856f-5a73-4f30-824b-94a92ac6594d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:12.601183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:12.602164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:12.603575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:12.603575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:12.603575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:12.603575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:12.603575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:12.603575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:12.603575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:12.603575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:12.603575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 354/705] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:12.615042 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9856f-5a73-4f30-824b-94a92ac6594d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:12.664460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:12.738229 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482489a0-3257-4352-8799-c2db876181da None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:12.788447 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:12.790124 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b67435-6d2c-44c4-9622-3511c6ba528e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:12.790832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b67435-6d2c-44c4-9622-3511c6ba528e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:12.791098 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b67435-6d2c-44c4-9622-3511c6ba528e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:12.791485 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b67435-6d2c-44c4-9622-3511c6ba528e None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:12.792185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b67435-6d2c-44c4-9622-3511c6ba528e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:12.792552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b67435-6d2c-44c4-9622-3511c6ba528e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:12.794066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b67435-6d2c-44c4-9622-3511c6ba528e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:12.809120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 352/706] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:12.815599 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b67435-6d2c-44c4-9622-3511c6ba528e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:12.819970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9856f-5a73-4f30-824b-94a92ac6594d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:12.863102 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fdfef-9769-4eab-827c-a582529cd85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:12.864305 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fdfef-9769-4eab-827c-a582529cd85a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:12.865038 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fdfef-9769-4eab-827c-a582529cd85a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:12.865534 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76fdfef-9769-4eab-827c-a582529cd85a None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:12.866685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fdfef-9769-4eab-827c-a582529cd85a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:12.867247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fdfef-9769-4eab-827c-a582529cd85a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:12.869325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76fdfef-9769-4eab-827c-a582529cd85a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:12.885947 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 355/707] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:12.898004 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fdfef-9769-4eab-827c-a582529cd85a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:12.936637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b67435-6d2c-44c4-9622-3511c6ba528e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:12.968164 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:12.981431 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074ddb7-3cc1-486b-883b-ae76e81f4b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:12.982097 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ddb7-3cc1-486b-883b-ae76e81f4b11 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:12.982378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ddb7-3cc1-486b-883b-ae76e81f4b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:12.982653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ddb7-3cc1-486b-883b-ae76e81f4b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:12.983500 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074ddb7-3cc1-486b-883b-ae76e81f4b11 None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=c8a36bb1e26f4faaa44fabff8b509fe7) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:12.983811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074ddb7-3cc1-486b-883b-ae76e81f4b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:12.985292 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074ddb7-3cc1-486b-883b-ae76e81f4b11 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:12.994483 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 353/708] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:12 2022] GET /identity/v3/role_assignments?user.id=c8a36bb1e26f4faaa44fabff8b509fe7&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:13.031987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76fdfef-9769-4eab-827c-a582529cd85a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:13.062302 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709642-4432-4848-846d-1235445c49ae None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:13.062673 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709642-4432-4848-846d-1235445c49ae None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:13.062921 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a709642-4432-4848-846d-1235445c49ae None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:13.065373 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 354/709] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:13.071675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e27af5-9b40-49bb-b3d8-716e6cd5ed60 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:13.071982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e27af5-9b40-49bb-b3d8-716e6cd5ed60 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:13.072245 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e27af5-9b40-49bb-b3d8-716e6cd5ed60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:13.086311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f991251-f4b4-4f76-aee5-c8c7fe39a1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:13.086988 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f991251-f4b4-4f76-aee5-c8c7fe39a1cc None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:13.087402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f991251-f4b4-4f76-aee5-c8c7fe39a1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:13.087674 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f991251-f4b4-4f76-aee5-c8c7fe39a1cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:13.088631 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f991251-f4b4-4f76-aee5-c8c7fe39a1cc None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=0c0e71329c4a4470935593636c025257) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:13.089033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f991251-f4b4-4f76-aee5-c8c7fe39a1cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:13.090505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f991251-f4b4-4f76-aee5-c8c7fe39a1cc None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:13.094878 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-11e27af5-9b40-49bb-b3d8-716e6cd5ed60 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:13.096736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-11e27af5-9b40-49bb-b3d8-716e6cd5ed60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:13.097665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 356/710] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:12 2022] GET /identity/v3/role_assignments?user.id=0c0e71329c4a4470935593636c025257&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:13.099010 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53f1b7-37f6-48ad-a5cc-f95b492645ac None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:13.099295 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53f1b7-37f6-48ad-a5cc-f95b492645ac None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:13.099571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53f1b7-37f6-48ad-a5cc-f95b492645ac None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:13.101394 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 357/711] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:13.107766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274b71b-08ab-4699-965f-c833d21896c5 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:13.108064 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274b71b-08ab-4699-965f-c833d21896c5 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:13.108355 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274b71b-08ab-4699-965f-c833d21896c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:13.113933 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:13.136697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-e274b71b-08ab-4699-965f-c833d21896c5 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:13.136697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-e274b71b-08ab-4699-965f-c833d21896c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:13.244052 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 355/712] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:47:13.270357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 358/713] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:47:13.274593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e27af5-9b40-49bb-b3d8-716e6cd5ed60 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:13.301655 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274b71b-08ab-4699-965f-c833d21896c5 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:13.409691 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274b71b-08ab-4699-965f-c833d21896c5 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:13.417192 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e27af5-9b40-49bb-b3d8-716e6cd5ed60 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:13.441058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:13.442502 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:13.446809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a719c-ed84-4143-af84-9a372c691e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:13.447614 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a719c-ed84-4143-af84-9a372c691e25 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:13.447894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a719c-ed84-4143-af84-9a372c691e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:13.448167 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a719c-ed84-4143-af84-9a372c691e25 None admin] PATH_INFO: `/v3/services/network` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:13.450964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a719c-ed84-4143-af84-9a372c691e25 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:13.452975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a719c-ed84-4143-af84-9a372c691e25 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:13.461625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad810034-24ef-4b19-95b8-a4d38492a2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:13.462275 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad810034-24ef-4b19-95b8-a4d38492a2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:13.462564 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad810034-24ef-4b19-95b8-a4d38492a2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:13.462840 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad810034-24ef-4b19-95b8-a4d38492a2b5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:13.463510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad810034-24ef-4b19-95b8-a4d38492a2b5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:13.464932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad810034-24ef-4b19-95b8-a4d38492a2b5 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-345a719c-ed84-4143-af84-9a372c691e25 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:13.477107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:13.478269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:13.479137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 359/714] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:13 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-ad810034-24ef-4b19-95b8-a4d38492a2b5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:13.486345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:13.488212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:47:13.489115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:13.491225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a719c-ed84-4143-af84-9a372c691e25 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:13.492760 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 356/715] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 19:47:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:13.510899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad810034-24ef-4b19-95b8-a4d38492a2b5 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:13.662630 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad810034-24ef-4b19-95b8-a4d38492a2b5 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:13.734595 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f0eea-2b2b-4375-bfc7-0569a80afdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:13.735389 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f0eea-2b2b-4375-bfc7-0569a80afdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:13.735560 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f0eea-2b2b-4375-bfc7-0569a80afdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:13.737987 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f0eea-2b2b-4375-bfc7-0569a80afdf3 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:13.738358 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a719c-ed84-4143-af84-9a372c691e25 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:13.740143 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f0eea-2b2b-4375-bfc7-0569a80afdf3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:13.741344 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f0eea-2b2b-4375-bfc7-0569a80afdf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:13.742763 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f0eea-2b2b-4375-bfc7-0569a80afdf3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:13.758368 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 357/716] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 19:47:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:13.767866 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f0eea-2b2b-4375-bfc7-0569a80afdf3 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:13.814575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b59a3-af28-4344-9539-724464540db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:13.815252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b59a3-af28-4344-9539-724464540db5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:13.815537 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b59a3-af28-4344-9539-724464540db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:13.815815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b59a3-af28-4344-9539-724464540db5 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:13.816564 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b59a3-af28-4344-9539-724464540db5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:13.816869 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b59a3-af28-4344-9539-724464540db5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:13.818253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b59a3-af28-4344-9539-724464540db5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:13.835018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 360/717] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:13.837938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b59a3-af28-4344-9539-724464540db5 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:13.879522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:13.968825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:13.995635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f0eea-2b2b-4375-bfc7-0569a80afdf3 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:14.059189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b59a3-af28-4344-9539-724464540db5 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:14.073353 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d185d7-095b-48b3-8163-9c314e446c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:14.074029 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d185d7-095b-48b3-8163-9c314e446c98 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:14.074313 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d185d7-095b-48b3-8163-9c314e446c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:14.074587 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d185d7-095b-48b3-8163-9c314e446c98 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:14.075342 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d185d7-095b-48b3-8163-9c314e446c98 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:14.075653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d185d7-095b-48b3-8163-9c314e446c98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:14.079691 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d185d7-095b-48b3-8163-9c314e446c98 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:14.098534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 358/718] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 19:47:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:14.129384 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dc85b-9d98-4d79-b299-8f8b007fda06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:14.130079 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc85b-9d98-4d79-b299-8f8b007fda06 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:14.130416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc85b-9d98-4d79-b299-8f8b007fda06 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:14.130692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dc85b-9d98-4d79-b299-8f8b007fda06 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:14.131461 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dc85b-9d98-4d79-b299-8f8b007fda06 None admin] RBAC: Adding query filter params (type=network) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:14.131771 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dc85b-9d98-4d79-b299-8f8b007fda06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:14.133192 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900dc85b-9d98-4d79-b299-8f8b007fda06 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:14.145879 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 361/719] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:15.405096 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436658a-2a27-4459-8f82-620d9fc4213a None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:15.405615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436658a-2a27-4459-8f82-620d9fc4213a None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:15.405615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e436658a-2a27-4459-8f82-620d9fc4213a None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:15.407252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 359/720] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:15.412964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3f8fd-b606-4e18-95ae-f4f4a13532ce None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:15.413411 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3f8fd-b606-4e18-95ae-f4f4a13532ce None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:15.413590 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3f8fd-b606-4e18-95ae-f4f4a13532ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:15.436303 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-cee3f8fd-b606-4e18-95ae-f4f4a13532ce None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:15.437510 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-cee3f8fd-b606-4e18-95ae-f4f4a13532ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:15.648028 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 362/721] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:47:15.672367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436658a-2a27-4459-8f82-620d9fc4213a None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:15.792110 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e436658a-2a27-4459-8f82-620d9fc4213a None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:15.816666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:15.846050 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cc2f7-6447-432d-aa8c-06f03b4311cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:15.847282 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cc2f7-6447-432d-aa8c-06f03b4311cc None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:15.847772 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cc2f7-6447-432d-aa8c-06f03b4311cc None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:15.848268 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cc2f7-6447-432d-aa8c-06f03b4311cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:15.869295 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cc2f7-6447-432d-aa8c-06f03b4311cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-1f1cc2f7-6447-432d-aa8c-06f03b4311cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:15.877953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:15.878973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:15.879810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:15.882767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 360/722] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:47:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:15.893573 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3f8fd-b606-4e18-95ae-f4f4a13532ce None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:15.983861 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:16.070495 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3f8fd-b606-4e18-95ae-f4f4a13532ce None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:16.093431 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbaedae-dc45-45f4-87a8-4c7384941a92 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.094069 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbaedae-dc45-45f4-87a8-4c7384941a92 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.094556 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbaedae-dc45-45f4-87a8-4c7384941a92 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.096384 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 361/723] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:16.101789 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7106dc-3dae-41b8-87f7-a048eca4d556 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.102315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7106dc-3dae-41b8-87f7-a048eca4d556 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.102841 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7106dc-3dae-41b8-87f7-a048eca4d556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.119006 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedde3e-78b4-468f-a2ab-23e606c83728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:16.119688 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedde3e-78b4-468f-a2ab-23e606c83728 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.120072 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedde3e-78b4-468f-a2ab-23e606c83728 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.120386 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aedde3e-78b4-468f-a2ab-23e606c83728 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.121133 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedde3e-78b4-468f-a2ab-23e606c83728 None admin] RBAC: Adding query filter params (name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:16.121439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedde3e-78b4-468f-a2ab-23e606c83728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:16.121883 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-1b7106dc-3dae-41b8-87f7-a048eca4d556 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:16.122833 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aedde3e-78b4-468f-a2ab-23e606c83728 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:16.123515 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-1b7106dc-3dae-41b8-87f7-a048eca4d556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:16.142751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 363/724] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:47:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:16.147913 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedde3e-78b4-468f-a2ab-23e606c83728 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:16.222703 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 362/725] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 19:47:16.227778 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949cb2b-ed9e-4b11-820f-4dd7bb29c697 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.227778 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949cb2b-ed9e-4b11-820f-4dd7bb29c697 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.227778 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949cb2b-ed9e-4b11-820f-4dd7bb29c697 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.227778 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 363/726] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:16.231645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a518fec-b166-4a77-b5da-140cba89df31 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.231933 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a518fec-b166-4a77-b5da-140cba89df31 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.232214 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a518fec-b166-4a77-b5da-140cba89df31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.260841 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-3a518fec-b166-4a77-b5da-140cba89df31 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:16.267146 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-3a518fec-b166-4a77-b5da-140cba89df31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:16.281623 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aedde3e-78b4-468f-a2ab-23e606c83728 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:16.301147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:16.323168 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ced339-617d-4c32-8d37-c25b9f9ab47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:16.324697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ced339-617d-4c32-8d37-c25b9f9ab47d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.326047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ced339-617d-4c32-8d37-c25b9f9ab47d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.326500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ced339-617d-4c32-8d37-c25b9f9ab47d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.349391 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ced339-617d-4c32-8d37-c25b9f9ab47d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:16.353106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ced339-617d-4c32-8d37-c25b9f9ab47d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:16.367386 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 364/727] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:47:16.368584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a518fec-b166-4a77-b5da-140cba89df31 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-20ced339-617d-4c32-8d37-c25b9f9ab47d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:16.388857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:16.389964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:16.390877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:16.390877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:16.390877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:16.390877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:16.392790 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 364/728] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:16.397904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ced339-617d-4c32-8d37-c25b9f9ab47d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:16.426204 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:16.513398 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a518fec-b166-4a77-b5da-140cba89df31 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:16.527419 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ced339-617d-4c32-8d37-c25b9f9ab47d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:16.551975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192c175-e745-489d-af45-4eddd2b29db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:16.553199 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192c175-e745-489d-af45-4eddd2b29db0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.553628 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192c175-e745-489d-af45-4eddd2b29db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.554061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192c175-e745-489d-af45-4eddd2b29db0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.567806 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192c175-e745-489d-af45-4eddd2b29db0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:16.570120 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c49df2-6480-4614-ab7c-d5b6d63b6dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:16.570788 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c49df2-6480-4614-ab7c-d5b6d63b6dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.571078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c49df2-6480-4614-ab7c-d5b6d63b6dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.571359 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c49df2-6480-4614-ab7c-d5b6d63b6dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.572098 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c49df2-6480-4614-ab7c-d5b6d63b6dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:16.572432 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c49df2-6480-4614-ab7c-d5b6d63b6dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:16.573799 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c49df2-6480-4614-ab7c-d5b6d63b6dc3 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-2192c175-e745-489d-af45-4eddd2b29db0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:16.577969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:16.579119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:16.580123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:16.586842 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 365/730] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:16.587062 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 365/730] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:16.587749 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c49df2-6480-4614-ab7c-d5b6d63b6dc3 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:16.588465 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192c175-e745-489d-af45-4eddd2b29db0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:16.632168 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:16.719255 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:16.728842 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c49df2-6480-4614-ab7c-d5b6d63b6dc3 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:16.733795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192c175-e745-489d-af45-4eddd2b29db0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:16.769417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc335b-13e5-4432-b082-f3dce54e97de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:16.770553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc335b-13e5-4432-b082-f3dce54e97de None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.771021 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc335b-13e5-4432-b082-f3dce54e97de None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.771474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc335b-13e5-4432-b082-f3dce54e97de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.779770 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b7214-8edc-40a9-9e92-7ea42f25c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:16.780731 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b7214-8edc-40a9-9e92-7ea42f25c88d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.781123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b7214-8edc-40a9-9e92-7ea42f25c88d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.781402 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b7214-8edc-40a9-9e92-7ea42f25c88d None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.782175 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b7214-8edc-40a9-9e92-7ea42f25c88d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:16.782484 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b7214-8edc-40a9-9e92-7ea42f25c88d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:16.783916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b7214-8edc-40a9-9e92-7ea42f25c88d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:16.788993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc335b-13e5-4432-b082-f3dce54e97de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:16.799721 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 366/731] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:47:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:16.799721 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b7214-8edc-40a9-9e92-7ea42f25c88d None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-bcdc335b-13e5-4432-b082-f3dce54e97de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:16.803067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:16.803991 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:16.804875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:16.807652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 366/732] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:16.808829 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc335b-13e5-4432-b082-f3dce54e97de None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:16.911026 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b7214-8edc-40a9-9e92-7ea42f25c88d None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:16.915599 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:16.928642 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc335b-13e5-4432-b082-f3dce54e97de None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:16.947781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd026b1e-acfe-4d91-97b4-5985b3640b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:16.948617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026b1e-acfe-4d91-97b4-5985b3640b17 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.949270 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026b1e-acfe-4d91-97b4-5985b3640b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.949270 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026b1e-acfe-4d91-97b4-5985b3640b17 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.957376 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd026b1e-acfe-4d91-97b4-5985b3640b17 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:16.958846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd026b1e-acfe-4d91-97b4-5985b3640b17 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:16.963963 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6179285-4912-4cb8-addf-dfd6168c72f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:16.964783 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6179285-4912-4cb8-addf-dfd6168c72f3 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.965063 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6179285-4912-4cb8-addf-dfd6168c72f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:16.965333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6179285-4912-4cb8-addf-dfd6168c72f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-bd026b1e-acfe-4d91-97b4-5985b3640b17 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:16.973805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:16.974952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:16.975875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:16.975875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:16.975875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:16.975875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:16.975875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:16.975875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:16.975875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:16.975875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:16.975875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:16.975875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:16.975875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 367/733] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 19:47:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:16.976512 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6179285-4912-4cb8-addf-dfd6168c72f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:16.976586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd026b1e-acfe-4d91-97b4-5985b3640b17 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:16.977222 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6179285-4912-4cb8-addf-dfd6168c72f3 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:16.992846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-a6179285-4912-4cb8-addf-dfd6168c72f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:16.996184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:16.997831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:16.998717 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:16.998717 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:16.998717 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:16.998717 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:16.998717 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 367/734] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:16.999384 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a73645-2553-4fe1-a569-14bd3bb9b6c9 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:16.999491 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a73645-2553-4fe1-a569-14bd3bb9b6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.000221 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a73645-2553-4fe1-a569-14bd3bb9b6c9 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.004494 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 368/735] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:17.005799 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e82a8-49fd-4e7b-8dc4-cce0173adebd None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.006092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e82a8-49fd-4e7b-8dc4-cce0173adebd None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.006370 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e82a8-49fd-4e7b-8dc4-cce0173adebd None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.007740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 369/736] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:17.008521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e82a8-49fd-4e7b-8dc4-cce0173adebd None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:17.078382 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd026b1e-acfe-4d91-97b4-5985b3640b17 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:17.107876 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e82a8-49fd-4e7b-8dc4-cce0173adebd None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:17.114000 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cb812-d771-46b7-8d98-562c941cf69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:17.114666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cb812-d771-46b7-8d98-562c941cf69a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.114950 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cb812-d771-46b7-8d98-562c941cf69a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.115218 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cb812-d771-46b7-8d98-562c941cf69a None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.115969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cb812-d771-46b7-8d98-562c941cf69a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:17.116271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cb812-d771-46b7-8d98-562c941cf69a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:17.117677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cb812-d771-46b7-8d98-562c941cf69a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:17.130912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 368/737] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:47:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:17.130912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cb812-d771-46b7-8d98-562c941cf69a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:17.142835 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4fe07-33bb-4f0d-9f9b-02296f03d869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:17.143504 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4fe07-33bb-4f0d-9f9b-02296f03d869 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.143781 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4fe07-33bb-4f0d-9f9b-02296f03d869 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.144050 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4fe07-33bb-4f0d-9f9b-02296f03d869 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.145040 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4fe07-33bb-4f0d-9f9b-02296f03d869 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:17.145351 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4fe07-33bb-4f0d-9f9b-02296f03d869 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:17.146784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de4fe07-33bb-4f0d-9f9b-02296f03d869 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:17.155836 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:17.164126 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 370/738] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 19:47:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:17.165392 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b40ee-4223-4ff8-a7b5-20f8d6b3eb45 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.165675 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b40ee-4223-4ff8-a7b5-20f8d6b3eb45 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.165943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b40ee-4223-4ff8-a7b5-20f8d6b3eb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.182554 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-3e9b40ee-4223-4ff8-a7b5-20f8d6b3eb45 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:17.183740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-3e9b40ee-4223-4ff8-a7b5-20f8d6b3eb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:17.212152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:17.235284 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cb812-d771-46b7-8d98-562c941cf69a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:17.249005 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 371/739] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:47:17.250265 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87465a-efc0-45e5-8aee-fc24b77422aa None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.250578 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87465a-efc0-45e5-8aee-fc24b77422aa None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.250823 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87465a-efc0-45e5-8aee-fc24b77422aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.268569 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd817f0c-cc24-491f-8856-d47d8f22b1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:17.269235 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd817f0c-cc24-491f-8856-d47d8f22b1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.269522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd817f0c-cc24-491f-8856-d47d8f22b1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.269963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd817f0c-cc24-491f-8856-d47d8f22b1c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.270679 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd817f0c-cc24-491f-8856-d47d8f22b1c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:17.270982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd817f0c-cc24-491f-8856-d47d8f22b1c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:17.272389 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd817f0c-cc24-491f-8856-d47d8f22b1c8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:17.280213 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-4c87465a-efc0-45e5-8aee-fc24b77422aa None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:17.281446 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-4c87465a-efc0-45e5-8aee-fc24b77422aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:17.284940 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 369/740] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:17.285769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd817f0c-cc24-491f-8856-d47d8f22b1c8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:17.351746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 372/741] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 19:47:17.352647 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87465a-efc0-45e5-8aee-fc24b77422aa None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:17.399743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd817f0c-cc24-491f-8856-d47d8f22b1c8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:17.412382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:17.434713 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182b047-1419-4c2b-93b7-1f60af2dfae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:17.435505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182b047-1419-4c2b-93b7-1f60af2dfae7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.435792 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182b047-1419-4c2b-93b7-1f60af2dfae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.436118 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182b047-1419-4c2b-93b7-1f60af2dfae7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.438908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:17.446701 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182b047-1419-4c2b-93b7-1f60af2dfae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:17.448328 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182b047-1419-4c2b-93b7-1f60af2dfae7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:17.452329 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87465a-efc0-45e5-8aee-fc24b77422aa None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-7182b047-1419-4c2b-93b7-1f60af2dfae7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:17.467410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:17.468329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:17.469432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:17.469432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:17.469432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:17.469432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:17.469432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 370/742] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:17.470140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182b047-1419-4c2b-93b7-1f60af2dfae7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:17.491329 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db044da8-7c23-41f6-8f01-f030ca121161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:17.493398 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db044da8-7c23-41f6-8f01-f030ca121161 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.493398 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db044da8-7c23-41f6-8f01-f030ca121161 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.493398 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db044da8-7c23-41f6-8f01-f030ca121161 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.560041 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db044da8-7c23-41f6-8f01-f030ca121161 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:17.561530 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db044da8-7c23-41f6-8f01-f030ca121161 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:17.570242 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182b047-1419-4c2b-93b7-1f60af2dfae7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:17.605767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc8d44-6499-476c-aad8-8940b9fcecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:17.606381 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc8d44-6499-476c-aad8-8940b9fcecfc None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.606663 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc8d44-6499-476c-aad8-8940b9fcecfc None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.606937 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc8d44-6499-476c-aad8-8940b9fcecfc None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.607661 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc8d44-6499-476c-aad8-8940b9fcecfc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:17.609108 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc8d44-6499-476c-aad8-8940b9fcecfc None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:17.618220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:17.659534 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 373/743] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 19:47:17.660524 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db044da8-7c23-41f6-8f01-f030ca121161 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:17.707788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 371/744] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:47:17 2022] POST /identity/v3/services => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:17.708692 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc8d44-6499-476c-aad8-8940b9fcecfc None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:17.770658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:17.770867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db044da8-7c23-41f6-8f01-f030ca121161 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:17.822767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b3463-3037-476e-b89e-3af4af103110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:17.823587 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b3463-3037-476e-b89e-3af4af103110 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.823871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b3463-3037-476e-b89e-3af4af103110 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.824150 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b3463-3037-476e-b89e-3af4af103110 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.833547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b3463-3037-476e-b89e-3af4af103110 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:17.835018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b3463-3037-476e-b89e-3af4af103110 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:17.837164 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc8d44-6499-476c-aad8-8940b9fcecfc None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-c53b3463-3037-476e-b89e-3af4af103110 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:17.851077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:17.858031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 374/745] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:17.859197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b3463-3037-476e-b89e-3af4af103110 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:17.876025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:17.885903 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fecb15-6101-4b6f-a069-8fb8544b428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:17.886586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fecb15-6101-4b6f-a069-8fb8544b428c None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.886867 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fecb15-6101-4b6f-a069-8fb8544b428c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.887721 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fecb15-6101-4b6f-a069-8fb8544b428c None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:17.888614 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fecb15-6101-4b6f-a069-8fb8544b428c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:17.889964 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fecb15-6101-4b6f-a069-8fb8544b428c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:17.963195 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b3463-3037-476e-b89e-3af4af103110 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:17.998317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517703f-adc7-4529-bdd0-fb520b8b1f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:17.998978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517703f-adc7-4529-bdd0-fb520b8b1f22 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:17.999268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517703f-adc7-4529-bdd0-fb520b8b1f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:17.999541 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c517703f-adc7-4529-bdd0-fb520b8b1f22 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:18.000322 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517703f-adc7-4529-bdd0-fb520b8b1f22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:18.000667 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517703f-adc7-4529-bdd0-fb520b8b1f22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:18.002092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c517703f-adc7-4529-bdd0-fb520b8b1f22 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:18.004523 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 372/746] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:47:17 2022] POST /identity/v3/services => generated 240 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:18.005722 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fecb15-6101-4b6f-a069-8fb8544b428c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:18.023170 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 375/747] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:18.024489 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517703f-adc7-4529-bdd0-fb520b8b1f22 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:18.114325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:18.149751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c517703f-adc7-4529-bdd0-fb520b8b1f22 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:18.167513 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fecb15-6101-4b6f-a069-8fb8544b428c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:18.171820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:18.199462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a840040-f507-47b7-8824-87153d7066eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:18.200821 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a840040-f507-47b7-8824-87153d7066eb None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:18.201175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a840040-f507-47b7-8824-87153d7066eb None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:18.201462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a840040-f507-47b7-8824-87153d7066eb None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:18.202320 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a840040-f507-47b7-8824-87153d7066eb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:18.202632 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a840040-f507-47b7-8824-87153d7066eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:18.204076 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a840040-f507-47b7-8824-87153d7066eb None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:18.212179 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadc5cf-b1ab-4e78-9852-efd635233ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:18.213449 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadc5cf-b1ab-4e78-9852-efd635233ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:18.213947 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadc5cf-b1ab-4e78-9852-efd635233ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:18.214512 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eadc5cf-b1ab-4e78-9852-efd635233ca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:18.223240 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 376/748] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 19:47:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:18.224474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a840040-f507-47b7-8824-87153d7066eb None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:18.226632 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eadc5cf-b1ab-4e78-9852-efd635233ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:18.228402 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eadc5cf-b1ab-4e78-9852-efd635233ca7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-9eadc5cf-b1ab-4e78-9852-efd635233ca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:18.249307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:18.250237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:18.251128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:18.251128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:18.251128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:18.251128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:18.252788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 373/749] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:18.253817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadc5cf-b1ab-4e78-9852-efd635233ca7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:18.345136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a840040-f507-47b7-8824-87153d7066eb None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:18.379324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:18.385185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489af97b-8daa-483d-95d3-45ca3e152da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:18.386196 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af97b-8daa-483d-95d3-45ca3e152da2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:18.386335 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eadc5cf-b1ab-4e78-9852-efd635233ca7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:18.386775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af97b-8daa-483d-95d3-45ca3e152da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:18.387218 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af97b-8daa-483d-95d3-45ca3e152da2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:18.395074 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489af97b-8daa-483d-95d3-45ca3e152da2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:18.397084 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489af97b-8daa-483d-95d3-45ca3e152da2 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-489af97b-8daa-483d-95d3-45ca3e152da2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:18.412936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:18.414169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:18.415245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:18.415245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:18.415245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:18.415245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:18.415245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:18.415245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:18.415245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:18.415245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:18.415245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:47:18.415245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:18.417813 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 377/750] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 19:47:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:18.423094 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489af97b-8daa-483d-95d3-45ca3e152da2 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:18.428596 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd524c56-d37b-4faf-920d-7ad3934b3623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:18.429357 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd524c56-d37b-4faf-920d-7ad3934b3623 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:18.429694 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd524c56-d37b-4faf-920d-7ad3934b3623 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:18.429974 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd524c56-d37b-4faf-920d-7ad3934b3623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:18.464809 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:18.524233 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd524c56-d37b-4faf-920d-7ad3934b3623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:18.525734 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd524c56-d37b-4faf-920d-7ad3934b3623 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:18.540247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489af97b-8daa-483d-95d3-45ca3e152da2 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:18.582042 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272bc23-6430-454d-814f-1a97bef5b7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:18.582827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272bc23-6430-454d-814f-1a97bef5b7dc None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:18.583015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272bc23-6430-454d-814f-1a97bef5b7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:18.583627 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272bc23-6430-454d-814f-1a97bef5b7dc None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:18.584466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272bc23-6430-454d-814f-1a97bef5b7dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:18.584854 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272bc23-6430-454d-814f-1a97bef5b7dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:18.586805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272bc23-6430-454d-814f-1a97bef5b7dc None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:18.601691 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 378/751] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:18.602699 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272bc23-6430-454d-814f-1a97bef5b7dc None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:18.664453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 374/752] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:47:18.664453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd524c56-d37b-4faf-920d-7ad3934b3623 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:18.680621 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:18.730572 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:18.745949 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272bc23-6430-454d-814f-1a97bef5b7dc None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:18.776522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd524c56-d37b-4faf-920d-7ad3934b3623 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:18.787337 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416cc3-0070-4867-8b5e-3a6c270627ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:18.790490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416cc3-0070-4867-8b5e-3a6c270627ce None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:18.790998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416cc3-0070-4867-8b5e-3a6c270627ce None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:18.791476 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2416cc3-0070-4867-8b5e-3a6c270627ce None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:18.792692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2416cc3-0070-4867-8b5e-3a6c270627ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:18.793257 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2416cc3-0070-4867-8b5e-3a6c270627ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:18.795536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2416cc3-0070-4867-8b5e-3a6c270627ce None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:18.815681 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 379/753] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 19:47:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:18.821364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416cc3-0070-4867-8b5e-3a6c270627ce None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:18.821987 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2ed7-fa18-47a9-9806-2cba1c3620b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:18.822846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2ed7-fa18-47a9-9806-2cba1c3620b4 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:18.823131 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2ed7-fa18-47a9-9806-2cba1c3620b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:18.823462 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2ed7-fa18-47a9-9806-2cba1c3620b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:18.832775 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c2ed7-fa18-47a9-9806-2cba1c3620b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:18.834788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c2ed7-fa18-47a9-9806-2cba1c3620b4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-ce8c2ed7-fa18-47a9-9806-2cba1c3620b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:18.858571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:18.859559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:18.860979 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:18.860979 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:18.860979 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:18.860979 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:18.860979 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:18.860979 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:18.860979 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:18.860979 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:18.860979 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 375/754] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:18.861491 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2ed7-fa18-47a9-9806-2cba1c3620b4 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:18.958404 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2416cc3-0070-4867-8b5e-3a6c270627ce None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:18.972892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:19.003251 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2ed7-fa18-47a9-9806-2cba1c3620b4 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:19.005612 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da50432-f2e8-4204-8d67-4a1fe9c3c877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:19.007629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50432-f2e8-4204-8d67-4a1fe9c3c877 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:19.008205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50432-f2e8-4204-8d67-4a1fe9c3c877 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:19.008768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50432-f2e8-4204-8d67-4a1fe9c3c877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:19.028399 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da50432-f2e8-4204-8d67-4a1fe9c3c877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:19.028625 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da50432-f2e8-4204-8d67-4a1fe9c3c877 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:19.028984 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-8da50432-f2e8-4204-8d67-4a1fe9c3c877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:19.048027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:19.049258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:19.050334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:19.050334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:19.050334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:19.050334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:19.050334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 380/755] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:19.051001 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da50432-f2e8-4204-8d67-4a1fe9c3c877 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:19.052911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee5db3-c8fd-44c0-b971-9c497137bea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:19.054016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee5db3-c8fd-44c0-b971-9c497137bea0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:19.054607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee5db3-c8fd-44c0-b971-9c497137bea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:19.055110 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee5db3-c8fd-44c0-b971-9c497137bea0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:19.066764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee5db3-c8fd-44c0-b971-9c497137bea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:19.068736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee5db3-c8fd-44c0-b971-9c497137bea0 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-f4ee5db3-c8fd-44c0-b971-9c497137bea0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:19.088170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:19.089140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:19.090028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:19.090028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:19.090028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:19.090028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:19.092873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 376/756] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:19.093873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee5db3-c8fd-44c0-b971-9c497137bea0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:19.167700 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da50432-f2e8-4204-8d67-4a1fe9c3c877 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:19.197589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee5db3-c8fd-44c0-b971-9c497137bea0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:19.205634 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750d6fd-a608-4ca1-9543-0eae14577dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:19.206302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750d6fd-a608-4ca1-9543-0eae14577dba None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:19.206590 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750d6fd-a608-4ca1-9543-0eae14577dba None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:19.206869 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750d6fd-a608-4ca1-9543-0eae14577dba None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:19.207844 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750d6fd-a608-4ca1-9543-0eae14577dba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:19.208160 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750d6fd-a608-4ca1-9543-0eae14577dba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:19.209729 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b750d6fd-a608-4ca1-9543-0eae14577dba None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:19.228047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 381/757] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:19.228047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750d6fd-a608-4ca1-9543-0eae14577dba None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:19.244020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeec161-b960-47c2-8dac-4adb4f1a9188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:19.245109 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeec161-b960-47c2-8dac-4adb4f1a9188 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:19.245596 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeec161-b960-47c2-8dac-4adb4f1a9188 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:19.246082 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeec161-b960-47c2-8dac-4adb4f1a9188 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:19.247066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeec161-b960-47c2-8dac-4adb4f1a9188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:19.247580 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeec161-b960-47c2-8dac-4adb4f1a9188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:19.249216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeec161-b960-47c2-8dac-4adb4f1a9188 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:19.252570 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:19.261748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 377/758] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:19.270656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeec161-b960-47c2-8dac-4adb4f1a9188 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:19.295985 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:19.350474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b750d6fd-a608-4ca1-9543-0eae14577dba None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:19.376961 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeec161-b960-47c2-8dac-4adb4f1a9188 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:19.386023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec178ed-e1ad-4f08-bf1e-c9d68ed9c2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:19.386909 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec178ed-e1ad-4f08-bf1e-c9d68ed9c2ca None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:19.387893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec178ed-e1ad-4f08-bf1e-c9d68ed9c2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:19.388427 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec178ed-e1ad-4f08-bf1e-c9d68ed9c2ca None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:19.389377 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec178ed-e1ad-4f08-bf1e-c9d68ed9c2ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:19.389886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec178ed-e1ad-4f08-bf1e-c9d68ed9c2ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:19.391542 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec178ed-e1ad-4f08-bf1e-c9d68ed9c2ca None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:19.405397 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 382/759] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:19.406537 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec178ed-e1ad-4f08-bf1e-c9d68ed9c2ca None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:19.416078 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ded0f9-d019-4def-aea5-e834cca8277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:19.416779 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ded0f9-d019-4def-aea5-e834cca8277e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:19.417067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ded0f9-d019-4def-aea5-e834cca8277e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:19.417349 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ded0f9-d019-4def-aea5-e834cca8277e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:19.418192 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ded0f9-d019-4def-aea5-e834cca8277e None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=f4b549638f5442b6a1be558df1686c5d) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:19.418861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ded0f9-d019-4def-aea5-e834cca8277e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:19.420090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ded0f9-d019-4def-aea5-e834cca8277e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:19.431198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 378/760] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:19 2022] GET /identity/v3/role_assignments?user.id=f4b549638f5442b6a1be558df1686c5d&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:19.431198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ded0f9-d019-4def-aea5-e834cca8277e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:19.537634 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:19.542776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:19.549123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ded0f9-d019-4def-aea5-e834cca8277e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:19.580524 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec178ed-e1ad-4f08-bf1e-c9d68ed9c2ca None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:19.587244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412854b-fa1f-4435-b1b4-46cb3dffd09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:19.588246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412854b-fa1f-4435-b1b4-46cb3dffd09e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:19.588870 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412854b-fa1f-4435-b1b4-46cb3dffd09e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:19.589291 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412854b-fa1f-4435-b1b4-46cb3dffd09e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:19.597283 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412854b-fa1f-4435-b1b4-46cb3dffd09e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:19.599356 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412854b-fa1f-4435-b1b4-46cb3dffd09e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-a412854b-fa1f-4435-b1b4-46cb3dffd09e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:19.615785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:19.616988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:19.617932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:19.617932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:19.617932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:19.617932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:19.617932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:19.617932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:19.617932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:19.617932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:19.620430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 379/761] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:19.629041 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412854b-fa1f-4435-b1b4-46cb3dffd09e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:19.653201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78867a8b-3068-403a-81a9-06349dcd0797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:19.654011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78867a8b-3068-403a-81a9-06349dcd0797 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:19.654293 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78867a8b-3068-403a-81a9-06349dcd0797 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:19.654562 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78867a8b-3068-403a-81a9-06349dcd0797 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:19.668404 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78867a8b-3068-403a-81a9-06349dcd0797 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:19.669887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78867a8b-3068-403a-81a9-06349dcd0797 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-78867a8b-3068-403a-81a9-06349dcd0797 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:19.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:19.700560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:19.701462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:19.701462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:19.701462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:19.701462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:19.701462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:19.701462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:19.701462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:19.701462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:19.709479 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 383/762] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:19.712198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78867a8b-3068-403a-81a9-06349dcd0797 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:19.795930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412854b-fa1f-4435-b1b4-46cb3dffd09e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:19.842881 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca67b4-6476-426b-adc3-fcf34eba58f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:19.843562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca67b4-6476-426b-adc3-fcf34eba58f0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:19.843843 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca67b4-6476-426b-adc3-fcf34eba58f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:19.844124 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca67b4-6476-426b-adc3-fcf34eba58f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:19.848326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca67b4-6476-426b-adc3-fcf34eba58f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:19.848666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca67b4-6476-426b-adc3-fcf34eba58f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:19.850136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca67b4-6476-426b-adc3-fcf34eba58f0 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:19.880314 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 380/763] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:19.886169 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca67b4-6476-426b-adc3-fcf34eba58f0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:19.910763 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78867a8b-3068-403a-81a9-06349dcd0797 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:19.915138 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:19.990087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47adb52-f304-4b1d-9dc1-c2aa1e3c09e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:19.990757 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47adb52-f304-4b1d-9dc1-c2aa1e3c09e6 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:19.991043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47adb52-f304-4b1d-9dc1-c2aa1e3c09e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:19.991321 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47adb52-f304-4b1d-9dc1-c2aa1e3c09e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:19.992119 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47adb52-f304-4b1d-9dc1-c2aa1e3c09e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:19.992449 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47adb52-f304-4b1d-9dc1-c2aa1e3c09e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:19.994817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47adb52-f304-4b1d-9dc1-c2aa1e3c09e6 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:20.019957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 384/764] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:20.023650 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47adb52-f304-4b1d-9dc1-c2aa1e3c09e6 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:20.026752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:20.093866 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca67b4-6476-426b-adc3-fcf34eba58f0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:20.151179 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c415b-9820-4792-ba3f-dbd2d5d12daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:20.152080 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c415b-9820-4792-ba3f-dbd2d5d12daa None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:20.152385 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c415b-9820-4792-ba3f-dbd2d5d12daa None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:20.152684 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c415b-9820-4792-ba3f-dbd2d5d12daa None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/0c0e71329c4a4470935593636c025257/roles/815bd48c3df0405babfd5b53430fd3ea` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:20.159990 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c415b-9820-4792-ba3f-dbd2d5d12daa None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=0c0e71329c4a4470935593636c025257, role_id=815bd48c3df0405babfd5b53430fd3ea)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:20.160917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c415b-9820-4792-ba3f-dbd2d5d12daa None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:20.180375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47adb52-f304-4b1d-9dc1-c2aa1e3c09e6 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:20.217532 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415ffe1d-bd66-44b4-8620-38e4255f1e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:20.218335 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ffe1d-bd66-44b4-8620-38e4255f1e37 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:20.218560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ffe1d-bd66-44b4-8620-38e4255f1e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:20.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415ffe1d-bd66-44b4-8620-38e4255f1e37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:20.219763 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ffe1d-bd66-44b4-8620-38e4255f1e37 None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=c8a36bb1e26f4faaa44fabff8b509fe7) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:20.220080 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ffe1d-bd66-44b4-8620-38e4255f1e37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:20.221786 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415ffe1d-bd66-44b4-8620-38e4255f1e37 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:20.232654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 385/765] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:20 2022] GET /identity/v3/role_assignments?user.id=c8a36bb1e26f4faaa44fabff8b509fe7&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:20.232654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9643be-7926-4ee5-9a16-62a6cd4fa26e None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:20.232654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9643be-7926-4ee5-9a16-62a6cd4fa26e None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:20.232654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9643be-7926-4ee5-9a16-62a6cd4fa26e None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:20.232654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 386/766] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:20.242762 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd48b21-d6c8-4cb6-9aeb-f04357b15eef None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:20.243110 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd48b21-d6c8-4cb6-9aeb-f04357b15eef None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:20.243393 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd48b21-d6c8-4cb6-9aeb-f04357b15eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:20.264072 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-3cd48b21-d6c8-4cb6-9aeb-f04357b15eef None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:20.268542 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-3cd48b21-d6c8-4cb6-9aeb-f04357b15eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:20.271245 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 381/767] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:47:19 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/0c0e71329c4a4470935593636c025257/roles/815bd48c3df0405babfd5b53430fd3ea => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:47:20.347905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 387/768] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:20.348526 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:20.379817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c415b-9820-4792-ba3f-dbd2d5d12daa None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:20.493624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:20.511552 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6bba1-4813-47d3-9ddf-c8156fc06350 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:20.511877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6bba1-4813-47d3-9ddf-c8156fc06350 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:20.512750 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6bba1-4813-47d3-9ddf-c8156fc06350 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:20.514284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 388/769] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:20.519998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395b622-12c5-4c35-a1fa-53dc1375b9da None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:20.520284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395b622-12c5-4c35-a1fa-53dc1375b9da None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:20.520613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395b622-12c5-4c35-a1fa-53dc1375b9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:20.537287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c415b-9820-4792-ba3f-dbd2d5d12daa None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:20.547852 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-8395b622-12c5-4c35-a1fa-53dc1375b9da None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:20.549137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-8395b622-12c5-4c35-a1fa-53dc1375b9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:20.577712 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641d808-07ac-4904-9132-09309807c544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:20.579212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641d808-07ac-4904-9132-09309807c544 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:20.579797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641d808-07ac-4904-9132-09309807c544 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:20.580272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641d808-07ac-4904-9132-09309807c544 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:20.581247 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641d808-07ac-4904-9132-09309807c544 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:20.582968 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641d808-07ac-4904-9132-09309807c544 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-5641d808-07ac-4904-9132-09309807c544 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:20.596876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:20.597825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:47:20.598655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:20.601624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 382/770] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 19:47:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:20.609236 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641d808-07ac-4904-9132-09309807c544 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:20.631831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 389/771] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:20.662635 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395b622-12c5-4c35-a1fa-53dc1375b9da None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:20.785020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641d808-07ac-4904-9132-09309807c544 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:20.847659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395b622-12c5-4c35-a1fa-53dc1375b9da None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:20.858677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd80c13-7acd-4089-b436-2ea113ec89c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:20.859347 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd80c13-7acd-4089-b436-2ea113ec89c4 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:20.859627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd80c13-7acd-4089-b436-2ea113ec89c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:20.859952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd80c13-7acd-4089-b436-2ea113ec89c4 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:20.860794 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd80c13-7acd-4089-b436-2ea113ec89c4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:20.861101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd80c13-7acd-4089-b436-2ea113ec89c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:20.862494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd80c13-7acd-4089-b436-2ea113ec89c4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:20.869555 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:20.884133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 383/772] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 19:47:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:20.884320 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:20.886264 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd80c13-7acd-4089-b436-2ea113ec89c4 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:20.893383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1f00c-bf20-491f-9604-7d705149dd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:20.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1f00c-bf20-491f-9604-7d705149dd52 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:20.894727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1f00c-bf20-491f-9604-7d705149dd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:20.895262 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1f00c-bf20-491f-9604-7d705149dd52 None admin] PATH_INFO: `/v3/services/network` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:20.897024 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1f00c-bf20-491f-9604-7d705149dd52 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:20.898501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1f00c-bf20-491f-9604-7d705149dd52 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-5fd1f00c-bf20-491f-9604-7d705149dd52 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:20.913775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:20.914812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:20.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 390/773] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:20 2022] GET /identity/v3/services/network => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:20.922064 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1f00c-bf20-491f-9604-7d705149dd52 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:21.064120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd80c13-7acd-4089-b436-2ea113ec89c4 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:21.113481 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f0b04-aa33-43ec-beee-c62734c2f76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:21.114070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f0b04-aa33-43ec-beee-c62734c2f76c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:21.114354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f0b04-aa33-43ec-beee-c62734c2f76c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:21.114625 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f0b04-aa33-43ec-beee-c62734c2f76c None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:21.115335 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f0b04-aa33-43ec-beee-c62734c2f76c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:21.115638 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f0b04-aa33-43ec-beee-c62734c2f76c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:21.116002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1f00c-bf20-491f-9604-7d705149dd52 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:21.117056 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f0b04-aa33-43ec-beee-c62734c2f76c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:21.128405 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 384/774] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 19:47:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:21.130355 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f0b04-aa33-43ec-beee-c62734c2f76c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:21.202596 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:21.218069 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbe8d1-9d7a-4484-a05c-392c4f5aa5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:21.218810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbe8d1-9d7a-4484-a05c-392c4f5aa5dc None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:21.219164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbe8d1-9d7a-4484-a05c-392c4f5aa5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:21.219512 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbe8d1-9d7a-4484-a05c-392c4f5aa5dc None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:21.220395 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbe8d1-9d7a-4484-a05c-392c4f5aa5dc None admin] RBAC: Adding query filter params (name=network) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:21.220771 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbe8d1-9d7a-4484-a05c-392c4f5aa5dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:21.222218 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbe8d1-9d7a-4484-a05c-392c4f5aa5dc None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:21.242674 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 391/775] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:21.250871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f0b04-aa33-43ec-beee-c62734c2f76c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:21.263144 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbe8d1-9d7a-4484-a05c-392c4f5aa5dc None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:21.291977 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17aea1c-c008-4de2-901e-c199538119db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:21.293989 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17aea1c-c008-4de2-901e-c199538119db None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:21.294457 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17aea1c-c008-4de2-901e-c199538119db None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:21.295195 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17aea1c-c008-4de2-901e-c199538119db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:21.296409 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17aea1c-c008-4de2-901e-c199538119db None admin] RBAC: Adding query filter params (interface=public, service_id=b470e71f52b841ebbdc3b939612967b9, region_id=RegionOne) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:21.297225 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17aea1c-c008-4de2-901e-c199538119db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:21.299268 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17aea1c-c008-4de2-901e-c199538119db None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:21.310890 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 385/776] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 19:47:21 2022] GET /identity/v3/endpoints?service_id=b470e71f52b841ebbdc3b939612967b9&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:21.317051 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17aea1c-c008-4de2-901e-c199538119db None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:21.384790 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:21.441798 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17aea1c-c008-4de2-901e-c199538119db None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:21.472447 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:21.482662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286dac5-5514-4f37-a51f-71cddd0326c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:21.483710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286dac5-5514-4f37-a51f-71cddd0326c7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:21.484182 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286dac5-5514-4f37-a51f-71cddd0326c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:21.485333 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286dac5-5514-4f37-a51f-71cddd0326c7 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:21.486462 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286dac5-5514-4f37-a51f-71cddd0326c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:21.488246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c286dac5-5514-4f37-a51f-71cddd0326c7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:21.495572 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 386/777] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 19:47:21 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:21.522811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbe8d1-9d7a-4484-a05c-392c4f5aa5dc None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:21.591335 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caea19f-e0cf-48b1-91cc-07b1d43070a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:21.592034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caea19f-e0cf-48b1-91cc-07b1d43070a6 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:21.592315 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caea19f-e0cf-48b1-91cc-07b1d43070a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:21.592608 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caea19f-e0cf-48b1-91cc-07b1d43070a6 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:21.593339 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caea19f-e0cf-48b1-91cc-07b1d43070a6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:21.593643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caea19f-e0cf-48b1-91cc-07b1d43070a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:21.595046 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caea19f-e0cf-48b1-91cc-07b1d43070a6 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:21.605326 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 392/778] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:21.611590 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286dac5-5514-4f37-a51f-71cddd0326c7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:21.734596 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286dac5-5514-4f37-a51f-71cddd0326c7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:21.770445 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26575f92-3bbe-4689-aca0-9843c25138ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:21.771584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26575f92-3bbe-4689-aca0-9843c25138ce None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:21.772075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26575f92-3bbe-4689-aca0-9843c25138ce None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:21.772577 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26575f92-3bbe-4689-aca0-9843c25138ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:21.773574 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26575f92-3bbe-4689-aca0-9843c25138ce None admin] RBAC: Adding query filter params (interface=public, service_id=8401182b2a6e458698be2f10da7e84aa, region_id=RegionOne) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:21.774099 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26575f92-3bbe-4689-aca0-9843c25138ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:21.775672 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26575f92-3bbe-4689-aca0-9843c25138ce None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:21.783333 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 387/779] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 19:47:21 2022] GET /identity/v3/endpoints?service_id=8401182b2a6e458698be2f10da7e84aa&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:21.788408 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caea19f-e0cf-48b1-91cc-07b1d43070a6 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:21.935308 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caea19f-e0cf-48b1-91cc-07b1d43070a6 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:21.958829 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:21.986534 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5b838-b661-4439-b27e-3c7d58e8efd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:21.987128 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5b838-b661-4439-b27e-3c7d58e8efd2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:21.987415 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5b838-b661-4439-b27e-3c7d58e8efd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:21.987696 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5b838-b661-4439-b27e-3c7d58e8efd2 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:21.988416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5b838-b661-4439-b27e-3c7d58e8efd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:21.989790 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5b838-b661-4439-b27e-3c7d58e8efd2 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:21.999199 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 393/780] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 19:47:21 2022] GET /identity/v3/services => generated 772 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:22.399384 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfde72c-6b04-4388-a92a-46d6917e4482 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:22.400038 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfde72c-6b04-4388-a92a-46d6917e4482 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:22.400564 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfde72c-6b04-4388-a92a-46d6917e4482 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:22.402142 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 388/781] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:22.409200 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15712e1-eeae-4c27-9c3d-4e64ff54a70f None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:22.409492 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15712e1-eeae-4c27-9c3d-4e64ff54a70f None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:22.409776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15712e1-eeae-4c27-9c3d-4e64ff54a70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:22.429011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-a15712e1-eeae-4c27-9c3d-4e64ff54a70f None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:22.430206 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-a15712e1-eeae-4c27-9c3d-4e64ff54a70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:22.505177 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 394/782] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:22.529783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfde72c-6b04-4388-a92a-46d6917e4482 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:22.609903 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:22.674061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfde72c-6b04-4388-a92a-46d6917e4482 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:22.690244 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910887c-903b-4dca-8945-375bd032a4c9 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:22.690540 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910887c-903b-4dca-8945-375bd032a4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:22.690811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910887c-903b-4dca-8945-375bd032a4c9 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:22.692471 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 395/783] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:22.701807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da49e72-6f69-4888-8617-75091018efb3 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:22.702380 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da49e72-6f69-4888-8617-75091018efb3 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:22.702664 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da49e72-6f69-4888-8617-75091018efb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:22.722793 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-0da49e72-6f69-4888-8617-75091018efb3 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:22.724012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-0da49e72-6f69-4888-8617-75091018efb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:22.740672 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69853779-bbd7-42fa-95c8-7af2057f31d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:22.741470 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69853779-bbd7-42fa-95c8-7af2057f31d0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:22.742816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69853779-bbd7-42fa-95c8-7af2057f31d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:22.742937 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69853779-bbd7-42fa-95c8-7af2057f31d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:22.766210 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69853779-bbd7-42fa-95c8-7af2057f31d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-69853779-bbd7-42fa-95c8-7af2057f31d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:22.783388 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:22.784444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:22.785296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:22.785928 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 389/784] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:47:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:22.795880 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69853779-bbd7-42fa-95c8-7af2057f31d0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:22.854012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 396/785] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:22.888726 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da49e72-6f69-4888-8617-75091018efb3 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:22.960886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:22.975379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69853779-bbd7-42fa-95c8-7af2057f31d0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:22.998577 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:23.020561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a3b27-2a0b-4b73-8712-dd0d6a5850eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:23.021228 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3b27-2a0b-4b73-8712-dd0d6a5850eb None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.021513 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3b27-2a0b-4b73-8712-dd0d6a5850eb None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.021931 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3b27-2a0b-4b73-8712-dd0d6a5850eb None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.022937 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a3b27-2a0b-4b73-8712-dd0d6a5850eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:23.023303 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a3b27-2a0b-4b73-8712-dd0d6a5850eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:23.024918 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a3b27-2a0b-4b73-8712-dd0d6a5850eb None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:23.026726 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da49e72-6f69-4888-8617-75091018efb3 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:23.037018 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 390/786] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:47:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:23.038546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ef78-bd97-437f-a61e-da552658bba5 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.038992 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ef78-bd97-437f-a61e-da552658bba5 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.039335 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc7ef78-bd97-437f-a61e-da552658bba5 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.040933 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 391/787] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:23.046395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7ef78-bd97-437f-a61e-da552658bba5 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:23.070516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095fdc0c-be3c-4d97-92af-611820c4d021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:23.071894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fdc0c-be3c-4d97-92af-611820c4d021 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.072182 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fdc0c-be3c-4d97-92af-611820c4d021 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.072505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095fdc0c-be3c-4d97-92af-611820c4d021 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.088456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095fdc0c-be3c-4d97-92af-611820c4d021 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-095fdc0c-be3c-4d97-92af-611820c4d021 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:23.098745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:23.099985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 397/788] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:23.101815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbcbab-f34f-4e69-8194-da0f7fa2a3bf None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.102779 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbcbab-f34f-4e69-8194-da0f7fa2a3bf None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.102779 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbcbab-f34f-4e69-8194-da0f7fa2a3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.134970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-c6fbcbab-f34f-4e69-8194-da0f7fa2a3bf None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:23.137908 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-c6fbcbab-f34f-4e69-8194-da0f7fa2a3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:23.172046 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc7ef78-bd97-437f-a61e-da552658bba5 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:23.220982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0aae4-3e23-4b64-a2d8-7d668d5e27a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:23.221776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0aae4-3e23-4b64-a2d8-7d668d5e27a9 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.222061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0aae4-3e23-4b64-a2d8-7d668d5e27a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.222338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0aae4-3e23-4b64-a2d8-7d668d5e27a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.224362 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 398/789] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:23.225397 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbcbab-f34f-4e69-8194-da0f7fa2a3bf None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:23.234522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0aae4-3e23-4b64-a2d8-7d668d5e27a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:23.235887 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c0aae4-3e23-4b64-a2d8-7d668d5e27a9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-79c0aae4-3e23-4b64-a2d8-7d668d5e27a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:23.257666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:23.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:23.261359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:23.261359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:23.261359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:23.261359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:23.261359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 392/790] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:23.261729 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0aae4-3e23-4b64-a2d8-7d668d5e27a9 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:23.277886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:23.308361 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:23.372837 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbcbab-f34f-4e69-8194-da0f7fa2a3bf None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:23.399003 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c0aae4-3e23-4b64-a2d8-7d668d5e27a9 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:23.420115 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444c3c4-c27f-4c0e-b384-2cbaa79c7694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:23.420820 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444c3c4-c27f-4c0e-b384-2cbaa79c7694 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.421103 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444c3c4-c27f-4c0e-b384-2cbaa79c7694 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.421380 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b444c3c4-c27f-4c0e-b384-2cbaa79c7694 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.422179 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444c3c4-c27f-4c0e-b384-2cbaa79c7694 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:23.422427 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444c3c4-c27f-4c0e-b384-2cbaa79c7694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:23.424545 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b444c3c4-c27f-4c0e-b384-2cbaa79c7694 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:23.437571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 399/791] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:47:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:23.438320 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edeb98-0df3-4509-821f-3fb560278d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:23.439312 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444c3c4-c27f-4c0e-b384-2cbaa79c7694 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:23.440174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edeb98-0df3-4509-821f-3fb560278d77 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.440743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edeb98-0df3-4509-821f-3fb560278d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.441246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edeb98-0df3-4509-821f-3fb560278d77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.453970 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edeb98-0df3-4509-821f-3fb560278d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-b6edeb98-0df3-4509-821f-3fb560278d77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:23.462100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:23.463066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:23.463910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:23.466645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 393/792] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:23.467644 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edeb98-0df3-4509-821f-3fb560278d77 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:23.551744 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:23.559560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b444c3c4-c27f-4c0e-b384-2cbaa79c7694 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:23.590552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:23.596832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6155e-c1e0-4886-8473-c9900a7dea52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:23.597786 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6155e-c1e0-4886-8473-c9900a7dea52 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.598420 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6155e-c1e0-4886-8473-c9900a7dea52 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.598914 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6155e-c1e0-4886-8473-c9900a7dea52 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.599820 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6155e-c1e0-4886-8473-c9900a7dea52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:23.600298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6155e-c1e0-4886-8473-c9900a7dea52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:23.601680 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edeb98-0df3-4509-821f-3fb560278d77 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:23.602363 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6155e-c1e0-4886-8473-c9900a7dea52 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:23.618250 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 400/793] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:23.619107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6155e-c1e0-4886-8473-c9900a7dea52 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:23.641453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce685cf-a0c8-4a2e-ae5c-db19ac3fc5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:23.642726 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce685cf-a0c8-4a2e-ae5c-db19ac3fc5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.643260 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce685cf-a0c8-4a2e-ae5c-db19ac3fc5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.643766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce685cf-a0c8-4a2e-ae5c-db19ac3fc5d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.655706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce685cf-a0c8-4a2e-ae5c-db19ac3fc5d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:23.657606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce685cf-a0c8-4a2e-ae5c-db19ac3fc5d9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-0ce685cf-a0c8-4a2e-ae5c-db19ac3fc5d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:23.677848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:23.679184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:23.680054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:23.680054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:23.680054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:23.680054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:23.682884 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 394/794] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:23.684605 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce685cf-a0c8-4a2e-ae5c-db19ac3fc5d9 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:23.744774 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6155e-c1e0-4886-8473-c9900a7dea52 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:23.780747 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1914e89-ce1e-47de-9e79-85eedb68da4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:23.781419 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914e89-ce1e-47de-9e79-85eedb68da4b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.781702 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914e89-ce1e-47de-9e79-85eedb68da4b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.781975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1914e89-ce1e-47de-9e79-85eedb68da4b None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.782719 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1914e89-ce1e-47de-9e79-85eedb68da4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:23.783019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1914e89-ce1e-47de-9e79-85eedb68da4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:23.784593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1914e89-ce1e-47de-9e79-85eedb68da4b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:23.798390 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 401/795] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:47:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:23.799393 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1914e89-ce1e-47de-9e79-85eedb68da4b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:23.820595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:23.827386 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce685cf-a0c8-4a2e-ae5c-db19ac3fc5d9 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:23.878469 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2fa44-2b94-4287-a514-f3b0b20a2f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:23.879577 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2fa44-2b94-4287-a514-f3b0b20a2f42 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.880010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2fa44-2b94-4287-a514-f3b0b20a2f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.880454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2fa44-2b94-4287-a514-f3b0b20a2f42 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.890726 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2fa44-2b94-4287-a514-f3b0b20a2f42 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:23.892804 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2fa44-2b94-4287-a514-f3b0b20a2f42 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:23.909184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1914e89-ce1e-47de-9e79-85eedb68da4b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-1af2fa44-2b94-4287-a514-f3b0b20a2f42 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:23.914502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:23.915636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:23.916800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:23.916800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:23.916800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:23.916800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:23.916800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:23.916800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:23.916800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:23.916800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:23.916800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:23.916800 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:23.919635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 395/796] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 19:47:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:23.921142 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2fa44-2b94-4287-a514-f3b0b20a2f42 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:23.924875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:23.954428 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dea8f1-ef57-47f0-99ec-db615d9717ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:23.955281 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dea8f1-ef57-47f0-99ec-db615d9717ec None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:23.955604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dea8f1-ef57-47f0-99ec-db615d9717ec None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:23.956005 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dea8f1-ef57-47f0-99ec-db615d9717ec None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:23.957778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dea8f1-ef57-47f0-99ec-db615d9717ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:23.958485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dea8f1-ef57-47f0-99ec-db615d9717ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:23.960007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dea8f1-ef57-47f0-99ec-db615d9717ec None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:23.974132 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 402/797] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:23.975182 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dea8f1-ef57-47f0-99ec-db615d9717ec None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:24.059350 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2fa44-2b94-4287-a514-f3b0b20a2f42 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:24.063879 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:24.088136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dea8f1-ef57-47f0-99ec-db615d9717ec None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:24.108260 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb24f8-2c7c-49f0-94b3-5f38737982f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:24.109732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb24f8-2c7c-49f0-94b3-5f38737982f6 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.110204 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb24f8-2c7c-49f0-94b3-5f38737982f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.110628 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb24f8-2c7c-49f0-94b3-5f38737982f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.123577 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb24f8-2c7c-49f0-94b3-5f38737982f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:24.125167 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb24f8-2c7c-49f0-94b3-5f38737982f6 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:24.132775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbce068-9835-4d02-8813-a67993c9f656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:24.133461 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbce068-9835-4d02-8813-a67993c9f656 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.133743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbce068-9835-4d02-8813-a67993c9f656 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.134397 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbce068-9835-4d02-8813-a67993c9f656 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.135339 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbce068-9835-4d02-8813-a67993c9f656 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:24.135558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbce068-9835-4d02-8813-a67993c9f656 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:24.137153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbce068-9835-4d02-8813-a67993c9f656 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-9aeb24f8-2c7c-49f0-94b3-5f38737982f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:24.143293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:24.144188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:24.145173 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:24.145173 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:24.145173 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:24.145173 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:24.149246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 396/798] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:24.149981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb24f8-2c7c-49f0-94b3-5f38737982f6 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:24.160784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 403/799] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 19:47:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:24.160784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b2273-7db2-470b-a6c8-f2e878c6bbeb None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.160784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b2273-7db2-470b-a6c8-f2e878c6bbeb None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.160784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b2273-7db2-470b-a6c8-f2e878c6bbeb None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.161247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 404/800] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:24.161854 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b2273-7db2-470b-a6c8-f2e878c6bbeb None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:24.209697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:24.259250 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb24f8-2c7c-49f0-94b3-5f38737982f6 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:24.285327 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b2273-7db2-470b-a6c8-f2e878c6bbeb None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:24.300256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627613c-1970-4a3f-99fd-e3c448775a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:24.301457 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627613c-1970-4a3f-99fd-e3c448775a4e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.301739 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627613c-1970-4a3f-99fd-e3c448775a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.302020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627613c-1970-4a3f-99fd-e3c448775a4e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.309938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627613c-1970-4a3f-99fd-e3c448775a4e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:24.311536 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627613c-1970-4a3f-99fd-e3c448775a4e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:24.323660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0051cf3-6881-4c2a-b16c-26aae81f8b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:24.324883 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0051cf3-6881-4c2a-b16c-26aae81f8b3a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.325162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0051cf3-6881-4c2a-b16c-26aae81f8b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.325432 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0051cf3-6881-4c2a-b16c-26aae81f8b3a None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.326156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0051cf3-6881-4c2a-b16c-26aae81f8b3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:24.326456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0051cf3-6881-4c2a-b16c-26aae81f8b3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:24.327804 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0051cf3-6881-4c2a-b16c-26aae81f8b3a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-7627613c-1970-4a3f-99fd-e3c448775a4e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:24.328668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:24.329561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:24.330495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:24.330495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:24.330495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:24.330495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:24.330495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:24.330495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:24.330495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:24.330495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:24.330495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:47:24.330495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:24.331051 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:24.331124 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 397/801] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:24.331664 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627613c-1970-4a3f-99fd-e3c448775a4e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:24.354055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 405/802] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:24.355645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d72e0-30d4-44b3-b10b-d314ba41cf6d None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.355934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d72e0-30d4-44b3-b10b-d314ba41cf6d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.356212 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878d72e0-30d4-44b3-b10b-d314ba41cf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.374018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-878d72e0-30d4-44b3-b10b-d314ba41cf6d None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:24.376219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-878d72e0-30d4-44b3-b10b-d314ba41cf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:24.452349 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627613c-1970-4a3f-99fd-e3c448775a4e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:24.464465 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 406/803] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:24.464796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d72e0-30d4-44b3-b10b-d314ba41cf6d None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:24.472521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:24.497398 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c52f55-7ae1-4dc2-aeef-a3258a47bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:24.498505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c52f55-7ae1-4dc2-aeef-a3258a47bf9f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.498982 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c52f55-7ae1-4dc2-aeef-a3258a47bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.499446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c52f55-7ae1-4dc2-aeef-a3258a47bf9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.511107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c52f55-7ae1-4dc2-aeef-a3258a47bf9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:24.513163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c52f55-7ae1-4dc2-aeef-a3258a47bf9f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-a4c52f55-7ae1-4dc2-aeef-a3258a47bf9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:24.532562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:24.533487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:24.535633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:24.535633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:24.535633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:24.535633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:24.535633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 398/804] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:24.535633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c52f55-7ae1-4dc2-aeef-a3258a47bf9f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:24.587766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878d72e0-30d4-44b3-b10b-d314ba41cf6d None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:24.615451 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:24.626748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9590a-61c1-4af3-9c1a-92b2c73b9fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:24.628737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9590a-61c1-4af3-9c1a-92b2c73b9fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.628737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9590a-61c1-4af3-9c1a-92b2c73b9fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.628737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9590a-61c1-4af3-9c1a-92b2c73b9fc3 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.628978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9590a-61c1-4af3-9c1a-92b2c73b9fc3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:24.629100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9590a-61c1-4af3-9c1a-92b2c73b9fc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:24.630507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9590a-61c1-4af3-9c1a-92b2c73b9fc3 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:24.648817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c52f55-7ae1-4dc2-aeef-a3258a47bf9f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:24.655321 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 407/805] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 19:47:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:24.655321 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9590a-61c1-4af3-9c1a-92b2c73b9fc3 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:24.696897 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8f1e3-c331-4f2b-a132-316fe2b59935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:24.697755 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8f1e3-c331-4f2b-a132-316fe2b59935 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.698042 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8f1e3-c331-4f2b-a132-316fe2b59935 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.698314 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8f1e3-c331-4f2b-a132-316fe2b59935 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.706857 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8f1e3-c331-4f2b-a132-316fe2b59935 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:24.708326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8f1e3-c331-4f2b-a132-316fe2b59935 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-96b8f1e3-c331-4f2b-a132-316fe2b59935 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:24.725816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:24.726797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:24.727748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:24.727748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:24.727748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:24.727748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:24.727748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:24.727748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:24.727748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:24.727748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:24.727748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:47:24.727748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:24.727748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 399/806] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 19:47:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:24.728378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8f1e3-c331-4f2b-a132-316fe2b59935 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:24.777404 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:24.788029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9590a-61c1-4af3-9c1a-92b2c73b9fc3 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:24.823548 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d711c49-f0df-477a-8c10-832c38fff07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:24.824988 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d711c49-f0df-477a-8c10-832c38fff07d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.825286 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d711c49-f0df-477a-8c10-832c38fff07d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.825562 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d711c49-f0df-477a-8c10-832c38fff07d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.826219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d711c49-f0df-477a-8c10-832c38fff07d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:24.827584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d711c49-f0df-477a-8c10-832c38fff07d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-3d711c49-f0df-477a-8c10-832c38fff07d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:24.839993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:24.841083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:24.841938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 408/807] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 19:47:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:24.842793 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d711c49-f0df-477a-8c10-832c38fff07d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:24.857660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8f1e3-c331-4f2b-a132-316fe2b59935 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:24.888454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:24.893296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd0e82-aa7f-4d7d-93a4-6ff7f1a5cdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:24.893952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd0e82-aa7f-4d7d-93a4-6ff7f1a5cdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.894234 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd0e82-aa7f-4d7d-93a4-6ff7f1a5cdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd0e82-aa7f-4d7d-93a4-6ff7f1a5cdf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.895231 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd0e82-aa7f-4d7d-93a4-6ff7f1a5cdf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:24.895538 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd0e82-aa7f-4d7d-93a4-6ff7f1a5cdf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:24.896952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd0e82-aa7f-4d7d-93a4-6ff7f1a5cdf6 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:24.909272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 400/808] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:24.910010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd0e82-aa7f-4d7d-93a4-6ff7f1a5cdf6 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:24.944724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d711c49-f0df-477a-8c10-832c38fff07d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:24.978830 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24216e-2cf7-4d61-99a9-b0c398069f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:24.979536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24216e-2cf7-4d61-99a9-b0c398069f48 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:24.979819 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24216e-2cf7-4d61-99a9-b0c398069f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:24.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24216e-2cf7-4d61-99a9-b0c398069f48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:24.991490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24216e-2cf7-4d61-99a9-b0c398069f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:24.992963 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24216e-2cf7-4d61-99a9-b0c398069f48 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:24.995174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:25.009367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd0e82-aa7f-4d7d-93a4-6ff7f1a5cdf6 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-2d24216e-2cf7-4d61-99a9-b0c398069f48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:25.010317 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:25.013219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:25.014100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:25.014100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:25.014100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:25.014100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:25.014100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 409/809] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:25.014100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47089c07-e41f-4f47-aa9c-59aba88862ff None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.014100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47089c07-e41f-4f47-aa9c-59aba88862ff None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.014100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47089c07-e41f-4f47-aa9c-59aba88862ff None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.019074 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 410/810] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:25.019874 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47089c07-e41f-4f47-aa9c-59aba88862ff None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:25.048279 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82599a-33f6-4f0d-b726-8cfa527c8dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:25.048980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82599a-33f6-4f0d-b726-8cfa527c8dfb None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.049260 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82599a-33f6-4f0d-b726-8cfa527c8dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.049541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c82599a-33f6-4f0d-b726-8cfa527c8dfb None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.050385 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82599a-33f6-4f0d-b726-8cfa527c8dfb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:25.050954 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82599a-33f6-4f0d-b726-8cfa527c8dfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:25.052399 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c82599a-33f6-4f0d-b726-8cfa527c8dfb None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:25.080445 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 401/811] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 19:47:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:25.081316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82599a-33f6-4f0d-b726-8cfa527c8dfb None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:25.121946 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:25.125672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47089c07-e41f-4f47-aa9c-59aba88862ff None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:25.159797 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a854801-7a67-4650-aed5-c01c647c96d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:25.160488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a854801-7a67-4650-aed5-c01c647c96d3 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.160773 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a854801-7a67-4650-aed5-c01c647c96d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.161051 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a854801-7a67-4650-aed5-c01c647c96d3 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.161750 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a854801-7a67-4650-aed5-c01c647c96d3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:25.162054 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a854801-7a67-4650-aed5-c01c647c96d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:25.163417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a854801-7a67-4650-aed5-c01c647c96d3 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:25.167748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 411/812] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 19:47:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:25.168581 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a854801-7a67-4650-aed5-c01c647c96d3 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:25.206350 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c82599a-33f6-4f0d-b726-8cfa527c8dfb None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:25.240315 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e994bd-7028-4b63-8703-d5401554f867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:25.241049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e994bd-7028-4b63-8703-d5401554f867 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.241337 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e994bd-7028-4b63-8703-d5401554f867 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.241604 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e994bd-7028-4b63-8703-d5401554f867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.248547 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e994bd-7028-4b63-8703-d5401554f867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:25.249991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e994bd-7028-4b63-8703-d5401554f867 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-e6e994bd-7028-4b63-8703-d5401554f867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:25.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:25.265502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:25.271001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:25.271001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:25.271001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:25.271001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:25.271001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:25.271001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:25.271001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:25.271001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:25.271479 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a854801-7a67-4650-aed5-c01c647c96d3 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:25.271555 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 402/813] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:25.271555 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a70f49-c2fa-4325-b9d9-209cbdbb5f07 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.271555 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a70f49-c2fa-4325-b9d9-209cbdbb5f07 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.271555 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a70f49-c2fa-4325-b9d9-209cbdbb5f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.273844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:25.284994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-d2a70f49-c2fa-4325-b9d9-209cbdbb5f07 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:25.286180 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-d2a70f49-c2fa-4325-b9d9-209cbdbb5f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:25.299669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed54f08-64d6-4711-909a-da744671fda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:25.300351 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed54f08-64d6-4711-909a-da744671fda4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.301057 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed54f08-64d6-4711-909a-da744671fda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.301340 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed54f08-64d6-4711-909a-da744671fda4 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.302161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed54f08-64d6-4711-909a-da744671fda4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:25.302466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed54f08-64d6-4711-909a-da744671fda4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:25.303880 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed54f08-64d6-4711-909a-da744671fda4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:25.315223 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 412/814] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:25.315931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed54f08-64d6-4711-909a-da744671fda4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:25.344290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:25.361448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 403/815] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:25.362405 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a70f49-c2fa-4325-b9d9-209cbdbb5f07 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:25.439611 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed54f08-64d6-4711-909a-da744671fda4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:25.462369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a70f49-c2fa-4325-b9d9-209cbdbb5f07 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:25.472455 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c1e48-f122-421e-b237-a6523ca5bbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:25.473092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c1e48-f122-421e-b237-a6523ca5bbff None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.473373 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c1e48-f122-421e-b237-a6523ca5bbff None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.473649 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c1e48-f122-421e-b237-a6523ca5bbff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.496734 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5f3af-cf61-459d-b241-9bce78f29d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:25.497322 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5f3af-cf61-459d-b241-9bce78f29d86 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.497606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5f3af-cf61-459d-b241-9bce78f29d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.497876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5f3af-cf61-459d-b241-9bce78f29d86 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.498574 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5f3af-cf61-459d-b241-9bce78f29d86 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:25.498877 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5f3af-cf61-459d-b241-9bce78f29d86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:25.500253 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b5f3af-cf61-459d-b241-9bce78f29d86 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:25.501530 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:25.505853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 404/816] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 19:47:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:25.506432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5f3af-cf61-459d-b241-9bce78f29d86 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:25.543419 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c1e48-f122-421e-b237-a6523ca5bbff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:25.544897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c1e48-f122-421e-b237-a6523ca5bbff None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:25.569630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:25.606306 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5f3af-cf61-459d-b241-9bce78f29d86 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:25.640766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e4b7e-e5da-486f-ad7c-9b6fff45e627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:25.641386 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e4b7e-e5da-486f-ad7c-9b6fff45e627 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.641669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e4b7e-e5da-486f-ad7c-9b6fff45e627 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.641944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e4b7e-e5da-486f-ad7c-9b6fff45e627 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.642726 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e4b7e-e5da-486f-ad7c-9b6fff45e627 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:25.643032 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e4b7e-e5da-486f-ad7c-9b6fff45e627 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:25.644500 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608e4b7e-e5da-486f-ad7c-9b6fff45e627 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:25.644897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 413/817] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:47:25.645745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c1e48-f122-421e-b237-a6523ca5bbff None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:25.661285 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 405/818] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:25.661285 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e4b7e-e5da-486f-ad7c-9b6fff45e627 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:25.717381 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:25.747867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c1e48-f122-421e-b237-a6523ca5bbff None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:25.759429 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e4b7e-e5da-486f-ad7c-9b6fff45e627 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:25.770668 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:25.781927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd8769-c2dd-4ced-9163-1ddd6a5a6fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:25.782722 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd8769-c2dd-4ced-9163-1ddd6a5a6fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.783006 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd8769-c2dd-4ced-9163-1ddd6a5a6fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.783291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edd8769-c2dd-4ced-9163-1ddd6a5a6fe4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.790109 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd8769-c2dd-4ced-9163-1ddd6a5a6fe4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:25.791628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edd8769-c2dd-4ced-9163-1ddd6a5a6fe4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:25.792592 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd4a63-3e53-48a8-9062-93ebb4834127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:25.793920 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd4a63-3e53-48a8-9062-93ebb4834127 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.794208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd4a63-3e53-48a8-9062-93ebb4834127 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.794480 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd4a63-3e53-48a8-9062-93ebb4834127 None admin] PATH_INFO: `/v3/services/network` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.795118 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd4a63-3e53-48a8-9062-93ebb4834127 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:25.796553 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd4a63-3e53-48a8-9062-93ebb4834127 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-7edd8769-c2dd-4ced-9163-1ddd6a5a6fe4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:25.807242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:25.808382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:25.810178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:25.810178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:25.810178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:25.810178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:25.810178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:25.810178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:25.810178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:25.810178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-53cd4a63-3e53-48a8-9062-93ebb4834127 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:25.810795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:25.812021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 19:47:25.813075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:25.814126 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 414/819] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:25.814972 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd8769-c2dd-4ced-9163-1ddd6a5a6fe4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:25.815244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 406/820] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:25 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:25.815850 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd4a63-3e53-48a8-9062-93ebb4834127 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:25.913976 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edd8769-c2dd-4ced-9163-1ddd6a5a6fe4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:25.914218 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd4a63-3e53-48a8-9062-93ebb4834127 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:25.937684 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:25.948084 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc49dd-3ff4-47dd-a1a2-6834995aa2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:25.948652 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12a1d6-4ace-4154-b9be-487cd729336e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:25.948767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc49dd-3ff4-47dd-a1a2-6834995aa2aa None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.949044 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc49dd-3ff4-47dd-a1a2-6834995aa2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.949314 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc49dd-3ff4-47dd-a1a2-6834995aa2aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.949418 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12a1d6-4ace-4154-b9be-487cd729336e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:25.949669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12a1d6-4ace-4154-b9be-487cd729336e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:25.949945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12a1d6-4ace-4154-b9be-487cd729336e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:25.950055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc49dd-3ff4-47dd-a1a2-6834995aa2aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:25.951505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc49dd-3ff4-47dd-a1a2-6834995aa2aa None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:25.960884 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12a1d6-4ace-4154-b9be-487cd729336e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:25.962396 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12a1d6-4ace-4154-b9be-487cd729336e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-5a12a1d6-4ace-4154-b9be-487cd729336e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:25.979863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:25.981543 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:25.982784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:25.982784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:25.982784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:25.982784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:25.982784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 407/821] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:25.982784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12a1d6-4ace-4154-b9be-487cd729336e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:26.065738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 415/822] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 19:47:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:26.066584 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc49dd-3ff4-47dd-a1a2-6834995aa2aa None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:26.081183 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12a1d6-4ace-4154-b9be-487cd729336e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:26.128074 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc57bad-4347-40ce-95db-4d93035f9f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:26.128687 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:26.129015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc57bad-4347-40ce-95db-4d93035f9f79 None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:26.129299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc57bad-4347-40ce-95db-4d93035f9f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:26.129584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc57bad-4347-40ce-95db-4d93035f9f79 None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/f4b549638f5442b6a1be558df1686c5d/roles/a615c7d83a0c46b1b26642bcc6e6ecef` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:26.146344 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc57bad-4347-40ce-95db-4d93035f9f79 None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=f4b549638f5442b6a1be558df1686c5d, role_id=a615c7d83a0c46b1b26642bcc6e6ecef)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:26.147827 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc57bad-4347-40ce-95db-4d93035f9f79 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:26.195352 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc49dd-3ff4-47dd-a1a2-6834995aa2aa None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:26.231650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 408/823] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:47:25 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/f4b549638f5442b6a1be558df1686c5d/roles/a615c7d83a0c46b1b26642bcc6e6ecef => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:47:26.232280 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc57bad-4347-40ce-95db-4d93035f9f79 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:26.243669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f458fd-ee50-44ac-b13a-ccc12dfe402d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:26.245017 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f458fd-ee50-44ac-b13a-ccc12dfe402d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:26.245017 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f458fd-ee50-44ac-b13a-ccc12dfe402d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:26.245334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f458fd-ee50-44ac-b13a-ccc12dfe402d None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:26.246438 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f458fd-ee50-44ac-b13a-ccc12dfe402d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:26.246867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f458fd-ee50-44ac-b13a-ccc12dfe402d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:26.248465 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f458fd-ee50-44ac-b13a-ccc12dfe402d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:26.263083 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:26.270095 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 416/824] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:26.270095 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f458fd-ee50-44ac-b13a-ccc12dfe402d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:26.383227 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc57bad-4347-40ce-95db-4d93035f9f79 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:26.401708 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f458fd-ee50-44ac-b13a-ccc12dfe402d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:26.424746 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:26.427249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0392dbd-e5c6-4327-9bbe-af06c8f924b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:26.428033 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0392dbd-e5c6-4327-9bbe-af06c8f924b4 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:26.428454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0392dbd-e5c6-4327-9bbe-af06c8f924b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:26.428832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0392dbd-e5c6-4327-9bbe-af06c8f924b4 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:26.429664 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0392dbd-e5c6-4327-9bbe-af06c8f924b4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:26.430077 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0392dbd-e5c6-4327-9bbe-af06c8f924b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:26.431566 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0392dbd-e5c6-4327-9bbe-af06c8f924b4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:26.438889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 409/825] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:26.439888 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0392dbd-e5c6-4327-9bbe-af06c8f924b4 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:26.447778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0812df-29ef-4f12-9d3d-0a0c1b340333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:26.448634 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0812df-29ef-4f12-9d3d-0a0c1b340333 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:26.449021 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0812df-29ef-4f12-9d3d-0a0c1b340333 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:26.449425 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0812df-29ef-4f12-9d3d-0a0c1b340333 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:26.450267 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0812df-29ef-4f12-9d3d-0a0c1b340333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:26.450666 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0812df-29ef-4f12-9d3d-0a0c1b340333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:26.452129 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0812df-29ef-4f12-9d3d-0a0c1b340333 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:26.467211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 417/826] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:26.468217 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0812df-29ef-4f12-9d3d-0a0c1b340333 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:26.537875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:26.554943 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0392dbd-e5c6-4327-9bbe-af06c8f924b4 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:26.596839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851f9d3-9b9b-4113-89b0-d73df74f7280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:26.597660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851f9d3-9b9b-4113-89b0-d73df74f7280 None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:26.598212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851f9d3-9b9b-4113-89b0-d73df74f7280 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:26.598500 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851f9d3-9b9b-4113-89b0-d73df74f7280 None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/c8a36bb1e26f4faaa44fabff8b509fe7/roles/815bd48c3df0405babfd5b53430fd3ea` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:26.601308 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851f9d3-9b9b-4113-89b0-d73df74f7280 None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=c8a36bb1e26f4faaa44fabff8b509fe7, role_id=815bd48c3df0405babfd5b53430fd3ea)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:26.603780 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851f9d3-9b9b-4113-89b0-d73df74f7280 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:26.609280 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0812df-29ef-4f12-9d3d-0a0c1b340333 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:26.645042 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2664f8-acbd-496e-ba93-967ec611dd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:26.645741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2664f8-acbd-496e-ba93-967ec611dd23 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:26.646028 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2664f8-acbd-496e-ba93-967ec611dd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:26.646299 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2664f8-acbd-496e-ba93-967ec611dd23 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:26.647013 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2664f8-acbd-496e-ba93-967ec611dd23 None admin] RBAC: Adding query filter params (type=network) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:26.647319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2664f8-acbd-496e-ba93-967ec611dd23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:26.648699 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2664f8-acbd-496e-ba93-967ec611dd23 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:26.654042 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 418/827] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:26.658667 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2664f8-acbd-496e-ba93-967ec611dd23 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:26.681530 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 410/828] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:47:26 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/c8a36bb1e26f4faaa44fabff8b509fe7/roles/815bd48c3df0405babfd5b53430fd3ea => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:47:26.683609 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851f9d3-9b9b-4113-89b0-d73df74f7280 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:26.686683 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:26.819017 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851f9d3-9b9b-4113-89b0-d73df74f7280 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:26.857051 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2664f8-acbd-496e-ba93-967ec611dd23 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:26.864350 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92ce25-16c3-4034-ab1f-c5424d8ea279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:26.865379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92ce25-16c3-4034-ab1f-c5424d8ea279 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:26.865913 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92ce25-16c3-4034-ab1f-c5424d8ea279 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:26.866401 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92ce25-16c3-4034-ab1f-c5424d8ea279 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:26.867384 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92ce25-16c3-4034-ab1f-c5424d8ea279 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:26.869073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92ce25-16c3-4034-ab1f-c5424d8ea279 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:26.925690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fee6cc-83da-4b11-85c6-398152e7fbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:26.926501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fee6cc-83da-4b11-85c6-398152e7fbf6 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:26.926784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fee6cc-83da-4b11-85c6-398152e7fbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:26.927059 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fee6cc-83da-4b11-85c6-398152e7fbf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:26.934897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fee6cc-83da-4b11-85c6-398152e7fbf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:26.944675 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fee6cc-83da-4b11-85c6-398152e7fbf6 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-48fee6cc-83da-4b11-85c6-398152e7fbf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:26.951828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:26.953226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:26.954205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:26.954205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:26.954205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:26.954205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:26.954205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:26.954205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:26.954205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:26.954205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:26.954205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 419/829] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:26.959712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fee6cc-83da-4b11-85c6-398152e7fbf6 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:26.978814 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:26.979765 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 411/830] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 19:47:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:27.060754 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:27.081565 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fee6cc-83da-4b11-85c6-398152e7fbf6 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:27.128978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cca5c9-a2b4-425f-ac1d-141cd57eaf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:27.129568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cca5c9-a2b4-425f-ac1d-141cd57eaf35 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:27.129850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cca5c9-a2b4-425f-ac1d-141cd57eaf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:27.130122 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cca5c9-a2b4-425f-ac1d-141cd57eaf35 None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:27.130921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cca5c9-a2b4-425f-ac1d-141cd57eaf35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:27.131228 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cca5c9-a2b4-425f-ac1d-141cd57eaf35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:27.132721 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cca5c9-a2b4-425f-ac1d-141cd57eaf35 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:27.147333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 420/831] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:27.153436 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92ce25-16c3-4034-ab1f-c5424d8ea279 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:27.301268 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92ce25-16c3-4034-ab1f-c5424d8ea279 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:27.339898 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9021107-bd63-4540-b90d-47fffa57a805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:27.340946 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9021107-bd63-4540-b90d-47fffa57a805 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:27.341458 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9021107-bd63-4540-b90d-47fffa57a805 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:27.341937 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9021107-bd63-4540-b90d-47fffa57a805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:27.343003 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9021107-bd63-4540-b90d-47fffa57a805 None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=0c0e71329c4a4470935593636c025257) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:27.343554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9021107-bd63-4540-b90d-47fffa57a805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:27.345237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9021107-bd63-4540-b90d-47fffa57a805 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:27.352668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 412/832] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:27 2022] GET /identity/v3/role_assignments?user.id=0c0e71329c4a4470935593636c025257&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:28.539279 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c35f1-17dd-4bc7-b0ae-18a2c7c9c1ca None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:28.539868 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c35f1-17dd-4bc7-b0ae-18a2c7c9c1ca None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:28.539868 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c35f1-17dd-4bc7-b0ae-18a2c7c9c1ca None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:28.541567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 421/833] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:28.547312 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a4b0c-819d-46e4-9ef2-96c750ff5cd4 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:28.547797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a4b0c-819d-46e4-9ef2-96c750ff5cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:28.547878 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a4b0c-819d-46e4-9ef2-96c750ff5cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:28.566856 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-837a4b0c-819d-46e4-9ef2-96c750ff5cd4 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:28.568131 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-837a4b0c-819d-46e4-9ef2-96c750ff5cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:28.590709 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9c510-f228-4902-8157-c1261765caf8 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:28.591003 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9c510-f228-4902-8157-c1261765caf8 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:28.591284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9c510-f228-4902-8157-c1261765caf8 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:28.592943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 422/834] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:47:28.598047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b35027-7db0-485d-adf2-bc785ae90787 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:28.598270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b35027-7db0-485d-adf2-bc785ae90787 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:28.598542 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b35027-7db0-485d-adf2-bc785ae90787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:28.615632 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-18b35027-7db0-485d-adf2-bc785ae90787 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:28.616836 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-18b35027-7db0-485d-adf2-bc785ae90787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:28.632701 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:28.644109 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 413/835] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:28.667185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a4b0c-819d-46e4-9ef2-96c750ff5cd4 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:28.689792 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 423/836] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:28.717793 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b35027-7db0-485d-adf2-bc785ae90787 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:28.787006 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:28.788422 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a4b0c-819d-46e4-9ef2-96c750ff5cd4 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:28.822369 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b35027-7db0-485d-adf2-bc785ae90787 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:28.825232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b6c7e-2787-481b-8725-f3cc0e6eedc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:28.826197 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b6c7e-2787-481b-8725-f3cc0e6eedc6 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:28.826636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b6c7e-2787-481b-8725-f3cc0e6eedc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:28.827057 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b6c7e-2787-481b-8725-f3cc0e6eedc6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:28.827861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b6c7e-2787-481b-8725-f3cc0e6eedc6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:28.829626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460b6c7e-2787-481b-8725-f3cc0e6eedc6 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-460b6c7e-2787-481b-8725-f3cc0e6eedc6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:28.844407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:28.845446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:47:28.846338 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:28.852808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 414/837] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:47:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:28.855489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b6c7e-2787-481b-8725-f3cc0e6eedc6 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:28.858750 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888cefb-669c-4828-86ad-e0486d76b7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:28.859616 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888cefb-669c-4828-86ad-e0486d76b7ed None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:28.860376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888cefb-669c-4828-86ad-e0486d76b7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:28.860376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888cefb-669c-4828-86ad-e0486d76b7ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:28.873708 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888cefb-669c-4828-86ad-e0486d76b7ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-f888cefb-669c-4828-86ad-e0486d76b7ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:28.881628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:28.882556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:47:28.883383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:28.888298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 424/838] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:47:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:28.893263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888cefb-669c-4828-86ad-e0486d76b7ed None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:28.931273 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:28.981106 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b6c7e-2787-481b-8725-f3cc0e6eedc6 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:29.008543 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888cefb-669c-4828-86ad-e0486d76b7ed None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:29.032823 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf704a-da1a-4052-81a0-c34d2c57bd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:29.033768 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf704a-da1a-4052-81a0-c34d2c57bd45 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.034054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf704a-da1a-4052-81a0-c34d2c57bd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.034336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf704a-da1a-4052-81a0-c34d2c57bd45 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.035046 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf704a-da1a-4052-81a0-c34d2c57bd45 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:29.035617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf704a-da1a-4052-81a0-c34d2c57bd45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:29.037752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf704a-da1a-4052-81a0-c34d2c57bd45 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:29.040080 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:29.048386 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 415/839] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:47:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:29.051322 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf704a-da1a-4052-81a0-c34d2c57bd45 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:29.055010 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccef962-4e5d-4b47-9091-aef6333e51c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:29.055884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccef962-4e5d-4b47-9091-aef6333e51c8 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.059290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccef962-4e5d-4b47-9091-aef6333e51c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.059290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccef962-4e5d-4b47-9091-aef6333e51c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.059290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccef962-4e5d-4b47-9091-aef6333e51c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:29.059290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccef962-4e5d-4b47-9091-aef6333e51c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:29.059290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccef962-4e5d-4b47-9091-aef6333e51c8 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:29.068914 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 425/840] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:47:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:29.074745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccef962-4e5d-4b47-9091-aef6333e51c8 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:29.193560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccef962-4e5d-4b47-9091-aef6333e51c8 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:29.199041 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf704a-da1a-4052-81a0-c34d2c57bd45 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:29.231194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:29.241011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069264f-a347-480f-bea6-f8723502cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:29.244462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069264f-a347-480f-bea6-f8723502cb29 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.244462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069264f-a347-480f-bea6-f8723502cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.244462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069264f-a347-480f-bea6-f8723502cb29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.244680 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3f7cc-b223-4db1-ad7b-700263c87534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:29.245416 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3f7cc-b223-4db1-ad7b-700263c87534 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.246466 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3f7cc-b223-4db1-ad7b-700263c87534 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.246977 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3f7cc-b223-4db1-ad7b-700263c87534 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.247913 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3f7cc-b223-4db1-ad7b-700263c87534 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:29.248446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3f7cc-b223-4db1-ad7b-700263c87534 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:29.250067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3f7cc-b223-4db1-ad7b-700263c87534 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:29.255161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069264f-a347-480f-bea6-f8723502cb29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:29.260614 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7069264f-a347-480f-bea6-f8723502cb29 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:29.260781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 416/841] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:47:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:29.280186 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9805179-8469-4cd6-89a0-fcebce038559 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.280503 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9805179-8469-4cd6-89a0-fcebce038559 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.280790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9805179-8469-4cd6-89a0-fcebce038559 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-7069264f-a347-480f-bea6-f8723502cb29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:29.284185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:29.285136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:29.286019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:29.286019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:29.286019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:29.286019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:29.286019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 426/842] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:29.287005 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 417/843] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:47:29.288506 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920d16f-9ab9-42f8-8e42-82810c34e921 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.290319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920d16f-9ab9-42f8-8e42-82810c34e921 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.290319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3920d16f-9ab9-42f8-8e42-82810c34e921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.291248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9805179-8469-4cd6-89a0-fcebce038559 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:29.319935 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-3920d16f-9ab9-42f8-8e42-82810c34e921 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:29.322303 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-3920d16f-9ab9-42f8-8e42-82810c34e921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:29.377505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:29.429084 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9805179-8469-4cd6-89a0-fcebce038559 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:29.430490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 427/844] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:29.455156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920d16f-9ab9-42f8-8e42-82810c34e921 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:29.476001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54e4c3-e823-4abe-8f47-0bae3ff23d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:29.476690 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54e4c3-e823-4abe-8f47-0bae3ff23d26 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.476980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54e4c3-e823-4abe-8f47-0bae3ff23d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.488862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54e4c3-e823-4abe-8f47-0bae3ff23d26 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.488862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54e4c3-e823-4abe-8f47-0bae3ff23d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:29.488862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54e4c3-e823-4abe-8f47-0bae3ff23d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:29.488862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54e4c3-e823-4abe-8f47-0bae3ff23d26 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:29.496944 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 418/845] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:29.503035 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54e4c3-e823-4abe-8f47-0bae3ff23d26 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:29.567360 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:29.581144 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3920d16f-9ab9-42f8-8e42-82810c34e921 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:29.620706 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecefc7-43c1-4e29-84f6-0070770d5c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:29.621756 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecefc7-43c1-4e29-84f6-0070770d5c70 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.622128 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecefc7-43c1-4e29-84f6-0070770d5c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.622536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecefc7-43c1-4e29-84f6-0070770d5c70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.634897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecefc7-43c1-4e29-84f6-0070770d5c70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:29.640473 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54e4c3-e823-4abe-8f47-0bae3ff23d26 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-74ecefc7-43c1-4e29-84f6-0070770d5c70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:29.644586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:29.645938 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:29.646806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 428/846] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:29.654395 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecefc7-43c1-4e29-84f6-0070770d5c70 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:29.697267 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:29.699657 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc6ef1-3ca4-4fdc-9a18-d18961b1fe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:29.700862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc6ef1-3ca4-4fdc-9a18-d18961b1fe5a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.701378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc6ef1-3ca4-4fdc-9a18-d18961b1fe5a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.701823 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc6ef1-3ca4-4fdc-9a18-d18961b1fe5a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.710835 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc6ef1-3ca4-4fdc-9a18-d18961b1fe5a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:29.712712 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc6ef1-3ca4-4fdc-9a18-d18961b1fe5a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-87fc6ef1-3ca4-4fdc-9a18-d18961b1fe5a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:29.730116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:29.736665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:29.737670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:29.737670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:29.737670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:29.737670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:29.737670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:29.737670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:29.737670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:29.737670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:29.737670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:29.737670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:29.737670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 419/847] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 19:47:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:29.739716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc6ef1-3ca4-4fdc-9a18-d18961b1fe5a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:29.790220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecefc7-43c1-4e29-84f6-0070770d5c70 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:29.828616 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f5c6c-a718-4adc-9775-d92e34b13979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:29.829291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f5c6c-a718-4adc-9775-d92e34b13979 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.829571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f5c6c-a718-4adc-9775-d92e34b13979 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.829836 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f5c6c-a718-4adc-9775-d92e34b13979 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.830665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f5c6c-a718-4adc-9775-d92e34b13979 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:29.830980 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f5c6c-a718-4adc-9775-d92e34b13979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:29.832420 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f5c6c-a718-4adc-9775-d92e34b13979 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:29.845472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 429/848] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:47:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:29.845472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e610a4d-68fb-442b-b4f9-3e841718779e None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.845472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e610a4d-68fb-442b-b4f9-3e841718779e None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.845472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e610a4d-68fb-442b-b4f9-3e841718779e None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.845472 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 430/849] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:29.848954 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e610a4d-68fb-442b-b4f9-3e841718779e None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:29.860600 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc6ef1-3ca4-4fdc-9a18-d18961b1fe5a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:29.868650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:29.896321 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16241de-d7b5-40a4-b0c5-cc70600c5668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:29.897015 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16241de-d7b5-40a4-b0c5-cc70600c5668 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.897297 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16241de-d7b5-40a4-b0c5-cc70600c5668 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.897570 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16241de-d7b5-40a4-b0c5-cc70600c5668 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.898402 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16241de-d7b5-40a4-b0c5-cc70600c5668 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:29.898709 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16241de-d7b5-40a4-b0c5-cc70600c5668 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:29.900239 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16241de-d7b5-40a4-b0c5-cc70600c5668 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:29.923614 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 420/850] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 19:47:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:29.923614 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a4cd4-bcfd-4528-bf3b-572487226e77 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.923614 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a4cd4-bcfd-4528-bf3b-572487226e77 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.923614 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a4cd4-bcfd-4528-bf3b-572487226e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:29.938450 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-cc3a4cd4-bcfd-4528-bf3b-572487226e77 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:29.939713 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-cc3a4cd4-bcfd-4528-bf3b-572487226e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:29.955502 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:29.964069 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e610a4d-68fb-442b-b4f9-3e841718779e None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:29.998692 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731491a7-ee21-4f4b-b57b-4ecc402403e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:29.999403 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731491a7-ee21-4f4b-b57b-4ecc402403e8 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:29.999688 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731491a7-ee21-4f4b-b57b-4ecc402403e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:29.999966 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731491a7-ee21-4f4b-b57b-4ecc402403e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:30.010051 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731491a7-ee21-4f4b-b57b-4ecc402403e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:30.011483 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731491a7-ee21-4f4b-b57b-4ecc402403e8 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:30.020541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 421/851] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:30.020541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a4cd4-bcfd-4528-bf3b-572487226e77 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-731491a7-ee21-4f4b-b57b-4ecc402403e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:30.038651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:30.039601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:30.040690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:30.040690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:30.040690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:30.040690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:30.040690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 431/852] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:30.043805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731491a7-ee21-4f4b-b57b-4ecc402403e8 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:30.121508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:30.128431 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a4cd4-bcfd-4528-bf3b-572487226e77 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:30.160296 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731491a7-ee21-4f4b-b57b-4ecc402403e8 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:30.163626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d4f0e-b55c-4826-b006-4e6a91962149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:30.164279 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d4f0e-b55c-4826-b006-4e6a91962149 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:30.164590 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d4f0e-b55c-4826-b006-4e6a91962149 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:30.164864 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d4f0e-b55c-4826-b006-4e6a91962149 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:30.197457 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371c88d-b8d4-4a45-9a40-529dd1266df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:30.198268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371c88d-b8d4-4a45-9a40-529dd1266df9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:30.198551 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371c88d-b8d4-4a45-9a40-529dd1266df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:30.198823 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371c88d-b8d4-4a45-9a40-529dd1266df9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:30.199622 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371c88d-b8d4-4a45-9a40-529dd1266df9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:30.200958 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9371c88d-b8d4-4a45-9a40-529dd1266df9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-9371c88d-b8d4-4a45-9a40-529dd1266df9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:30.214316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:30.215832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:30.216737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 432/853] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 19:47:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:30.217589 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371c88d-b8d4-4a45-9a40-529dd1266df9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:30.231861 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:30.245826 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d4f0e-b55c-4826-b006-4e6a91962149 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:30.247289 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d4f0e-b55c-4826-b006-4e6a91962149 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:30.334433 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371c88d-b8d4-4a45-9a40-529dd1266df9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:30.344796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:30.353100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 422/854] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:47:30.358168 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d4f0e-b55c-4826-b006-4e6a91962149 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:30.375941 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd818f-16b3-4851-a3cc-e2a216ed49bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:30.376700 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd818f-16b3-4851-a3cc-e2a216ed49bd None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:30.376992 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd818f-16b3-4851-a3cc-e2a216ed49bd None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:30.377277 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd818f-16b3-4851-a3cc-e2a216ed49bd None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:30.378036 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd818f-16b3-4851-a3cc-e2a216ed49bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:30.378389 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd818f-16b3-4851-a3cc-e2a216ed49bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:30.379808 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd818f-16b3-4851-a3cc-e2a216ed49bd None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:30.392381 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 433/855] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:30.393301 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd818f-16b3-4851-a3cc-e2a216ed49bd None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:30.473116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d4f0e-b55c-4826-b006-4e6a91962149 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:30.473306 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:30.502172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd818f-16b3-4851-a3cc-e2a216ed49bd None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:30.505584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a0435-b5b3-4712-b9cb-94908e56fc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:30.506503 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a0435-b5b3-4712-b9cb-94908e56fc1f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:30.506972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a0435-b5b3-4712-b9cb-94908e56fc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:30.507395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a0435-b5b3-4712-b9cb-94908e56fc1f None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:30.508359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a0435-b5b3-4712-b9cb-94908e56fc1f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:30.508819 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a0435-b5b3-4712-b9cb-94908e56fc1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:30.510366 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a0435-b5b3-4712-b9cb-94908e56fc1f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:30.521256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 423/856] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 19:47:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:30.521256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a0435-b5b3-4712-b9cb-94908e56fc1f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:30.546750 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22cc0e-e9fb-440c-b2e5-91976bc6c948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:30.547844 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22cc0e-e9fb-440c-b2e5-91976bc6c948 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:30.548115 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22cc0e-e9fb-440c-b2e5-91976bc6c948 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:30.548416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22cc0e-e9fb-440c-b2e5-91976bc6c948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:30.559135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22cc0e-e9fb-440c-b2e5-91976bc6c948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:30.560590 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b22cc0e-e9fb-440c-b2e5-91976bc6c948 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-4b22cc0e-e9fb-440c-b2e5-91976bc6c948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:30.579789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:30.580796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:30.582173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:30.582173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:30.582173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:30.582173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:30.582173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 434/857] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:30.582173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22cc0e-e9fb-440c-b2e5-91976bc6c948 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:30.590250 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:30.635861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a0435-b5b3-4712-b9cb-94908e56fc1f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:30.673458 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cea492-14da-43a8-8b92-56da2288d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:30.674541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cea492-14da-43a8-8b92-56da2288d59e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:30.674838 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cea492-14da-43a8-8b92-56da2288d59e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:30.675263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cea492-14da-43a8-8b92-56da2288d59e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:30.684166 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cea492-14da-43a8-8b92-56da2288d59e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:30.685292 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22cc0e-e9fb-440c-b2e5-91976bc6c948 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:30.685663 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cea492-14da-43a8-8b92-56da2288d59e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-d5cea492-14da-43a8-8b92-56da2288d59e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:30.703873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:30.705072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 424/858] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:30.706016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cea492-14da-43a8-8b92-56da2288d59e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:30.715873 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:30.730199 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a02fd7-dc89-49ea-a2a4-4a7dd52ab025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:30.730199 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a02fd7-dc89-49ea-a2a4-4a7dd52ab025 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:30.730199 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a02fd7-dc89-49ea-a2a4-4a7dd52ab025 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:30.730199 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a02fd7-dc89-49ea-a2a4-4a7dd52ab025 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:30.730843 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a02fd7-dc89-49ea-a2a4-4a7dd52ab025 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:30.731173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a02fd7-dc89-49ea-a2a4-4a7dd52ab025 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:30.732628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a02fd7-dc89-49ea-a2a4-4a7dd52ab025 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:30.737169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 435/859] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 19:47:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:30.737835 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a02fd7-dc89-49ea-a2a4-4a7dd52ab025 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:30.849158 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cea492-14da-43a8-8b92-56da2288d59e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:30.856221 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a02fd7-dc89-49ea-a2a4-4a7dd52ab025 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:30.875267 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:30.898825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b2034-e7cb-4f5f-bb80-4f3d9d12c355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:30.899503 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b2034-e7cb-4f5f-bb80-4f3d9d12c355 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:30.899789 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b2034-e7cb-4f5f-bb80-4f3d9d12c355 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:30.900067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b2034-e7cb-4f5f-bb80-4f3d9d12c355 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:30.900859 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b2034-e7cb-4f5f-bb80-4f3d9d12c355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:30.901165 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b2034-e7cb-4f5f-bb80-4f3d9d12c355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:30.902687 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b2034-e7cb-4f5f-bb80-4f3d9d12c355 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:30.903856 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f06e3-d19d-4947-877d-85f22f3758d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:30.905102 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f06e3-d19d-4947-877d-85f22f3758d5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:30.906112 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f06e3-d19d-4947-877d-85f22f3758d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:30.906599 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f06e3-d19d-4947-877d-85f22f3758d5 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:30.907663 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f06e3-d19d-4947-877d-85f22f3758d5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:30.908183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f06e3-d19d-4947-877d-85f22f3758d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:30.909855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f06e3-d19d-4947-877d-85f22f3758d5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:30.920295 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 425/860] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:30.922325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b2034-e7cb-4f5f-bb80-4f3d9d12c355 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:30.938765 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 436/861] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 19:47:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:30.945504 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f06e3-d19d-4947-877d-85f22f3758d5 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:31.000963 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:31.058312 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b2034-e7cb-4f5f-bb80-4f3d9d12c355 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:31.065850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f06e3-d19d-4947-877d-85f22f3758d5 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:31.095213 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b9031-4d0f-461b-84f9-4109645024f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:31.096007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b9031-4d0f-461b-84f9-4109645024f8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:31.096281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b9031-4d0f-461b-84f9-4109645024f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:31.096588 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b9031-4d0f-461b-84f9-4109645024f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:31.102633 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f77c29-4481-4adf-9cb2-25e680a05ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:31.103287 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f77c29-4481-4adf-9cb2-25e680a05ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:31.103570 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f77c29-4481-4adf-9cb2-25e680a05ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:31.103794 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b9031-4d0f-461b-84f9-4109645024f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:31.103900 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f77c29-4481-4adf-9cb2-25e680a05ae2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:31.105353 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b9031-4d0f-461b-84f9-4109645024f8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-2b1b9031-4d0f-461b-84f9-4109645024f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:31.120159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:31.121185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:31.122067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:31.122067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:31.122067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:31.122067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:31.122067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:31.122067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:31.122067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:31.122067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:31.122067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 426/862] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:31.128286 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b9031-4d0f-461b-84f9-4109645024f8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:31.171762 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:31.177012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f77c29-4481-4adf-9cb2-25e680a05ae2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:31.178606 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f77c29-4481-4adf-9cb2-25e680a05ae2 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:31.228306 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:31.238328 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b9031-4d0f-461b-84f9-4109645024f8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:31.275303 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c7f19-9ad5-4d06-a304-ad18a94a0180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:31.276300 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c7f19-9ad5-4d06-a304-ad18a94a0180 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:31.276789 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c7f19-9ad5-4d06-a304-ad18a94a0180 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:31.277799 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c7f19-9ad5-4d06-a304-ad18a94a0180 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:31.278783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c7f19-9ad5-4d06-a304-ad18a94a0180 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:31.279283 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c7f19-9ad5-4d06-a304-ad18a94a0180 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:31.281068 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c7f19-9ad5-4d06-a304-ad18a94a0180 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:31.288223 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 437/863] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:47:31.293631 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f77c29-4481-4adf-9cb2-25e680a05ae2 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:31.298030 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 427/864] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:31.303397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c7f19-9ad5-4d06-a304-ad18a94a0180 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:31.416953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:31.419998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c7f19-9ad5-4d06-a304-ad18a94a0180 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:31.420314 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f77c29-4481-4adf-9cb2-25e680a05ae2 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:31.457954 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddba955-43f7-42ae-9dbb-b4c41ea032f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:31.458614 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddba955-43f7-42ae-9dbb-b4c41ea032f0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:31.458898 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddba955-43f7-42ae-9dbb-b4c41ea032f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:31.459176 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddba955-43f7-42ae-9dbb-b4c41ea032f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:31.460025 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddba955-43f7-42ae-9dbb-b4c41ea032f0 None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=f4b549638f5442b6a1be558df1686c5d) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:31.460367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddba955-43f7-42ae-9dbb-b4c41ea032f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:31.461516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcaee8b-c758-4f33-828b-5e0f7e7d626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:31.461837 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddba955-43f7-42ae-9dbb-b4c41ea032f0 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:31.462752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaee8b-c758-4f33-828b-5e0f7e7d626c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:31.463364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaee8b-c758-4f33-828b-5e0f7e7d626c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:31.463906 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcaee8b-c758-4f33-828b-5e0f7e7d626c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:31.473423 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 428/865] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:31 2022] GET /identity/v3/role_assignments?user.id=f4b549638f5442b6a1be558df1686c5d&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:31.479916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcaee8b-c758-4f33-828b-5e0f7e7d626c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:31.481471 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcaee8b-c758-4f33-828b-5e0f7e7d626c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:31.492885 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-8fcaee8b-c758-4f33-828b-5e0f7e7d626c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:31.502467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:31.509910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:31.510871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:31.510871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:31.510871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:31.510871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:31.510871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 438/866] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:31.513526 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddba955-43f7-42ae-9dbb-b4c41ea032f0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:31.601872 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24399b3c-9eb7-4a74-b30d-164ff0bd9636 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:31.602406 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24399b3c-9eb7-4a74-b30d-164ff0bd9636 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:31.602406 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24399b3c-9eb7-4a74-b30d-164ff0bd9636 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:31.603980 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 439/867] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:47:31.616112 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8f877-0ac3-4a27-874c-4e5ff441e4ba None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:31.616486 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8f877-0ac3-4a27-874c-4e5ff441e4ba None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:31.616695 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc8f877-0ac3-4a27-874c-4e5ff441e4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:31.638602 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-6bc8f877-0ac3-4a27-874c-4e5ff441e4ba None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:31.640438 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-6bc8f877-0ac3-4a27-874c-4e5ff441e4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:31.643111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddba955-43f7-42ae-9dbb-b4c41ea032f0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:31.687186 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2babe1-10fb-4823-ac8c-5d293d72068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:31.688109 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2babe1-10fb-4823-ac8c-5d293d72068c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:31.688454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2babe1-10fb-4823-ac8c-5d293d72068c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:31.688703 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2babe1-10fb-4823-ac8c-5d293d72068c None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:31.689535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2babe1-10fb-4823-ac8c-5d293d72068c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:31.689848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2babe1-10fb-4823-ac8c-5d293d72068c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:31.691239 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2babe1-10fb-4823-ac8c-5d293d72068c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:31.704196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 429/868] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:31.709648 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2babe1-10fb-4823-ac8c-5d293d72068c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:31.735545 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 440/869] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:31.743220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:31.758492 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8f877-0ac3-4a27-874c-4e5ff441e4ba None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:31.837415 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2babe1-10fb-4823-ac8c-5d293d72068c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:31.873604 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823df366-8bd7-4e21-a2b6-653776425736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:31.874397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823df366-8bd7-4e21-a2b6-653776425736 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:31.874678 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823df366-8bd7-4e21-a2b6-653776425736 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:31.874951 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823df366-8bd7-4e21-a2b6-653776425736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:31.877902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc8f877-0ac3-4a27-874c-4e5ff441e4ba None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:31.882476 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823df366-8bd7-4e21-a2b6-653776425736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:31.883947 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823df366-8bd7-4e21-a2b6-653776425736 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-823df366-8bd7-4e21-a2b6-653776425736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:31.907281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:31.908252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:31.909198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:31.909198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:31.909198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:31.909198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:31.909198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:31.909198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:31.909198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:31.909198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:31.909629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:31.913906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 430/870] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:31.915254 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823df366-8bd7-4e21-a2b6-653776425736 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:31.918330 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fba13c-7113-4d7c-b7ec-00cc4e8be8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:31.919005 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fba13c-7113-4d7c-b7ec-00cc4e8be8ef None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:31.919284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fba13c-7113-4d7c-b7ec-00cc4e8be8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:31.919557 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fba13c-7113-4d7c-b7ec-00cc4e8be8ef None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:31.920295 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fba13c-7113-4d7c-b7ec-00cc4e8be8ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:31.921688 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fba13c-7113-4d7c-b7ec-00cc4e8be8ef None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:32.005103 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 441/871] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:47:31 2022] POST /identity/v3/services => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:47:32.014241 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:32.038512 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823df366-8bd7-4e21-a2b6-653776425736 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:32.087096 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2e438-e656-499f-8b43-44dc05a9e6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:32.087885 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2e438-e656-499f-8b43-44dc05a9e6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:32.088163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2e438-e656-499f-8b43-44dc05a9e6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:32.088525 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2e438-e656-499f-8b43-44dc05a9e6e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:32.089331 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2e438-e656-499f-8b43-44dc05a9e6e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:32.090304 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2e438-e656-499f-8b43-44dc05a9e6e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:32.091976 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2e438-e656-499f-8b43-44dc05a9e6e3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:32.111556 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 431/872] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:32.113516 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fba13c-7113-4d7c-b7ec-00cc4e8be8ef None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:32.233437 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fba13c-7113-4d7c-b7ec-00cc4e8be8ef None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:32.272672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215dee3d-0f54-4913-a28e-48219ad137de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:32.273342 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dee3d-0f54-4913-a28e-48219ad137de None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:32.273626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dee3d-0f54-4913-a28e-48219ad137de None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:32.273900 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dee3d-0f54-4913-a28e-48219ad137de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:32.274753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215dee3d-0f54-4913-a28e-48219ad137de None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=c8a36bb1e26f4faaa44fabff8b509fe7) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:32.275071 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215dee3d-0f54-4913-a28e-48219ad137de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:32.276591 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215dee3d-0f54-4913-a28e-48219ad137de None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:32.283901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 442/873] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:32 2022] GET /identity/v3/role_assignments?user.id=c8a36bb1e26f4faaa44fabff8b509fe7&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:33.116110 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e42b3-ba84-4177-97ac-f8b44fc0b176 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:33.116658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e42b3-ba84-4177-97ac-f8b44fc0b176 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:33.116658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e42b3-ba84-4177-97ac-f8b44fc0b176 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:33.118321 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 432/874] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:33.123665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b739b2-c6cc-4740-b9d2-c0776ae37ede None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:33.124093 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b739b2-c6cc-4740-b9d2-c0776ae37ede None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:33.124269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b739b2-c6cc-4740-b9d2-c0776ae37ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:33.141260 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-69b739b2-c6cc-4740-b9d2-c0776ae37ede None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:33.142430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-69b739b2-c6cc-4740-b9d2-c0776ae37ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:33.190641 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:33.219170 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 443/875] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:33.238114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e42b3-ba84-4177-97ac-f8b44fc0b176 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:33.360935 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e42b3-ba84-4177-97ac-f8b44fc0b176 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:33.397584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d56f1e-419d-4935-9d82-4d414b728374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:33.398167 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56f1e-419d-4935-9d82-4d414b728374 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:33.398447 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56f1e-419d-4935-9d82-4d414b728374 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:33.398715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56f1e-419d-4935-9d82-4d414b728374 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:33.399438 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d56f1e-419d-4935-9d82-4d414b728374 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:33.400917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d56f1e-419d-4935-9d82-4d414b728374 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:33.495671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:33.508562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 433/876] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:47:33 2022] POST /identity/v3/services => generated 269 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:33.882379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacacf6-d653-426d-b706-e2d25e76b433 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:33.882772 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacacf6-d653-426d-b706-e2d25e76b433 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:33.882981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacacf6-d653-426d-b706-e2d25e76b433 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:33.884619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 444/877] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:33.890759 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8910-e528-49e9-b7fe-2be888e80cc3 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:33.891050 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8910-e528-49e9-b7fe-2be888e80cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:33.891336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8910-e528-49e9-b7fe-2be888e80cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:33.908262 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-782b8910-e528-49e9-b7fe-2be888e80cc3 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:33.909486 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-782b8910-e528-49e9-b7fe-2be888e80cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:33.995622 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 434/878] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:34.019029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacacf6-d653-426d-b706-e2d25e76b433 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:34.128509 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeacacf6-d653-426d-b706-e2d25e76b433 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:34.166460 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb214dc9-f836-404b-aceb-3fa6529178a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:34.167217 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb214dc9-f836-404b-aceb-3fa6529178a9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:34.167534 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb214dc9-f836-404b-aceb-3fa6529178a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:34.167812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb214dc9-f836-404b-aceb-3fa6529178a9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:34.180701 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb214dc9-f836-404b-aceb-3fa6529178a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-cb214dc9-f836-404b-aceb-3fa6529178a9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:34.189273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:34.190311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:34.191172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 445/879] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 19:47:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:34.196495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b8910-e528-49e9-b7fe-2be888e80cc3 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:34.300126 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b8910-e528-49e9-b7fe-2be888e80cc3 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:34.333165 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:34.339051 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4d30c-3203-4618-b9e0-7d1f037692c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:34.340298 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d30c-3203-4618-b9e0-7d1f037692c9 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:34.340616 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d30c-3203-4618-b9e0-7d1f037692c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:34.340893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4d30c-3203-4618-b9e0-7d1f037692c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:34.341722 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4d30c-3203-4618-b9e0-7d1f037692c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:34.342086 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4d30c-3203-4618-b9e0-7d1f037692c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:34.343470 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4d30c-3203-4618-b9e0-7d1f037692c9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:34.352437 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 435/880] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 19:47:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:34.358154 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb214dc9-f836-404b-aceb-3fa6529178a9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:34.365132 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97e20e-5175-49b1-b6da-b185376bd587 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:34.365425 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97e20e-5175-49b1-b6da-b185376bd587 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:34.365711 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97e20e-5175-49b1-b6da-b185376bd587 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:34.367363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 436/881] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:34.372653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d4046-cdb0-4542-a123-a9e1619b847b None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:34.372949 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d4046-cdb0-4542-a123-a9e1619b847b None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:34.373232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d4046-cdb0-4542-a123-a9e1619b847b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:34.390842 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-7e4d4046-cdb0-4542-a123-a9e1619b847b None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:34.391967 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-7e4d4046-cdb0-4542-a123-a9e1619b847b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:34.405202 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:34.477922 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb214dc9-f836-404b-aceb-3fa6529178a9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:34.482092 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 437/882] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:34.506671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d4046-cdb0-4542-a123-a9e1619b847b None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:34.525633 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5c598-c235-4892-8963-c47771c7b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:34.526430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c598-c235-4892-8963-c47771c7b699 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:34.531189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c598-c235-4892-8963-c47771c7b699 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:34.531189 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c598-c235-4892-8963-c47771c7b699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:34.539066 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5c598-c235-4892-8963-c47771c7b699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:34.540554 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5c598-c235-4892-8963-c47771c7b699 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-baa5c598-c235-4892-8963-c47771c7b699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:34.559444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:34.560434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:34.561340 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:34.561340 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:34.561340 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:34.561340 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:34.561340 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 446/883] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:34.566086 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5c598-c235-4892-8963-c47771c7b699 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:34.619728 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d4046-cdb0-4542-a123-a9e1619b847b None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:34.637261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:34.656754 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9816a650-f2c1-40d6-97cd-32391962e685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:34.657790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a650-f2c1-40d6-97cd-32391962e685 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:34.658219 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a650-f2c1-40d6-97cd-32391962e685 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:34.658636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9816a650-f2c1-40d6-97cd-32391962e685 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:34.659460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816a650-f2c1-40d6-97cd-32391962e685 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:34.661109 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9816a650-f2c1-40d6-97cd-32391962e685 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:34.663892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:34.674374 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5c598-c235-4892-8963-c47771c7b699 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-9816a650-f2c1-40d6-97cd-32391962e685 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:34.675505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:34.676520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:34.677364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 438/884] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:47:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:34.678170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f58c0-2c49-4fdf-af05-306000875284 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:34.678899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f58c0-2c49-4fdf-af05-306000875284 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:34.678899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f58c0-2c49-4fdf-af05-306000875284 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:34.679891 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 439/885] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:34.684365 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f58c0-2c49-4fdf-af05-306000875284 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:34.720967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e5199-b46a-4966-a3bf-c15919874f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:34.722428 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e5199-b46a-4966-a3bf-c15919874f9a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:34.724000 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e5199-b46a-4966-a3bf-c15919874f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:34.724573 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e5199-b46a-4966-a3bf-c15919874f9a None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:34.725659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e5199-b46a-4966-a3bf-c15919874f9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:34.726185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e5199-b46a-4966-a3bf-c15919874f9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:34.727828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e5199-b46a-4966-a3bf-c15919874f9a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:34.746685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 447/886] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:34.746685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584aaa-a7fd-42de-aee9-e4bb03c16f13 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:34.746685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584aaa-a7fd-42de-aee9-e4bb03c16f13 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:34.746685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584aaa-a7fd-42de-aee9-e4bb03c16f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:34.763006 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-2a584aaa-a7fd-42de-aee9-e4bb03c16f13 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:34.764174 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-2a584aaa-a7fd-42de-aee9-e4bb03c16f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:34.804899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f58c0-2c49-4fdf-af05-306000875284 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:34.835832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 448/887] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:34.836728 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a584aaa-a7fd-42de-aee9-e4bb03c16f13 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:34.858457 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbda6a-0297-400a-b57c-256dfcd3182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:34.859156 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbda6a-0297-400a-b57c-256dfcd3182e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:34.859409 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbda6a-0297-400a-b57c-256dfcd3182e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:34.859736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbda6a-0297-400a-b57c-256dfcd3182e None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:34.860819 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbda6a-0297-400a-b57c-256dfcd3182e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:34.860819 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbda6a-0297-400a-b57c-256dfcd3182e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:34.862132 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbda6a-0297-400a-b57c-256dfcd3182e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:34.867330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 440/888] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:47:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:34.868371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbda6a-0297-400a-b57c-256dfcd3182e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:34.924857 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:34.936967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:34.954978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a584aaa-a7fd-42de-aee9-e4bb03c16f13 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:34.983857 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbda6a-0297-400a-b57c-256dfcd3182e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:34.989758 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eb8e4-365e-41db-9209-1bb2c9eb3966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:34.990596 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eb8e4-365e-41db-9209-1bb2c9eb3966 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:34.990872 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eb8e4-365e-41db-9209-1bb2c9eb3966 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:34.991143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55eb8e4-365e-41db-9209-1bb2c9eb3966 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:34.998636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55eb8e4-365e-41db-9209-1bb2c9eb3966 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:34.999998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55eb8e4-365e-41db-9209-1bb2c9eb3966 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-f55eb8e4-365e-41db-9209-1bb2c9eb3966 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:35.015832 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:35.016982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 449/889] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 19:47:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:35.018536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eb8e4-365e-41db-9209-1bb2c9eb3966 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:35.027589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de734a-635c-4c61-8c4b-3a0621da116d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:35.028849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de734a-635c-4c61-8c4b-3a0621da116d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:35.029133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de734a-635c-4c61-8c4b-3a0621da116d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:35.029407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de734a-635c-4c61-8c4b-3a0621da116d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:35.030430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de734a-635c-4c61-8c4b-3a0621da116d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:35.032139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de734a-635c-4c61-8c4b-3a0621da116d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-e0de734a-635c-4c61-8c4b-3a0621da116d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:35.044849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:35.045911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:35.046732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 441/890] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 19:47:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:35.048079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de734a-635c-4c61-8c4b-3a0621da116d None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:35.140240 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55eb8e4-365e-41db-9209-1bb2c9eb3966 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:35.166301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de734a-635c-4c61-8c4b-3a0621da116d None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:35.176472 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:35.176888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614fb4b3-f911-466f-b720-5569be34d7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:35.177767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fb4b3-f911-466f-b720-5569be34d7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:35.178056 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fb4b3-f911-466f-b720-5569be34d7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:35.178527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614fb4b3-f911-466f-b720-5569be34d7c3 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:35.179287 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614fb4b3-f911-466f-b720-5569be34d7c3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:35.179499 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614fb4b3-f911-466f-b720-5569be34d7c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:35.180896 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614fb4b3-f911-466f-b720-5569be34d7c3 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:35.186606 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 450/891] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:47:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:35.187311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614fb4b3-f911-466f-b720-5569be34d7c3 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:35.197546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:35.206624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef30148-0c14-4a20-9885-3a8747851c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:35.207301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30148-0c14-4a20-9885-3a8747851c90 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:35.207639 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30148-0c14-4a20-9885-3a8747851c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:35.207922 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef30148-0c14-4a20-9885-3a8747851c90 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:35.208985 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef30148-0c14-4a20-9885-3a8747851c90 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:35.209298 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef30148-0c14-4a20-9885-3a8747851c90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:35.210770 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef30148-0c14-4a20-9885-3a8747851c90 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:35.234619 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 442/892] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 19:47:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:35.234619 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef30148-0c14-4a20-9885-3a8747851c90 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:35.298751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614fb4b3-f911-466f-b720-5569be34d7c3 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:35.341727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f03b66-663d-4f51-97d8-0033977044c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:35.341727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f03b66-663d-4f51-97d8-0033977044c9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:35.341727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f03b66-663d-4f51-97d8-0033977044c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:35.341727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f03b66-663d-4f51-97d8-0033977044c9 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:35.342139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f03b66-663d-4f51-97d8-0033977044c9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:35.342139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f03b66-663d-4f51-97d8-0033977044c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:35.342139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f03b66-663d-4f51-97d8-0033977044c9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:35.347023 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef30148-0c14-4a20-9885-3a8747851c90 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:35.349041 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 451/893] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 19:47:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:35.349041 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f03b66-663d-4f51-97d8-0033977044c9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:35.387886 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f992557-2535-492a-a782-4fad1873630a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:35.388576 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f992557-2535-492a-a782-4fad1873630a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:35.388866 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f992557-2535-492a-a782-4fad1873630a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:35.389141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f992557-2535-492a-a782-4fad1873630a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:35.389943 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f992557-2535-492a-a782-4fad1873630a None admin] RBAC: Adding query filter params (interface=public, service_id=563b90c5c72e4896ad9b9975c744332d, region_id=RegionOne) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:35.390255 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f992557-2535-492a-a782-4fad1873630a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:35.391651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f992557-2535-492a-a782-4fad1873630a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:35.401312 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 443/894] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 19:47:35 2022] GET /identity/v3/endpoints?service_id=563b90c5c72e4896ad9b9975c744332d&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:35.401312 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f992557-2535-492a-a782-4fad1873630a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:35.435906 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:35.440194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:35.468356 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f03b66-663d-4f51-97d8-0033977044c9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:35.504532 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebccce-5c95-4565-b2c6-284311f04936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:35.505186 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebccce-5c95-4565-b2c6-284311f04936 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:35.505473 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebccce-5c95-4565-b2c6-284311f04936 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:35.505751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebccce-5c95-4565-b2c6-284311f04936 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:35.516884 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f992557-2535-492a-a782-4fad1873630a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:35.554827 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2b118-8f61-434a-937a-01a31d319862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:35.555401 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2b118-8f61-434a-937a-01a31d319862 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:35.555681 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2b118-8f61-434a-937a-01a31d319862 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:35.555954 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2b118-8f61-434a-937a-01a31d319862 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:35.556808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2b118-8f61-434a-937a-01a31d319862 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:35.557108 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2b118-8f61-434a-937a-01a31d319862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:35.559098 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2b118-8f61-434a-937a-01a31d319862 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:35.563667 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 444/895] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 19:47:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:35.564477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2b118-8f61-434a-937a-01a31d319862 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:35.605575 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebccce-5c95-4565-b2c6-284311f04936 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:35.607079 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebccce-5c95-4565-b2c6-284311f04936 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:35.691916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:35.701012 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2b118-8f61-434a-937a-01a31d319862 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:35.714319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:35.743404 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8458e1-2bfa-4999-8f24-cb59a6094f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:35.744237 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8458e1-2bfa-4999-8f24-cb59a6094f52 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:35.745016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8458e1-2bfa-4999-8f24-cb59a6094f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:35.745016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8458e1-2bfa-4999-8f24-cb59a6094f52 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:35.746602 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8458e1-2bfa-4999-8f24-cb59a6094f52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:35.748117 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8458e1-2bfa-4999-8f24-cb59a6094f52 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:35.764435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 445/896] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 19:47:35 2022] GET /identity/v3/services => generated 1268 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:47:35.770168 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 452/897] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:35 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:47:35.776000 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8458e1-2bfa-4999-8f24-cb59a6094f52 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:35.845117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0f12-7f7e-4961-b092-eaa882f28ca2 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:35.845329 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0f12-7f7e-4961-b092-eaa882f28ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:35.845603 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0f12-7f7e-4961-b092-eaa882f28ca2 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:35.847260 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 453/898] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:35.852815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b864f-33ba-423f-ab4f-40ef0883be21 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:35.853161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b864f-33ba-423f-ab4f-40ef0883be21 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:35.853382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b864f-33ba-423f-ab4f-40ef0883be21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:35.871059 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-a55b864f-33ba-423f-ab4f-40ef0883be21 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:35.872269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-a55b864f-33ba-423f-ab4f-40ef0883be21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:35.906469 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8458e1-2bfa-4999-8f24-cb59a6094f52 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:35.960569 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddf6f2-0a47-4465-bd15-08805240a1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:35.961382 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddf6f2-0a47-4465-bd15-08805240a1ff None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:35.961661 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddf6f2-0a47-4465-bd15-08805240a1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:35.961930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ddf6f2-0a47-4465-bd15-08805240a1ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:35.962032 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 454/899] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:35.978003 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddf6f2-0a47-4465-bd15-08805240a1ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:35.979556 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ddf6f2-0a47-4465-bd15-08805240a1ff None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:35.987196 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b864f-33ba-423f-ab4f-40ef0883be21 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-83ddf6f2-0a47-4465-bd15-08805240a1ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:36.001059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:36.002034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:36.002946 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:36.002946 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:36.002946 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:36.002946 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:36.003604 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 446/900] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:36.009260 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddf6f2-0a47-4465-bd15-08805240a1ff None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:36.025947 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:36.107614 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:36.111332 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b864f-33ba-423f-ab4f-40ef0883be21 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:36.130346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ddf6f2-0a47-4465-bd15-08805240a1ff None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:36.154986 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228343fb-1777-491c-8434-7aaeb20d2e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:36.155845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228343fb-1777-491c-8434-7aaeb20d2e22 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.156176 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228343fb-1777-491c-8434-7aaeb20d2e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.156359 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228343fb-1777-491c-8434-7aaeb20d2e22 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.157019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228343fb-1777-491c-8434-7aaeb20d2e22 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:36.158389 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228343fb-1777-491c-8434-7aaeb20d2e22 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-228343fb-1777-491c-8434-7aaeb20d2e22 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:36.176834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:36.177907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:47:36.178752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:36.180985 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c87ca-f389-4606-97b9-8f9aae7f1acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:36.181908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c87ca-f389-4606-97b9-8f9aae7f1acd None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.182391 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c87ca-f389-4606-97b9-8f9aae7f1acd None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.183439 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c87ca-f389-4606-97b9-8f9aae7f1acd None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.184463 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 455/901] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 19:47:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:36.184674 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c87ca-f389-4606-97b9-8f9aae7f1acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:36.185194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c87ca-f389-4606-97b9-8f9aae7f1acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:36.185432 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228343fb-1777-491c-8434-7aaeb20d2e22 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:36.187009 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c87ca-f389-4606-97b9-8f9aae7f1acd None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:36.201520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 447/902] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:36.206542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c87ca-f389-4606-97b9-8f9aae7f1acd None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:36.287105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:36.323327 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c87ca-f389-4606-97b9-8f9aae7f1acd None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:36.324502 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228343fb-1777-491c-8434-7aaeb20d2e22 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:36.359877 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2121542-c1a9-49c4-a8aa-d5ee8ef0688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:36.361050 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2121542-c1a9-49c4-a8aa-d5ee8ef0688d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.361486 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2121542-c1a9-49c4-a8aa-d5ee8ef0688d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.361972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2121542-c1a9-49c4-a8aa-d5ee8ef0688d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.368195 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abee4c-41df-42cc-a804-e88e98e635f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:36.369007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abee4c-41df-42cc-a804-e88e98e635f9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.369295 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abee4c-41df-42cc-a804-e88e98e635f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.369710 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abee4c-41df-42cc-a804-e88e98e635f9 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.370312 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abee4c-41df-42cc-a804-e88e98e635f9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:36.371163 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abee4c-41df-42cc-a804-e88e98e635f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:36.371261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2121542-c1a9-49c4-a8aa-d5ee8ef0688d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:36.372666 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abee4c-41df-42cc-a804-e88e98e635f9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:36.374322 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2121542-c1a9-49c4-a8aa-d5ee8ef0688d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:36.382521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 456/903] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 19:47:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:36.383420 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abee4c-41df-42cc-a804-e88e98e635f9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-d2121542-c1a9-49c4-a8aa-d5ee8ef0688d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:36.389615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:36.390508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:36.391379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:36.391379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:36.391379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:36.391379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:36.391379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:36.391379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:36.391379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:36.391379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:36.395903 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 448/904] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:36.396798 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2121542-c1a9-49c4-a8aa-d5ee8ef0688d None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:36.412712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:36.503247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abee4c-41df-42cc-a804-e88e98e635f9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:36.530405 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2121542-c1a9-49c4-a8aa-d5ee8ef0688d None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:36.544979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27754d89-8913-423b-b721-d5fec126ad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:36.545647 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27754d89-8913-423b-b721-d5fec126ad62 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.545933 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27754d89-8913-423b-b721-d5fec126ad62 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.546211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27754d89-8913-423b-b721-d5fec126ad62 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.547181 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27754d89-8913-423b-b721-d5fec126ad62 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:36.547487 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27754d89-8913-423b-b721-d5fec126ad62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:36.549758 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27754d89-8913-423b-b721-d5fec126ad62 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:36.560519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 457/905] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 19:47:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:36.562958 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27754d89-8913-423b-b721-d5fec126ad62 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:36.571547 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:36.582773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38bc9d-9f9b-4eef-96cd-ec3c60cabb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:36.589899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38bc9d-9f9b-4eef-96cd-ec3c60cabb0b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.589899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38bc9d-9f9b-4eef-96cd-ec3c60cabb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.589899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38bc9d-9f9b-4eef-96cd-ec3c60cabb0b None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.589899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38bc9d-9f9b-4eef-96cd-ec3c60cabb0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:36.589899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38bc9d-9f9b-4eef-96cd-ec3c60cabb0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:36.589899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38bc9d-9f9b-4eef-96cd-ec3c60cabb0b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:36.605560 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 449/906] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:36.613121 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38bc9d-9f9b-4eef-96cd-ec3c60cabb0b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:36.665605 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:36.676302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27754d89-8913-423b-b721-d5fec126ad62 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:36.719429 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115a582-dedc-438c-abd3-8a0bdc082669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:36.720609 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115a582-dedc-438c-abd3-8a0bdc082669 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.721550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115a582-dedc-438c-abd3-8a0bdc082669 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.721984 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115a582-dedc-438c-abd3-8a0bdc082669 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.723210 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115a582-dedc-438c-abd3-8a0bdc082669 None admin] RBAC: Adding query filter params (interface=public, service_id=2c6724f8a7c240f085a75e4dc3131677, region_id=RegionOne) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:36.723670 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115a582-dedc-438c-abd3-8a0bdc082669 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:36.725878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115a582-dedc-438c-abd3-8a0bdc082669 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:36.742365 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 458/907] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 19:47:36 2022] GET /identity/v3/endpoints?service_id=2c6724f8a7c240f085a75e4dc3131677&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:36.744770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115a582-dedc-438c-abd3-8a0bdc082669 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:36.746628 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38bc9d-9f9b-4eef-96cd-ec3c60cabb0b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:36.785158 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c362b-2a8c-4778-a9ee-952e7b11cd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:36.786167 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c362b-2a8c-4778-a9ee-952e7b11cd90 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.786770 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c362b-2a8c-4778-a9ee-952e7b11cd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.786986 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c362b-2a8c-4778-a9ee-952e7b11cd90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.787865 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c362b-2a8c-4778-a9ee-952e7b11cd90 None admin] RBAC: Adding query filter params (role.id=f7602c7b55f44757bd1e1a99ee31b24d, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=f4b549638f5442b6a1be558df1686c5d) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:36.788197 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c362b-2a8c-4778-a9ee-952e7b11cd90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:36.789724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c362b-2a8c-4778-a9ee-952e7b11cd90 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:36.797225 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 450/908] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:36 2022] GET /identity/v3/role_assignments?user.id=f4b549638f5442b6a1be558df1686c5d&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=f7602c7b55f44757bd1e1a99ee31b24d => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:36.873206 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115a582-dedc-438c-abd3-8a0bdc082669 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:36.912708 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b865cea-fd37-44dc-b3b5-62dc9787275d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:36.913768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b865cea-fd37-44dc-b3b5-62dc9787275d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:36.914377 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b865cea-fd37-44dc-b3b5-62dc9787275d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:36.914857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b865cea-fd37-44dc-b3b5-62dc9787275d None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:36.915807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b865cea-fd37-44dc-b3b5-62dc9787275d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:36.917476 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b865cea-fd37-44dc-b3b5-62dc9787275d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:36.924807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 459/909] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 19:47:36 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:47:37.874757 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff404f-5e89-42eb-8455-2388d1e4f6c9 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:37.875612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff404f-5e89-42eb-8455-2388d1e4f6c9 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:37.876293 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff404f-5e89-42eb-8455-2388d1e4f6c9 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:37.878085 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 451/910] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:37.883802 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64d1bc-0dfe-42e2-986a-8299ce7f97f6 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:37.884607 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64d1bc-0dfe-42e2-986a-8299ce7f97f6 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:37.885025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64d1bc-0dfe-42e2-986a-8299ce7f97f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:37.903235 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-fb64d1bc-0dfe-42e2-986a-8299ce7f97f6 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:37.904621 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-fb64d1bc-0dfe-42e2-986a-8299ce7f97f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:37.919608 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:37.988487 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 460/911] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:38.011496 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff404f-5e89-42eb-8455-2388d1e4f6c9 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:38.081469 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:38.125813 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff404f-5e89-42eb-8455-2388d1e4f6c9 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:38.144276 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c828-c571-4dd6-b45a-9f6822a868d8 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:38.144600 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c828-c571-4dd6-b45a-9f6822a868d8 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:38.144875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c828-c571-4dd6-b45a-9f6822a868d8 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:38.146469 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 461/912] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:38.152280 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f45da-fbab-4741-9d2a-f6e551555cba None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:38.152601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f45da-fbab-4741-9d2a-f6e551555cba None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:38.152877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f45da-fbab-4741-9d2a-f6e551555cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:38.163556 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e5d4c-9353-46e2-9489-7be522ca3851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:38.164459 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e5d4c-9353-46e2-9489-7be522ca3851 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:38.164890 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e5d4c-9353-46e2-9489-7be522ca3851 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:38.165310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e5d4c-9353-46e2-9489-7be522ca3851 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:38.166252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e5d4c-9353-46e2-9489-7be522ca3851 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:38.167898 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e5d4c-9353-46e2-9489-7be522ca3851 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:38.170016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-ac7f45da-fbab-4741-9d2a-f6e551555cba None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:38.172002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-ac7f45da-fbab-4741-9d2a-f6e551555cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:38.235317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 452/913] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:47:38 2022] POST /identity/v3/services => generated 241 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:38.260267 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 462/914] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:38.285302 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e5d4c-9353-46e2-9489-7be522ca3851 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:38.421638 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e5d4c-9353-46e2-9489-7be522ca3851 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:38.436833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:38.459382 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47edff-d26c-4c80-b11b-5b36cfaee6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:38.460540 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47edff-d26c-4c80-b11b-5b36cfaee6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:38.461047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47edff-d26c-4c80-b11b-5b36cfaee6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:38.461561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47edff-d26c-4c80-b11b-5b36cfaee6a0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:38.462415 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47edff-d26c-4c80-b11b-5b36cfaee6a0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:38.464130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47edff-d26c-4c80-b11b-5b36cfaee6a0 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-0e47edff-d26c-4c80-b11b-5b36cfaee6a0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:38.476981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:38.478133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 19:47:38.478980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:38.482053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 453/915] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:47:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:38.487775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f45da-fbab-4741-9d2a-f6e551555cba None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:38.571035 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:38.597519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f45da-fbab-4741-9d2a-f6e551555cba None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:38.633206 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69004be6-d08c-44e4-96bc-dcf969fc4b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:38.633890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69004be6-d08c-44e4-96bc-dcf969fc4b7f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:38.634175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69004be6-d08c-44e4-96bc-dcf969fc4b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:38.634443 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69004be6-d08c-44e4-96bc-dcf969fc4b7f None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:38.635171 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69004be6-d08c-44e4-96bc-dcf969fc4b7f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:38.635477 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69004be6-d08c-44e4-96bc-dcf969fc4b7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:38.637437 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69004be6-d08c-44e4-96bc-dcf969fc4b7f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:38.642982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 463/916] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:47:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:38.648362 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47edff-d26c-4c80-b11b-5b36cfaee6a0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:38.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47edff-d26c-4c80-b11b-5b36cfaee6a0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:38.799900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af84a9c-92c1-4ed0-b406-b169199057ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:38.801158 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af84a9c-92c1-4ed0-b406-b169199057ef None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:38.801638 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af84a9c-92c1-4ed0-b406-b169199057ef None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:38.802160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af84a9c-92c1-4ed0-b406-b169199057ef None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:38.803074 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af84a9c-92c1-4ed0-b406-b169199057ef None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:38.803561 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af84a9c-92c1-4ed0-b406-b169199057ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:38.805122 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af84a9c-92c1-4ed0-b406-b169199057ef None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:38.811130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 454/917] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 19:47:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:38.817661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69004be6-d08c-44e4-96bc-dcf969fc4b7f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:38.927957 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69004be6-d08c-44e4-96bc-dcf969fc4b7f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:38.973338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1707d-9e5f-47b5-84b6-a60a5ca5c171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:38.974360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1707d-9e5f-47b5-84b6-a60a5ca5c171 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:38.974819 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1707d-9e5f-47b5-84b6-a60a5ca5c171 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:38.975278 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1707d-9e5f-47b5-84b6-a60a5ca5c171 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:38.976551 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1707d-9e5f-47b5-84b6-a60a5ca5c171 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:38.978807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1707d-9e5f-47b5-84b6-a60a5ca5c171 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:38.993191 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:39.061342 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 464/918] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 19:47:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:39.173196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb8852-5b63-4643-b97c-d421d6a25073 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.173751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb8852-5b63-4643-b97c-d421d6a25073 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.173751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb8852-5b63-4643-b97c-d421d6a25073 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.175782 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 455/919] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:39.181630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9251e40-0b16-4425-8e70-df5a1022066d None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.181918 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9251e40-0b16-4425-8e70-df5a1022066d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.182203 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9251e40-0b16-4425-8e70-df5a1022066d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.202374 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-d9251e40-0b16-4425-8e70-df5a1022066d None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:39.203610 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-d9251e40-0b16-4425-8e70-df5a1022066d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:39.230002 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993c4e3-1fa5-43ba-a717-fc319c9fd64f None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.230397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993c4e3-1fa5-43ba-a717-fc319c9fd64f None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.230751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993c4e3-1fa5-43ba-a717-fc319c9fd64f None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.232519 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 456/920] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:47:39.238127 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911f68-ccfb-4a30-a9c1-4a9db37c17af None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.238482 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911f68-ccfb-4a30-a9c1-4a9db37c17af None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.238948 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0911f68-ccfb-4a30-a9c1-4a9db37c17af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.259841 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-b0911f68-ccfb-4a30-a9c1-4a9db37c17af None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:39.261641 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-b0911f68-ccfb-4a30-a9c1-4a9db37c17af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:39.291450 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 465/921] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:39.312001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:39.317105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9251e40-0b16-4425-8e70-df5a1022066d None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:39.354268 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 457/922] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:39.379996 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0911f68-ccfb-4a30-a9c1-4a9db37c17af None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:39.445542 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9251e40-0b16-4425-8e70-df5a1022066d None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:39.476715 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:39.483747 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa10bab-3fe0-4c31-92e9-d3d4ecf0a6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:39.485015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa10bab-3fe0-4c31-92e9-d3d4ecf0a6fb None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.485480 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa10bab-3fe0-4c31-92e9-d3d4ecf0a6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.485899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa10bab-3fe0-4c31-92e9-d3d4ecf0a6fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.490950 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0911f68-ccfb-4a30-a9c1-4a9db37c17af None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:39.499010 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa10bab-3fe0-4c31-92e9-d3d4ecf0a6fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-7aa10bab-3fe0-4c31-92e9-d3d4ecf0a6fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:39.507993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:39.509272 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:39.515263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 466/923] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 19:47:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:39.517689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa10bab-3fe0-4c31-92e9-d3d4ecf0a6fb None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:39.540252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcecd09-6d30-4d7c-9efb-0c54e87e6c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:39.541484 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcecd09-6d30-4d7c-9efb-0c54e87e6c72 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.541830 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcecd09-6d30-4d7c-9efb-0c54e87e6c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.542740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcecd09-6d30-4d7c-9efb-0c54e87e6c72 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.543473 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcecd09-6d30-4d7c-9efb-0c54e87e6c72 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:39.545505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcecd09-6d30-4d7c-9efb-0c54e87e6c72 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-2fcecd09-6d30-4d7c-9efb-0c54e87e6c72 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:39.563160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:39.564232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:39.565494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 458/924] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 19:47:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:39.570296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcecd09-6d30-4d7c-9efb-0c54e87e6c72 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:39.613585 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:39.654144 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa10bab-3fe0-4c31-92e9-d3d4ecf0a6fb None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:39.676246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcecd09-6d30-4d7c-9efb-0c54e87e6c72 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:39.696008 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81322a0-e965-42a2-b126-f839166b1e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:39.697084 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81322a0-e965-42a2-b126-f839166b1e86 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.697597 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81322a0-e965-42a2-b126-f839166b1e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.698068 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81322a0-e965-42a2-b126-f839166b1e86 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.699037 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81322a0-e965-42a2-b126-f839166b1e86 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:39.699536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81322a0-e965-42a2-b126-f839166b1e86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:39.701124 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81322a0-e965-42a2-b126-f839166b1e86 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:39.714935 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 467/925] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 19:47:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:39.716585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81322a0-e965-42a2-b126-f839166b1e86 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:39.720033 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a722709-80d7-44ed-804b-5fe6716db490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:39.721837 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a722709-80d7-44ed-804b-5fe6716db490 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.722329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a722709-80d7-44ed-804b-5fe6716db490 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.722819 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a722709-80d7-44ed-804b-5fe6716db490 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.723782 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a722709-80d7-44ed-804b-5fe6716db490 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:39.724281 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a722709-80d7-44ed-804b-5fe6716db490 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:39.726497 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a722709-80d7-44ed-804b-5fe6716db490 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:39.734122 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 459/926] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 19:47:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:39.739545 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a722709-80d7-44ed-804b-5fe6716db490 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:39.779218 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:39.848509 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81322a0-e965-42a2-b126-f839166b1e86 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:39.861397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:39.865706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a722709-80d7-44ed-804b-5fe6716db490 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:39.891150 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637bc03-a604-46d8-a7ea-fdcfbb1372e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:39.891959 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637bc03-a604-46d8-a7ea-fdcfbb1372e0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.892296 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637bc03-a604-46d8-a7ea-fdcfbb1372e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.892611 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637bc03-a604-46d8-a7ea-fdcfbb1372e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.907809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637bc03-a604-46d8-a7ea-fdcfbb1372e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:39.909284 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5c439-6d09-4ce5-993e-614b5344e765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:39.909440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637bc03-a604-46d8-a7ea-fdcfbb1372e0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:39.910486 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5c439-6d09-4ce5-993e-614b5344e765 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:39.910971 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5c439-6d09-4ce5-993e-614b5344e765 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:39.911593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5c439-6d09-4ce5-993e-614b5344e765 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:39.913000 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5c439-6d09-4ce5-993e-614b5344e765 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:39.913544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5c439-6d09-4ce5-993e-614b5344e765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:39.915107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5c439-6d09-4ce5-993e-614b5344e765 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:39.927960 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 460/927] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 19:47:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:39.929028 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5c439-6d09-4ce5-993e-614b5344e765 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-0637bc03-a604-46d8-a7ea-fdcfbb1372e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:39.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:39.933401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:39.934290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:39.934290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:39.934290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:39.934290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:39.943757 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 468/928] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:39.945936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637bc03-a604-46d8-a7ea-fdcfbb1372e0 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:40.038788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5c439-6d09-4ce5-993e-614b5344e765 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:40.064296 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:40.071071 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637bc03-a604-46d8-a7ea-fdcfbb1372e0 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:40.076494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684f2b3-35e8-41fd-a9cb-5239031085b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:40.077460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684f2b3-35e8-41fd-a9cb-5239031085b6 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:40.077754 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684f2b3-35e8-41fd-a9cb-5239031085b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:40.101072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2684f2b3-35e8-41fd-a9cb-5239031085b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:40.101072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684f2b3-35e8-41fd-a9cb-5239031085b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:40.101072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2684f2b3-35e8-41fd-a9cb-5239031085b6 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:40.111340 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156eb3b-cdcf-4743-bf55-2b399b1b875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:40.112223 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156eb3b-cdcf-4743-bf55-2b399b1b875a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:40.112792 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156eb3b-cdcf-4743-bf55-2b399b1b875a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:40.113226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156eb3b-cdcf-4743-bf55-2b399b1b875a None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:40.114133 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156eb3b-cdcf-4743-bf55-2b399b1b875a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:40.114577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156eb3b-cdcf-4743-bf55-2b399b1b875a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:40.116130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e156eb3b-cdcf-4743-bf55-2b399b1b875a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:40.133036 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 469/929] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:40.134559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156eb3b-cdcf-4743-bf55-2b399b1b875a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:40.204579 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 461/930] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 19:47:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:40.273302 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156eb3b-cdcf-4743-bf55-2b399b1b875a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:40.325280 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff789a-72fe-487a-bd31-33e310018a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:40.326619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff789a-72fe-487a-bd31-33e310018a4f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:40.327173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff789a-72fe-487a-bd31-33e310018a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:40.327652 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff789a-72fe-487a-bd31-33e310018a4f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:40.339495 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff789a-72fe-487a-bd31-33e310018a4f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:40.341441 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff789a-72fe-487a-bd31-33e310018a4f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-7cff789a-72fe-487a-bd31-33e310018a4f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:40.358646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:40.359651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:40.360638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:40.360638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:40.360638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:40.360638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:40.360638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:40.360638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:40.360638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:40.360638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:40.360638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:40.360638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:40.364505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 470/931] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 19:47:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:40.366330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684f2b3-35e8-41fd-a9cb-5239031085b6 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:40.392707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:40.485100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2684f2b3-35e8-41fd-a9cb-5239031085b6 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:40.522877 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb927ab3-7a9a-4e5c-8da0-a00893127e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:40.523832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb927ab3-7a9a-4e5c-8da0-a00893127e7d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:40.524269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb927ab3-7a9a-4e5c-8da0-a00893127e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:40.524721 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb927ab3-7a9a-4e5c-8da0-a00893127e7d None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:40.525797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb927ab3-7a9a-4e5c-8da0-a00893127e7d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:40.526323 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb927ab3-7a9a-4e5c-8da0-a00893127e7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:40.528075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb927ab3-7a9a-4e5c-8da0-a00893127e7d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:40.547820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 462/932] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 19:47:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:40.560361 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff789a-72fe-487a-bd31-33e310018a4f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:40.570093 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:40.637211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359e009-ab61-47ec-862d-ff58399e9ee4 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:40.637515 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359e009-ab61-47ec-862d-ff58399e9ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:40.637713 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359e009-ab61-47ec-862d-ff58399e9ee4 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:40.639335 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 463/933] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:40.644668 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c70c6-bbe8-434b-8d3b-1e8b07bbad48 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:40.644960 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c70c6-bbe8-434b-8d3b-1e8b07bbad48 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:40.645241 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c70c6-bbe8-434b-8d3b-1e8b07bbad48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:40.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-9f3c70c6-bbe8-434b-8d3b-1e8b07bbad48 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:40.664191 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-9f3c70c6-bbe8-434b-8d3b-1e8b07bbad48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:40.692139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff789a-72fe-487a-bd31-33e310018a4f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:40.732920 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43034-651f-4b94-8b88-23d50cc90e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:40.733780 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43034-651f-4b94-8b88-23d50cc90e11 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:40.734062 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43034-651f-4b94-8b88-23d50cc90e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:40.734335 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a43034-651f-4b94-8b88-23d50cc90e11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:40.745739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a43034-651f-4b94-8b88-23d50cc90e11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:40.747174 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a43034-651f-4b94-8b88-23d50cc90e11 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:40.756487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 464/934] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-93a43034-651f-4b94-8b88-23d50cc90e11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:40.771011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:40.771952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:40.772890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:40.772890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:40.772890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:40.772890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:40.777114 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 471/935] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:40.778344 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43034-651f-4b94-8b88-23d50cc90e11 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:40.778527 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c70c6-bbe8-434b-8d3b-1e8b07bbad48 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:40.794624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:40.868369 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:40.914944 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a43034-651f-4b94-8b88-23d50cc90e11 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:40.915701 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c70c6-bbe8-434b-8d3b-1e8b07bbad48 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:40.952984 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde554b-3e6b-4265-9472-e53f010f8dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:40.953808 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde554b-3e6b-4265-9472-e53f010f8dcd None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:40.954187 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde554b-3e6b-4265-9472-e53f010f8dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:40.954612 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde554b-3e6b-4265-9472-e53f010f8dcd None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:40.955441 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde554b-3e6b-4265-9472-e53f010f8dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:40.955844 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde554b-3e6b-4265-9472-e53f010f8dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:40.957727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde554b-3e6b-4265-9472-e53f010f8dcd None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:40.959694 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b8ace-cadc-46f7-840e-ec3ce2923e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:40.960635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b8ace-cadc-46f7-840e-ec3ce2923e9b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:40.960931 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b8ace-cadc-46f7-840e-ec3ce2923e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:40.961209 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b8ace-cadc-46f7-840e-ec3ce2923e9b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:40.961867 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b8ace-cadc-46f7-840e-ec3ce2923e9b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:40.963244 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901b8ace-cadc-46f7-840e-ec3ce2923e9b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:40.976576 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 472/936] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-901b8ace-cadc-46f7-840e-ec3ce2923e9b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:40.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:40.978107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:40.978941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:40.981024 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde554b-3e6b-4265-9472-e53f010f8dcd None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:40.988375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 465/937] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 19:47:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:40.988375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b8ace-cadc-46f7-840e-ec3ce2923e9b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:41.066426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:41.095165 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde554b-3e6b-4265-9472-e53f010f8dcd None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:41.100965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901b8ace-cadc-46f7-840e-ec3ce2923e9b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:41.120761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:41.132531 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b90df-c215-4981-b856-befe1f107d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:41.133691 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b90df-c215-4981-b856-befe1f107d9f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:41.133977 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b90df-c215-4981-b856-befe1f107d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:41.134252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b90df-c215-4981-b856-befe1f107d9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:41.140741 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94434ccb-f642-4d7a-a2cf-8ac6681fae30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:41.141631 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94434ccb-f642-4d7a-a2cf-8ac6681fae30 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:41.142067 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94434ccb-f642-4d7a-a2cf-8ac6681fae30 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:41.142655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94434ccb-f642-4d7a-a2cf-8ac6681fae30 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:41.143067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b90df-c215-4981-b856-befe1f107d9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:41.143701 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94434ccb-f642-4d7a-a2cf-8ac6681fae30 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:41.144156 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94434ccb-f642-4d7a-a2cf-8ac6681fae30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:41.144607 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b90df-c215-4981-b856-befe1f107d9f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:41.145985 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94434ccb-f642-4d7a-a2cf-8ac6681fae30 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:41.156996 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 466/938] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 19:47:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:41.158286 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94434ccb-f642-4d7a-a2cf-8ac6681fae30 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-528b90df-c215-4981-b856-befe1f107d9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:41.161087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:41.162243 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:41.163253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:41.163253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:41.163253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:41.163253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:41.163253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:41.163253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:41.163253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:41.163253 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:41.167404 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 473/939] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:41.168579 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b90df-c215-4981-b856-befe1f107d9f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:41.280754 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94434ccb-f642-4d7a-a2cf-8ac6681fae30 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:41.282318 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b90df-c215-4981-b856-befe1f107d9f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:41.319328 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa230d-0858-4e45-a1b5-9b062745f6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:41.319755 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4cd0f-3c24-4181-810c-2e933fdac7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:41.320442 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4cd0f-3c24-4181-810c-2e933fdac7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:41.320650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa230d-0858-4e45-a1b5-9b062745f6cb None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:41.320758 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4cd0f-3c24-4181-810c-2e933fdac7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:41.320981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4cd0f-3c24-4181-810c-2e933fdac7b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:41.321327 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa230d-0858-4e45-a1b5-9b062745f6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:41.321747 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa230d-0858-4e45-a1b5-9b062745f6cb None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:41.321865 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4cd0f-3c24-4181-810c-2e933fdac7b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:41.322126 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4cd0f-3c24-4181-810c-2e933fdac7b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:41.322868 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa230d-0858-4e45-a1b5-9b062745f6cb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:41.323877 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa230d-0858-4e45-a1b5-9b062745f6cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:41.324278 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4cd0f-3c24-4181-810c-2e933fdac7b4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:41.325608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa230d-0858-4e45-a1b5-9b062745f6cb None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:41.328380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:41.336511 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 467/940] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 19:47:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:41.337784 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa230d-0858-4e45-a1b5-9b062745f6cb None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:41.342479 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 474/941] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:41.347681 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4cd0f-3c24-4181-810c-2e933fdac7b4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:41.404725 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:41.451258 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa230d-0858-4e45-a1b5-9b062745f6cb None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:41.463080 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4cd0f-3c24-4181-810c-2e933fdac7b4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:41.489639 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2a55a-61ac-4ae9-bbb3-1ddbf0210f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:41.490294 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2a55a-61ac-4ae9-bbb3-1ddbf0210f79 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:41.490576 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2a55a-61ac-4ae9-bbb3-1ddbf0210f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:41.490888 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2a55a-61ac-4ae9-bbb3-1ddbf0210f79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:41.491727 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2a55a-61ac-4ae9-bbb3-1ddbf0210f79 None admin] RBAC: Adding query filter params (interface=public, service_id=761235e214ff4a6590fa2c30e24e0878, region_id=RegionOne) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:41.492285 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2a55a-61ac-4ae9-bbb3-1ddbf0210f79 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:41.493440 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2a55a-61ac-4ae9-bbb3-1ddbf0210f79 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:41.503697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 468/942] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 19:47:41 2022] GET /identity/v3/endpoints?service_id=761235e214ff4a6590fa2c30e24e0878&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:41.503697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6b95a-04fa-4c7f-8909-13ded134a615 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:41.503697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6b95a-04fa-4c7f-8909-13ded134a615 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:41.503697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6b95a-04fa-4c7f-8909-13ded134a615 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:41.503697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 469/943] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:41.505796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8217abd-1652-4747-b213-fd4cd0e499e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:41.512384 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8217abd-1652-4747-b213-fd4cd0e499e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:41.512384 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8217abd-1652-4747-b213-fd4cd0e499e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:41.512384 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8217abd-1652-4747-b213-fd4cd0e499e5 None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/f4b549638f5442b6a1be558df1686c5d/roles/f7602c7b55f44757bd1e1a99ee31b24d` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:41.512665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6b95a-04fa-4c7f-8909-13ded134a615 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:41.512758 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8217abd-1652-4747-b213-fd4cd0e499e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=f4b549638f5442b6a1be558df1686c5d, role_id=f7602c7b55f44757bd1e1a99ee31b24d)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:41.512758 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8217abd-1652-4747-b213-fd4cd0e499e5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:41.575734 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:41.614918 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 475/944] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:47:41 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/f4b549638f5442b6a1be558df1686c5d/roles/f7602c7b55f44757bd1e1a99ee31b24d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:47:41.616036 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ada74-e3b5-4bb1-91e7-de5bc9c70d0b None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:41.616324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ada74-e3b5-4bb1-91e7-de5bc9c70d0b None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:41.616708 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ada74-e3b5-4bb1-91e7-de5bc9c70d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:41.626208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6b95a-04fa-4c7f-8909-13ded134a615 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:41.640245 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-001ada74-e3b5-4bb1-91e7-de5bc9c70d0b None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:41.642012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-001ada74-e3b5-4bb1-91e7-de5bc9c70d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:41.669959 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06c039-eb5f-4e6a-bdb9-b65e21b628d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:41.671101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06c039-eb5f-4e6a-bdb9-b65e21b628d4 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:41.671556 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06c039-eb5f-4e6a-bdb9-b65e21b628d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:41.672034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06c039-eb5f-4e6a-bdb9-b65e21b628d4 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:41.673157 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06c039-eb5f-4e6a-bdb9-b65e21b628d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:41.675367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06c039-eb5f-4e6a-bdb9-b65e21b628d4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:41.684252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 470/945] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 19:47:41 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:47:41.745095 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 476/946] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:41.768824 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06c039-eb5f-4e6a-bdb9-b65e21b628d4 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:41.890365 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06c039-eb5f-4e6a-bdb9-b65e21b628d4 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:41.920676 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:41.928776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355c63d-4cd3-45d2-a692-f891f0585296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:41.929436 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355c63d-4cd3-45d2-a692-f891f0585296 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:41.934217 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355c63d-4cd3-45d2-a692-f891f0585296 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:41.934217 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355c63d-4cd3-45d2-a692-f891f0585296 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:41.934217 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355c63d-4cd3-45d2-a692-f891f0585296 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:41.934217 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355c63d-4cd3-45d2-a692-f891f0585296 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:41.934217 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: WARNING keystone.server.flask.common [None req-c355c63d-4cd3-45d2-a692-f891f0585296 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 19:47:41.985717 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.sql.core [None req-c355c63d-4cd3-45d2-a692-f891f0585296 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:47:41.985717 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 19:47:41.985717 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [parameters: {'id': 'c28908130e664710ab123bff88547a9f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 19:47:41.985717 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80453) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-c355c63d-4cd3-45d2-a692-f891f0585296 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c28908130e664710ab123bff88547a9f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c28908130e664710ab123bff88547a9f. May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application conn.query(q) May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:47:42.002938 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application result.read() May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 19:47:42.003769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application next(self.gen) May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application yield resource May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application next(self.gen) May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise self.value May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application session.commit() May 19 19:47:42.004720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self.session.flush() May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application compat.raise_( May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:47:42.005662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:47:42.006549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application conn.query(q) May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application result.read() May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:47:42.007352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [parameters: {'id': 'c28908130e664710ab123bff88547a9f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:42.008190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:42.011004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:42.013248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:42.013248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 19:47:42.013248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 19:47:42.013248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:42.013248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:42.013248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 19:47:42.013248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 19:47:42.013248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c28908130e664710ab123bff88547a9f. May 19 19:47:42.013248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:42.013248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 471/947] 10.222.0.88 () {60 vars in 1140 bytes} [Thu May 19 19:47:41 2022] POST /identity/v3/projects => generated 262 bytes in 240 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:42.014519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ada74-e3b5-4bb1-91e7-de5bc9c70d0b None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:42.050765 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:42.121147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ada74-e3b5-4bb1-91e7-de5bc9c70d0b None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:42.157982 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305fe2ac-c82f-4eaf-ac22-4cfcddcabcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:42.159129 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fe2ac-c82f-4eaf-ac22-4cfcddcabcbf None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:42.159619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fe2ac-c82f-4eaf-ac22-4cfcddcabcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:42.160088 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305fe2ac-c82f-4eaf-ac22-4cfcddcabcbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:42.168271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fe2ac-c82f-4eaf-ac22-4cfcddcabcbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:42.169999 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305fe2ac-c82f-4eaf-ac22-4cfcddcabcbf None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-305fe2ac-c82f-4eaf-ac22-4cfcddcabcbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:42.184998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:42.185934 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:42.186833 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:42.186833 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:42.186833 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:42.186833 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:42.186833 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:42.186833 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:42.186833 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:42.186833 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:42.189314 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 477/948] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:42.196372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355c63d-4cd3-45d2-a692-f891f0585296 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:42.306009 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355c63d-4cd3-45d2-a692-f891f0585296 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:42.354116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18814e62-468b-47a3-beab-f6fcdf8d0221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:42.354116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18814e62-468b-47a3-beab-f6fcdf8d0221 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:42.354116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18814e62-468b-47a3-beab-f6fcdf8d0221 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:42.354116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18814e62-468b-47a3-beab-f6fcdf8d0221 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:42.354522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18814e62-468b-47a3-beab-f6fcdf8d0221 None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:42.354522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18814e62-468b-47a3-beab-f6fcdf8d0221 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:42.354522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18814e62-468b-47a3-beab-f6fcdf8d0221 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:42.378493 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 472/949] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:42.605769 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6b773-a806-4450-a77d-15c7d65f1542 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:42.606004 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6b773-a806-4450-a77d-15c7d65f1542 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:42.606259 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6b773-a806-4450-a77d-15c7d65f1542 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:42.607864 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 478/950] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:42.613283 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889bb7-42f7-477a-a5ae-be130d4c22b2 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:42.613577 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889bb7-42f7-477a-a5ae-be130d4c22b2 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:42.613860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8889bb7-42f7-477a-a5ae-be130d4c22b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:42.631902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-f8889bb7-42f7-477a-a5ae-be130d4c22b2 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:42.633479 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-f8889bb7-42f7-477a-a5ae-be130d4c22b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:42.678366 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:42.712463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 473/951] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:42.728396 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6b773-a806-4450-a77d-15c7d65f1542 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:42.829348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:42.838564 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae6b773-a806-4450-a77d-15c7d65f1542 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:42.878571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645bc9d-b2e8-403c-a958-ace1505f16f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:42.879710 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645bc9d-b2e8-403c-a958-ace1505f16f5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:42.880316 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645bc9d-b2e8-403c-a958-ace1505f16f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:42.880828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645bc9d-b2e8-403c-a958-ace1505f16f5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:42.881745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645bc9d-b2e8-403c-a958-ace1505f16f5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:42.883522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645bc9d-b2e8-403c-a958-ace1505f16f5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-e645bc9d-b2e8-403c-a958-ace1505f16f5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:42.896858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:42.897965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:47:42.898951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:42.902013 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 479/952] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:42.909129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8889bb7-42f7-477a-a5ae-be130d4c22b2 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:43.018253 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8889bb7-42f7-477a-a5ae-be130d4c22b2 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:43.055899 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006cdbc-f010-4cda-a2be-e79a86b176e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:43.056849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006cdbc-f010-4cda-a2be-e79a86b176e9 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:43.057296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006cdbc-f010-4cda-a2be-e79a86b176e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:43.057744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006cdbc-f010-4cda-a2be-e79a86b176e9 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:43.058589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006cdbc-f010-4cda-a2be-e79a86b176e9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:43.059048 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006cdbc-f010-4cda-a2be-e79a86b176e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:43.060577 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006cdbc-f010-4cda-a2be-e79a86b176e9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:43.066172 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 474/953] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:43.074031 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645bc9d-b2e8-403c-a958-ace1505f16f5 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:43.183238 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645bc9d-b2e8-403c-a958-ace1505f16f5 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:43.218522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1f2fd-477f-4492-b9cb-42979ae81144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:43.219183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1f2fd-477f-4492-b9cb-42979ae81144 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:43.219466 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1f2fd-477f-4492-b9cb-42979ae81144 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:43.219744 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1f2fd-477f-4492-b9cb-42979ae81144 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:43.220479 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1f2fd-477f-4492-b9cb-42979ae81144 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:43.220782 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1f2fd-477f-4492-b9cb-42979ae81144 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:43.222133 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1f2fd-477f-4492-b9cb-42979ae81144 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:43.232506 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 480/954] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:44.030700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ac17c-1c94-44e2-9efd-93802ed110b7 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.031255 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ac17c-1c94-44e2-9efd-93802ed110b7 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.031336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ac17c-1c94-44e2-9efd-93802ed110b7 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.033009 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 475/955] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:44.038797 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f4e80-5749-4174-91da-59925bf30950 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.039087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f4e80-5749-4174-91da-59925bf30950 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.039365 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8f4e80-5749-4174-91da-59925bf30950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.058539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-5a8f4e80-5749-4174-91da-59925bf30950 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:44.059829 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-5a8f4e80-5749-4174-91da-59925bf30950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:44.062720 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea436f-f033-499a-8ebf-77eb0219c670 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.063000 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea436f-f033-499a-8ebf-77eb0219c670 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.063273 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea436f-f033-499a-8ebf-77eb0219c670 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.064993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 476/956] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:47:44.071050 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b3b33-b334-426b-abd9-c7162eb7a162 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.071363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b3b33-b334-426b-abd9-c7162eb7a162 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.071601 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b3b33-b334-426b-abd9-c7162eb7a162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.084944 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:44.095707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-7d6b3b33-b334-426b-abd9-c7162eb7a162 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:44.097607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-7d6b3b33-b334-426b-abd9-c7162eb7a162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:44.136482 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:44.144404 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 481/957] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:44.169408 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f4e80-5749-4174-91da-59925bf30950 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:44.177584 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 477/958] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:44.201663 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b3b33-b334-426b-abd9-c7162eb7a162 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:44.282758 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8f4e80-5749-4174-91da-59925bf30950 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:44.312471 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b3b33-b334-426b-abd9-c7162eb7a162 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:44.323753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da904b48-8eae-4ade-8e84-f899f2330003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:44.324993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da904b48-8eae-4ade-8e84-f899f2330003 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.325513 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da904b48-8eae-4ade-8e84-f899f2330003 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.325995 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da904b48-8eae-4ade-8e84-f899f2330003 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.340019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da904b48-8eae-4ade-8e84-f899f2330003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-da904b48-8eae-4ade-8e84-f899f2330003 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:44.349630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:44.350659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:47:44.351507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:44.360007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 482/959] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 19:47:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:44.361314 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da904b48-8eae-4ade-8e84-f899f2330003 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:44.362498 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753c141-8b4c-4495-aaaf-1234dec06046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:44.363317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753c141-8b4c-4495-aaaf-1234dec06046 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.363597 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753c141-8b4c-4495-aaaf-1234dec06046 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.363868 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753c141-8b4c-4495-aaaf-1234dec06046 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.364553 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753c141-8b4c-4495-aaaf-1234dec06046 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:44.366066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c753c141-8b4c-4495-aaaf-1234dec06046 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:44.379038 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-c753c141-8b4c-4495-aaaf-1234dec06046 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:44.380089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:44.381397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:44.382256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 478/960] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 19:47:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:44.387396 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753c141-8b4c-4495-aaaf-1234dec06046 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:44.420439 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:44.482019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da904b48-8eae-4ade-8e84-f899f2330003 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:44.493412 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753c141-8b4c-4495-aaaf-1234dec06046 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:44.520992 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4786c3-6535-4ded-9531-4534f5bb7e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:44.522102 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4786c3-6535-4ded-9531-4534f5bb7e69 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.522626 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4786c3-6535-4ded-9531-4534f5bb7e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.523125 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4786c3-6535-4ded-9531-4534f5bb7e69 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.524183 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4786c3-6535-4ded-9531-4534f5bb7e69 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:44.524799 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4786c3-6535-4ded-9531-4534f5bb7e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:44.527100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4786c3-6535-4ded-9531-4534f5bb7e69 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:44.529974 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cabf22-fa3d-4b25-a295-4853d599b5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:44.530621 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cabf22-fa3d-4b25-a295-4853d599b5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.530904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cabf22-fa3d-4b25-a295-4853d599b5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.531178 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cabf22-fa3d-4b25-a295-4853d599b5b5 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.531881 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cabf22-fa3d-4b25-a295-4853d599b5b5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:44.532191 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cabf22-fa3d-4b25-a295-4853d599b5b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:44.533822 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cabf22-fa3d-4b25-a295-4853d599b5b5 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:44.542927 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 479/961] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 19:47:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:44.543107 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 483/962] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 19:47:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:44.544420 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cabf22-fa3d-4b25-a295-4853d599b5b5 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:44.545505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4786c3-6535-4ded-9531-4534f5bb7e69 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:44.658917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:44.661206 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:44.666053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cabf22-fa3d-4b25-a295-4853d599b5b5 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:44.692984 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4786c3-6535-4ded-9531-4534f5bb7e69 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:44.706273 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d2533-8dcf-46d8-abe0-203cd2c68874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:44.707078 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d2533-8dcf-46d8-abe0-203cd2c68874 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.707374 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d2533-8dcf-46d8-abe0-203cd2c68874 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.707655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d2533-8dcf-46d8-abe0-203cd2c68874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.723059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d2533-8dcf-46d8-abe0-203cd2c68874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:44.724867 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d2533-8dcf-46d8-abe0-203cd2c68874 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:44.733804 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb8458-aeff-44a0-baa5-fd6aa0aef5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:44.734738 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb8458-aeff-44a0-baa5-fd6aa0aef5cd None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.735230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb8458-aeff-44a0-baa5-fd6aa0aef5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.735809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb8458-aeff-44a0-baa5-fd6aa0aef5cd None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.736838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb8458-aeff-44a0-baa5-fd6aa0aef5cd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:44.737371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb8458-aeff-44a0-baa5-fd6aa0aef5cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:44.739019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb8458-aeff-44a0-baa5-fd6aa0aef5cd None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-480d2533-8dcf-46d8-abe0-203cd2c68874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:44.746171 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:44.747128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:44.748093 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:44.748093 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:44.748093 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:44.748093 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:44.750086 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 484/963] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 19:47:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:44.754614 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 480/964] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:44.755836 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb8458-aeff-44a0-baa5-fd6aa0aef5cd None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:44.755983 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d2533-8dcf-46d8-abe0-203cd2c68874 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:44.867666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d2533-8dcf-46d8-abe0-203cd2c68874 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:44.868987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb8458-aeff-44a0-baa5-fd6aa0aef5cd None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:44.905732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f093240-f905-4b81-804a-8ddaaa5b7457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:44.906389 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f093240-f905-4b81-804a-8ddaaa5b7457 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.906669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f093240-f905-4b81-804a-8ddaaa5b7457 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.906939 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f093240-f905-4b81-804a-8ddaaa5b7457 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.907653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f093240-f905-4b81-804a-8ddaaa5b7457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:44.907951 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f093240-f905-4b81-804a-8ddaaa5b7457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:44.909331 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71187395-501c-43df-a1ce-95cd99587392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:44.909495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f093240-f905-4b81-804a-8ddaaa5b7457 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:44.909952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187395-501c-43df-a1ce-95cd99587392 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.910239 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187395-501c-43df-a1ce-95cd99587392 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.910519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187395-501c-43df-a1ce-95cd99587392 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.911271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71187395-501c-43df-a1ce-95cd99587392 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:44.912651 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71187395-501c-43df-a1ce-95cd99587392 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:44.920940 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:44.922571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 481/965] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:44.923823 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15186d9-9ec7-4a35-80d1-1d6e004e8c85 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:44.924112 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15186d9-9ec7-4a35-80d1-1d6e004e8c85 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:44.924416 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15186d9-9ec7-4a35-80d1-1d6e004e8c85 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:44.926035 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 482/966] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:44.929277 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15186d9-9ec7-4a35-80d1-1d6e004e8c85 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:44.940734 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:45.032707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15186d9-9ec7-4a35-80d1-1d6e004e8c85 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:45.042319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 485/967] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 19:47:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:45.043890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2542b1-01e2-46d5-8af7-c6d4f5cb065a None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:45.044210 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2542b1-01e2-46d5-8af7-c6d4f5cb065a None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:45.044514 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2542b1-01e2-46d5-8af7-c6d4f5cb065a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:45.068555 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-cd2542b1-01e2-46d5-8af7-c6d4f5cb065a None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:45.070597 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-cd2542b1-01e2-46d5-8af7-c6d4f5cb065a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:45.094848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef66fc-69aa-4390-8ce5-db8c4b5ca006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:45.096552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef66fc-69aa-4390-8ce5-db8c4b5ca006 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:45.097128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef66fc-69aa-4390-8ce5-db8c4b5ca006 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:45.097632 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef66fc-69aa-4390-8ce5-db8c4b5ca006 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:45.112010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef66fc-69aa-4390-8ce5-db8c4b5ca006 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:45.113570 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef66fc-69aa-4390-8ce5-db8c4b5ca006 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-7cef66fc-69aa-4390-8ce5-db8c4b5ca006 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:45.130047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:45.131090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:45.132059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:45.132059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:45.132059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:45.132059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:45.132059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:45.132059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:45.132059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:45.132059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:45.132059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 19:47:45.132059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:45.136472 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 483/968] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 19:47:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:45.137492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef66fc-69aa-4390-8ce5-db8c4b5ca006 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:45.174329 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 486/969] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:45.195835 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2542b1-01e2-46d5-8af7-c6d4f5cb065a None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:45.219824 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:45.265613 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef66fc-69aa-4390-8ce5-db8c4b5ca006 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:45.318665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a1fa8-0730-4db2-9873-5d9a5164a8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:45.319665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a1fa8-0730-4db2-9873-5d9a5164a8d8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:45.319947 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a1fa8-0730-4db2-9873-5d9a5164a8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:45.320545 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996a1fa8-0730-4db2-9873-5d9a5164a8d8 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:45.321893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a1fa8-0730-4db2-9873-5d9a5164a8d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:45.322472 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a1fa8-0730-4db2-9873-5d9a5164a8d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:45.325155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996a1fa8-0730-4db2-9873-5d9a5164a8d8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:45.327653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2542b1-01e2-46d5-8af7-c6d4f5cb065a None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:45.352392 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 484/970] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 19:47:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:45.355745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a1fa8-0730-4db2-9873-5d9a5164a8d8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:45.358432 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:45.369929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceea512-fd25-47b9-b7b1-dfd665e5950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:45.370628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea512-fd25-47b9-b7b1-dfd665e5950c None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:45.373508 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea512-fd25-47b9-b7b1-dfd665e5950c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:45.373508 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea512-fd25-47b9-b7b1-dfd665e5950c None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:45.373508 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceea512-fd25-47b9-b7b1-dfd665e5950c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:45.373508 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceea512-fd25-47b9-b7b1-dfd665e5950c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:45.374830 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: WARNING keystone.server.flask.common [None req-6ceea512-fd25-47b9-b7b1-dfd665e5950c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 19:47:45.414788 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.sql.core [None req-6ceea512-fd25-47b9-b7b1-dfd665e5950c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:47:45.414788 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 19:47:45.414788 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [parameters: {'user_id': 'ace2b967b8514f73943485de5f6ad9a2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 19:47:45.414788 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80452) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-6ceea512-fd25-47b9-b7b1-dfd665e5950c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application conn.query(q) May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application result.read() May 19 19:47:45.425379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:47:45.426334 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application next(self.gen) May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application yield resource May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application next(self.gen) May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self.session.rollback() May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self.force_reraise() May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise self.value May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application session.commit() May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 19:47:45.427297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._prepare_impl() May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self.session.flush() May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._flush(objects) May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application compat.raise_( May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application flush_context.execute() May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rec.execute(self) May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 19:47:45.428207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 19:47:45.429141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application result = self._query(query) May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application conn.query(q) May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application result.read() May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 19:47:45.430002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ace2b967b8514f73943485de5f6ad9a2', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:45.430892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 19:47:45.431665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 19:47:45.433952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 19:47:45.433952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 19:47:45.433952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 19:47:45.433952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 19:47:45.433952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 19:47:45.433952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:45.433952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 487/971] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 19:47:45 2022] POST /identity/v3/users => generated 160 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:45.433952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceea512-fd25-47b9-b7b1-dfd665e5950c None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:45.488088 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996a1fa8-0730-4db2-9873-5d9a5164a8d8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:45.513022 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:45.525563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b321e0-b248-4dbc-9168-04be9686ffcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:45.526792 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b321e0-b248-4dbc-9168-04be9686ffcd None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:45.527551 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b321e0-b248-4dbc-9168-04be9686ffcd None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:45.527676 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b321e0-b248-4dbc-9168-04be9686ffcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:45.552287 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceea512-fd25-47b9-b7b1-dfd665e5950c None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:45.589752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbc8f2-8b6f-4600-8301-bde2643d1bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:45.590562 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbc8f2-8b6f-4600-8301-bde2643d1bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:45.590828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbc8f2-8b6f-4600-8301-bde2643d1bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:45.591093 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbc8f2-8b6f-4600-8301-bde2643d1bf5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:45.598811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b321e0-b248-4dbc-9168-04be9686ffcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:45.598971 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbc8f2-8b6f-4600-8301-bde2643d1bf5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:45.600286 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b321e0-b248-4dbc-9168-04be9686ffcd None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:45.600615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbc8f2-8b6f-4600-8301-bde2643d1bf5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-94fbc8f2-8b6f-4600-8301-bde2643d1bf5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:45.615341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:45.616467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 488/972] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:47:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a264d80-48ae-48bd-994a-bd76adfa8357 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:45.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a264d80-48ae-48bd-994a-bd76adfa8357 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:45.618917 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a264d80-48ae-48bd-994a-bd76adfa8357 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:45.624376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 489/973] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:45.624376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a264d80-48ae-48bd-994a-bd76adfa8357 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:45.680101 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:45.722569 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 485/974] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:45 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:47:45.724043 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef818fe1-f9a7-49eb-9de6-36ce813bf835 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:45.724341 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef818fe1-f9a7-49eb-9de6-36ce813bf835 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:45.724642 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef818fe1-f9a7-49eb-9de6-36ce813bf835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:45.736864 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a264d80-48ae-48bd-994a-bd76adfa8357 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:45.741234 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-ef818fe1-f9a7-49eb-9de6-36ce813bf835 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:45.742426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-ef818fe1-f9a7-49eb-9de6-36ce813bf835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:45.760125 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:45.775235 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139aa41-cf39-4a21-af28-45555ada4225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:45.776002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139aa41-cf39-4a21-af28-45555ada4225 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:45.776287 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139aa41-cf39-4a21-af28-45555ada4225 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:45.776589 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139aa41-cf39-4a21-af28-45555ada4225 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:45.777397 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139aa41-cf39-4a21-af28-45555ada4225 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:45.777700 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139aa41-cf39-4a21-af28-45555ada4225 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:45.779119 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139aa41-cf39-4a21-af28-45555ada4225 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:45.816389 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 486/975] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:45.816389 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef818fe1-f9a7-49eb-9de6-36ce813bf835 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:45.819087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 490/976] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 19:47:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:45.838893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139aa41-cf39-4a21-af28-45555ada4225 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:45.952627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef818fe1-f9a7-49eb-9de6-36ce813bf835 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:45.972670 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139aa41-cf39-4a21-af28-45555ada4225 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:45.976840 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:45.992848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f666c4-2801-4f81-8657-94946f13dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:45.993653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f666c4-2801-4f81-8657-94946f13dd8d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:45.993931 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f666c4-2801-4f81-8657-94946f13dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:45.994200 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f666c4-2801-4f81-8657-94946f13dd8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:46.006422 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f666c4-2801-4f81-8657-94946f13dd8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:46.007919 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f666c4-2801-4f81-8657-94946f13dd8d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:46.018865 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30afc6-4218-4a4c-92da-96a17668a9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:46.020058 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30afc6-4218-4a4c-92da-96a17668a9c3 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:46.020625 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30afc6-4218-4a4c-92da-96a17668a9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:46.021098 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30afc6-4218-4a4c-92da-96a17668a9c3 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:46.022080 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30afc6-4218-4a4c-92da-96a17668a9c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:46.023665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30afc6-4218-4a4c-92da-96a17668a9c3 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-c0f666c4-2801-4f81-8657-94946f13dd8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:46.029766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:46.030671 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:46.031558 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:46.031558 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:46.031558 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:46.031558 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:46.031558 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 487/977] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:46.037798 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f666c4-2801-4f81-8657-94946f13dd8d None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:46.042761 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:46.134490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 491/978] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:47:45 2022] POST /identity/v3/services => generated 242 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:47:46.161700 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f666c4-2801-4f81-8657-94946f13dd8d None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:46.209955 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3d7ec-0114-42db-abd5-9a6ff8dff0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:46.210552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3d7ec-0114-42db-abd5-9a6ff8dff0ea None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:46.210898 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3d7ec-0114-42db-abd5-9a6ff8dff0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:46.211180 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3d7ec-0114-42db-abd5-9a6ff8dff0ea None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:46.211909 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3d7ec-0114-42db-abd5-9a6ff8dff0ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:46.212215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3d7ec-0114-42db-abd5-9a6ff8dff0ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:46.214208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3d7ec-0114-42db-abd5-9a6ff8dff0ea None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:46.232165 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 488/979] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:46.233501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30afc6-4218-4a4c-92da-96a17668a9c3 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:46.351576 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30afc6-4218-4a4c-92da-96a17668a9c3 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:46.387812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0d4a0-da37-4a8d-85a2-364ed0deae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:46.389049 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0d4a0-da37-4a8d-85a2-364ed0deae67 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:46.389538 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0d4a0-da37-4a8d-85a2-364ed0deae67 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:46.389953 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0d4a0-da37-4a8d-85a2-364ed0deae67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:46.397067 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0d4a0-da37-4a8d-85a2-364ed0deae67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:46.398680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d0d4a0-da37-4a8d-85a2-364ed0deae67 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-f9d0d4a0-da37-4a8d-85a2-364ed0deae67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:46.413546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:46.414619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:46.415724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:46.415724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:46.415724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:46.415724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:46.415724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:46.415724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:46.415724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:46.415724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:46.421911 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 492/980] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:46.428160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3d7ec-0114-42db-abd5-9a6ff8dff0ea None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:46.492487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:46.533230 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3d7ec-0114-42db-abd5-9a6ff8dff0ea None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:46.568748 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a9931-1609-4ec6-b0bd-ace24fb9a9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:46.569403 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a9931-1609-4ec6-b0bd-ace24fb9a9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:46.569684 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a9931-1609-4ec6-b0bd-ace24fb9a9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:46.569956 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a9931-1609-4ec6-b0bd-ace24fb9a9b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:46.570747 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a9931-1609-4ec6-b0bd-ace24fb9a9b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:46.571052 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a9931-1609-4ec6-b0bd-ace24fb9a9b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:46.572511 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a9931-1609-4ec6-b0bd-ace24fb9a9b6 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:46.585114 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 489/981] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:46.590919 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0d4a0-da37-4a8d-85a2-364ed0deae67 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:46.616994 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:46.694141 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d0d4a0-da37-4a8d-85a2-364ed0deae67 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:46.729743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d96475-60d4-473b-923c-49b591fa5836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:46.730397 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d96475-60d4-473b-923c-49b591fa5836 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:46.730679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d96475-60d4-473b-923c-49b591fa5836 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:46.730954 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d96475-60d4-473b-923c-49b591fa5836 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:46.731789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d96475-60d4-473b-923c-49b591fa5836 None admin] RBAC: Adding query filter params (role.id=f7602c7b55f44757bd1e1a99ee31b24d, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=f4b549638f5442b6a1be558df1686c5d) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:46.732099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d96475-60d4-473b-923c-49b591fa5836 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:46.733727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d96475-60d4-473b-923c-49b591fa5836 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:46.748448 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 493/982] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:46 2022] GET /identity/v3/role_assignments?user.id=f4b549638f5442b6a1be558df1686c5d&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=f7602c7b55f44757bd1e1a99ee31b24d => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:47.383531 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c29e-02d5-45ac-b567-7cbee28d9e00 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:47.384044 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c29e-02d5-45ac-b567-7cbee28d9e00 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:47.384153 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4c29e-02d5-45ac-b567-7cbee28d9e00 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:47.385679 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 490/983] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:47.390724 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ac3ac-02b9-49f4-8240-b3327cd3025a None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:47.391111 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ac3ac-02b9-49f4-8240-b3327cd3025a None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:47.391271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ac3ac-02b9-49f4-8240-b3327cd3025a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:47.408627 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-d09ac3ac-02b9-49f4-8240-b3327cd3025a None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:47.410118 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-d09ac3ac-02b9-49f4-8240-b3327cd3025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:47.481574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 494/984] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:47.503024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4c29e-02d5-45ac-b567-7cbee28d9e00 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:47.610840 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4c29e-02d5-45ac-b567-7cbee28d9e00 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:47.631737 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:47.646652 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf0e16-7253-4daa-a43f-d1a0493d2e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:47.647461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf0e16-7253-4daa-a43f-d1a0493d2e0b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:47.647749 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf0e16-7253-4daa-a43f-d1a0493d2e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:47.649252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf0e16-7253-4daa-a43f-d1a0493d2e0b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:47.649252 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf0e16-7253-4daa-a43f-d1a0493d2e0b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:47.650149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf0e16-7253-4daa-a43f-d1a0493d2e0b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-2edf0e16-7253-4daa-a43f-d1a0493d2e0b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:47.663188 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:47.664196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:47.665053 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:47.668491 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 491/985] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 19:47:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:47.670828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ac3ac-02b9-49f4-8240-b3327cd3025a None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:47.689873 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:47.773509 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ac3ac-02b9-49f4-8240-b3327cd3025a None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:47.808311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99238f-af5b-4427-81df-7d7b23635ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:47.809322 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99238f-af5b-4427-81df-7d7b23635ccf None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:47.809807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99238f-af5b-4427-81df-7d7b23635ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:47.810276 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99238f-af5b-4427-81df-7d7b23635ccf None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:47.811171 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99238f-af5b-4427-81df-7d7b23635ccf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:47.811659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99238f-af5b-4427-81df-7d7b23635ccf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:47.813333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99238f-af5b-4427-81df-7d7b23635ccf None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:47.823530 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 495/986] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 19:47:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:47.825221 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf0e16-7253-4daa-a43f-d1a0493d2e0b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:47.936677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf0e16-7253-4daa-a43f-d1a0493d2e0b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:47.971739 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb376-5249-49a0-b2e8-623c13c53a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:47.972434 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb376-5249-49a0-b2e8-623c13c53a19 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:47.972724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb376-5249-49a0-b2e8-623c13c53a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:47.973115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb376-5249-49a0-b2e8-623c13c53a19 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:47.973831 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89eb376-5249-49a0-b2e8-623c13c53a19 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:47.974139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89eb376-5249-49a0-b2e8-623c13c53a19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:47.975511 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89eb376-5249-49a0-b2e8-623c13c53a19 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:47.985466 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 492/987] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 19:47:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:47.986743 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99238f-af5b-4427-81df-7d7b23635ccf None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:48.072507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:48.092595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99238f-af5b-4427-81df-7d7b23635ccf None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:48.127791 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a072d-c183-4138-b694-3e3dba0f1bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:48.128486 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a072d-c183-4138-b694-3e3dba0f1bca None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:48.128775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a072d-c183-4138-b694-3e3dba0f1bca None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:48.129056 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a072d-c183-4138-b694-3e3dba0f1bca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:48.129860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a072d-c183-4138-b694-3e3dba0f1bca None admin] RBAC: Adding query filter params (interface=admin, service_id=761235e214ff4a6590fa2c30e24e0878, region_id=RegionOne) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:48.130171 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a072d-c183-4138-b694-3e3dba0f1bca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:48.131528 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a072d-c183-4138-b694-3e3dba0f1bca None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:48.141470 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 496/988] 10.222.0.88 () {56 vars in 1249 bytes} [Thu May 19 19:47:47 2022] GET /identity/v3/endpoints?service_id=761235e214ff4a6590fa2c30e24e0878&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:48.142744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb376-5249-49a0-b2e8-623c13c53a19 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:48.200545 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:48.256967 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb376-5249-49a0-b2e8-623c13c53a19 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:48.292931 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81846b-3516-4157-9c90-7e9281310ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:48.293611 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81846b-3516-4157-9c90-7e9281310ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:48.293892 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81846b-3516-4157-9c90-7e9281310ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:48.294159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81846b-3516-4157-9c90-7e9281310ec3 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:48.294832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81846b-3516-4157-9c90-7e9281310ec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:48.296194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81846b-3516-4157-9c90-7e9281310ec3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:48.302980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 493/989] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 19:47:48 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:47:48.415242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f90396-de91-402e-8595-f742d4abd7db None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:48.415767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f90396-de91-402e-8595-f742d4abd7db None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:48.415852 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f90396-de91-402e-8595-f742d4abd7db None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:48.417972 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 497/990] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:48.423615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3b547-128e-417d-b1a2-10ebb320fd07 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:48.424223 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3b547-128e-417d-b1a2-10ebb320fd07 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:48.424223 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab3b547-128e-417d-b1a2-10ebb320fd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:48.442919 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-8ab3b547-128e-417d-b1a2-10ebb320fd07 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:48.444130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-8ab3b547-128e-417d-b1a2-10ebb320fd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:48.516661 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 494/991] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:48.539025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f90396-de91-402e-8595-f742d4abd7db None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:48.649658 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f90396-de91-402e-8595-f742d4abd7db None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:48.685983 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86100782-bc7b-4b73-b000-d3df750fb476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:48.686775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86100782-bc7b-4b73-b000-d3df750fb476 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:48.687056 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86100782-bc7b-4b73-b000-d3df750fb476 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:48.687329 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86100782-bc7b-4b73-b000-d3df750fb476 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:48.687985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86100782-bc7b-4b73-b000-d3df750fb476 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:48.689369 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86100782-bc7b-4b73-b000-d3df750fb476 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-86100782-bc7b-4b73-b000-d3df750fb476 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:48.701198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:48.702348 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:48.703521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 498/992] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:48.708548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3b547-128e-417d-b1a2-10ebb320fd07 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:48.754427 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:48.813910 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab3b547-128e-417d-b1a2-10ebb320fd07 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:48.859695 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ef691-a427-49d0-bc74-c816a3868e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:48.860683 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ef691-a427-49d0-bc74-c816a3868e1d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:48.861135 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ef691-a427-49d0-bc74-c816a3868e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:48.861575 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ef691-a427-49d0-bc74-c816a3868e1d None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:48.862438 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ef691-a427-49d0-bc74-c816a3868e1d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:48.862950 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ef691-a427-49d0-bc74-c816a3868e1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:48.864565 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ef691-a427-49d0-bc74-c816a3868e1d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:48.870222 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 495/993] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:48.875810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86100782-bc7b-4b73-b000-d3df750fb476 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:48.897688 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:48.993182 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86100782-bc7b-4b73-b000-d3df750fb476 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:49.030216 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c36c80-bcfe-44ea-94f2-510ebbc5c1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:49.030216 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c36c80-bcfe-44ea-94f2-510ebbc5c1ea None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:49.030216 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c36c80-bcfe-44ea-94f2-510ebbc5c1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:49.030216 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c36c80-bcfe-44ea-94f2-510ebbc5c1ea None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:49.030894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c36c80-bcfe-44ea-94f2-510ebbc5c1ea None admin] RBAC: Adding query filter params (type=compute) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:49.031250 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c36c80-bcfe-44ea-94f2-510ebbc5c1ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:49.032661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c36c80-bcfe-44ea-94f2-510ebbc5c1ea None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:49.033433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0cc0f-b04a-48a7-9eb8-d0153ac79898 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:49.033718 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0cc0f-b04a-48a7-9eb8-d0153ac79898 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:49.033996 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0cc0f-b04a-48a7-9eb8-d0153ac79898 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:49.035444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 496/994] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:49.038690 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 499/995] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:49.040423 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e22f8a-bd39-4ed8-9b67-9af846ca8c9e None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:49.040709 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e22f8a-bd39-4ed8-9b67-9af846ca8c9e None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:49.040985 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e22f8a-bd39-4ed8-9b67-9af846ca8c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:49.044071 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c36c80-bcfe-44ea-94f2-510ebbc5c1ea None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:49.058250 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-51e22f8a-bd39-4ed8-9b67-9af846ca8c9e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:49.059446 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-51e22f8a-bd39-4ed8-9b67-9af846ca8c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:49.127184 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:49.130436 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 497/996] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:49.153798 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c36c80-bcfe-44ea-94f2-510ebbc5c1ea None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:49.153988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e22f8a-bd39-4ed8-9b67-9af846ca8c9e None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:49.192802 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:49.196014 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418b3e4-5ede-4c66-9f2d-56fe07fba14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:49.196704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418b3e4-5ede-4c66-9f2d-56fe07fba14c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:49.197066 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418b3e4-5ede-4c66-9f2d-56fe07fba14c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:49.197273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418b3e4-5ede-4c66-9f2d-56fe07fba14c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:49.198638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418b3e4-5ede-4c66-9f2d-56fe07fba14c None admin] RBAC: Adding query filter params (interface=public, service_id=3cef5352492f48bc81e1e62fb11e7cd5, region_id=RegionOne) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:49.205026 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418b3e4-5ede-4c66-9f2d-56fe07fba14c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:49.205026 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418b3e4-5ede-4c66-9f2d-56fe07fba14c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:49.205893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 500/997] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 19:47:49 2022] GET /identity/v3/endpoints?service_id=3cef5352492f48bc81e1e62fb11e7cd5&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:49.211119 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418b3e4-5ede-4c66-9f2d-56fe07fba14c None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:49.262503 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e22f8a-bd39-4ed8-9b67-9af846ca8c9e None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:49.299985 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaeabad-6254-4ceb-b9e3-992d94b807c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:49.300808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeabad-6254-4ceb-b9e3-992d94b807c5 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:49.301091 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeabad-6254-4ceb-b9e3-992d94b807c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:49.301366 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeabad-6254-4ceb-b9e3-992d94b807c5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:49.302018 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaeabad-6254-4ceb-b9e3-992d94b807c5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:49.303425 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaeabad-6254-4ceb-b9e3-992d94b807c5 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-5eaeabad-6254-4ceb-b9e3-992d94b807c5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:49.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:49.318152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:47:49.318973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:49.326461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 498/998] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 19:47:49 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:49.327860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaeabad-6254-4ceb-b9e3-992d94b807c5 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:49.332614 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418b3e4-5ede-4c66-9f2d-56fe07fba14c None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:49.371525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424c653-ba5f-4271-a13f-3444f3a24535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:49.372383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c653-ba5f-4271-a13f-3444f3a24535 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:49.372740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c653-ba5f-4271-a13f-3444f3a24535 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:49.373083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c653-ba5f-4271-a13f-3444f3a24535 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:49.373838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424c653-ba5f-4271-a13f-3444f3a24535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:49.375311 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424c653-ba5f-4271-a13f-3444f3a24535 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:49.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 501/999] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 19:47:49 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:47:49.440091 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:49.444610 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaeabad-6254-4ceb-b9e3-992d94b807c5 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:49.485344 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549d380-c817-4664-9b05-f0276c6e658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:49.486036 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549d380-c817-4664-9b05-f0276c6e658c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:49.486318 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549d380-c817-4664-9b05-f0276c6e658c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:49.486653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549d380-c817-4664-9b05-f0276c6e658c None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:49.487322 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549d380-c817-4664-9b05-f0276c6e658c None admin] RBAC: Adding query filter params (name=image) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:49.487639 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549d380-c817-4664-9b05-f0276c6e658c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:49.489071 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549d380-c817-4664-9b05-f0276c6e658c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:49.500116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 499/1000] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:49.501234 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424c653-ba5f-4271-a13f-3444f3a24535 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:49.504482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:49.699889 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424c653-ba5f-4271-a13f-3444f3a24535 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:49.734219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cadeae-1fad-4cf7-9771-48c1e2c6a7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:49.734892 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cadeae-1fad-4cf7-9771-48c1e2c6a7de None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:49.735178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cadeae-1fad-4cf7-9771-48c1e2c6a7de None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:49.735454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cadeae-1fad-4cf7-9771-48c1e2c6a7de None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:49.736165 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cadeae-1fad-4cf7-9771-48c1e2c6a7de None admin] RBAC: Adding query filter params (type=image) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:49.736501 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cadeae-1fad-4cf7-9771-48c1e2c6a7de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:49.737855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cadeae-1fad-4cf7-9771-48c1e2c6a7de None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:49.742599 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 502/1001] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:50.562021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102b6c-8a4a-4fec-8473-80ccb00c1b1b None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:50.562519 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102b6c-8a4a-4fec-8473-80ccb00c1b1b None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:50.562595 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3102b6c-8a4a-4fec-8473-80ccb00c1b1b None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:50.563930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 500/1002] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:50.569558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ff081-2e5e-424b-94fe-7c1b841ed7e0 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:50.569973 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ff081-2e5e-424b-94fe-7c1b841ed7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:50.570143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ff081-2e5e-424b-94fe-7c1b841ed7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:50.587718 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-0f9ff081-2e5e-424b-94fe-7c1b841ed7e0 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:50.589561 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-0f9ff081-2e5e-424b-94fe-7c1b841ed7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:50.651554 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:50.659265 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 503/1003] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:50.681803 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3102b6c-8a4a-4fec-8473-80ccb00c1b1b None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:50.785849 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3102b6c-8a4a-4fec-8473-80ccb00c1b1b None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:50.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e823c92-66cc-419a-9b9c-d7645edcd6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:50.820614 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823c92-66cc-419a-9b9c-d7645edcd6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:50.820901 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823c92-66cc-419a-9b9c-d7645edcd6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:50.821176 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e823c92-66cc-419a-9b9c-d7645edcd6f9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:50.821826 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e823c92-66cc-419a-9b9c-d7645edcd6f9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:50.823729 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e823c92-66cc-419a-9b9c-d7645edcd6f9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-8e823c92-66cc-419a-9b9c-d7645edcd6f9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:50.835647 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:50.836788 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:50.843546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 501/1004] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 19:47:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:50.844371 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ff081-2e5e-424b-94fe-7c1b841ed7e0 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:50.948237 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ff081-2e5e-424b-94fe-7c1b841ed7e0 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:50.982815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321ef46-9a8a-4db4-bf33-8c4b9ad5b3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:50.983487 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321ef46-9a8a-4db4-bf33-8c4b9ad5b3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:50.983770 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321ef46-9a8a-4db4-bf33-8c4b9ad5b3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:50.984044 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321ef46-9a8a-4db4-bf33-8c4b9ad5b3a3 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:50.984780 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321ef46-9a8a-4db4-bf33-8c4b9ad5b3a3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:50.985085 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321ef46-9a8a-4db4-bf33-8c4b9ad5b3a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:50.986446 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321ef46-9a8a-4db4-bf33-8c4b9ad5b3a3 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:50.996725 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 504/1005] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 19:47:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:50.998042 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e823c92-66cc-419a-9b9c-d7645edcd6f9 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:51.027594 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:51.109055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e823c92-66cc-419a-9b9c-d7645edcd6f9 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:51.144829 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2049e8-c8ce-40e4-a20a-5c9a01296a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:51.145496 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2049e8-c8ce-40e4-a20a-5c9a01296a4f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:51.145781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2049e8-c8ce-40e4-a20a-5c9a01296a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:51.146057 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2049e8-c8ce-40e4-a20a-5c9a01296a4f None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:51.146766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2049e8-c8ce-40e4-a20a-5c9a01296a4f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:51.147070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2049e8-c8ce-40e4-a20a-5c9a01296a4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:51.148469 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2049e8-c8ce-40e4-a20a-5c9a01296a4f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:51.157963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 502/1006] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 19:47:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:51.159203 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321ef46-9a8a-4db4-bf33-8c4b9ad5b3a3 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:51.222877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:51.265157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321ef46-9a8a-4db4-bf33-8c4b9ad5b3a3 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:51.301129 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3f06b-42d1-4902-b222-d1716dd41a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:51.302034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3f06b-42d1-4902-b222-d1716dd41a03 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:51.302488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3f06b-42d1-4902-b222-d1716dd41a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:51.302905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3f06b-42d1-4902-b222-d1716dd41a03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:51.303795 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3f06b-42d1-4902-b222-d1716dd41a03 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:51.305392 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3f06b-42d1-4902-b222-d1716dd41a03 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:51.390257 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 505/1007] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 19:47:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:47:51.612444 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aa696-aeaf-4712-bda9-00697ba84dd9 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:51.612961 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aa696-aeaf-4712-bda9-00697ba84dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:51.613050 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aa696-aeaf-4712-bda9-00697ba84dd9 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:51.614386 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 503/1008] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:51.620907 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ef0dd-eba7-4ae7-b931-cee44ddae623 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:51.621365 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ef0dd-eba7-4ae7-b931-cee44ddae623 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:51.621469 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ef0dd-eba7-4ae7-b931-cee44ddae623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:51.639138 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-322ef0dd-eba7-4ae7-b931-cee44ddae623 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:51.640438 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-322ef0dd-eba7-4ae7-b931-cee44ddae623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:51.717547 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 506/1009] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:51.739983 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1aa696-aeaf-4712-bda9-00697ba84dd9 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:51.842727 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:51.852920 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1aa696-aeaf-4712-bda9-00697ba84dd9 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:51.888765 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390c56b-49e7-4847-89fe-efdb15a4fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:51.889480 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390c56b-49e7-4847-89fe-efdb15a4fac6 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:51.889765 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390c56b-49e7-4847-89fe-efdb15a4fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:51.890095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390c56b-49e7-4847-89fe-efdb15a4fac6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:51.890715 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390c56b-49e7-4847-89fe-efdb15a4fac6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:51.892070 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390c56b-49e7-4847-89fe-efdb15a4fac6 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-e390c56b-49e7-4847-89fe-efdb15a4fac6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:51.907783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:51.909426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:51.910452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 504/1010] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:51.915463 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ef0dd-eba7-4ae7-b931-cee44ddae623 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:51.966044 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:51.977415 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a7c1b-7628-4b6f-b18b-8a9408e0a0c2 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:51.977699 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a7c1b-7628-4b6f-b18b-8a9408e0a0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:51.977972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a7c1b-7628-4b6f-b18b-8a9408e0a0c2 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:51.979552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 505/1011] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:47:51.984664 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b39b7-ba48-4d77-9e1d-14d5150f807a None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:51.984977 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b39b7-ba48-4d77-9e1d-14d5150f807a None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:51.985215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b39b7-ba48-4d77-9e1d-14d5150f807a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:52.006348 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-a32b39b7-ba48-4d77-9e1d-14d5150f807a None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:52.006348 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-a32b39b7-ba48-4d77-9e1d-14d5150f807a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:52.030782 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ef0dd-eba7-4ae7-b931-cee44ddae623 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:52.070197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175457df-07eb-45e0-b6d6-7765202a1b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:52.070855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175457df-07eb-45e0-b6d6-7765202a1b8e None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:52.071157 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175457df-07eb-45e0-b6d6-7765202a1b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:52.071406 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175457df-07eb-45e0-b6d6-7765202a1b8e None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:52.072108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175457df-07eb-45e0-b6d6-7765202a1b8e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:52.072457 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175457df-07eb-45e0-b6d6-7765202a1b8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:52.073899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175457df-07eb-45e0-b6d6-7765202a1b8e None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:52.083331 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 507/1012] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:52.083498 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 506/1013] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:52.085106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175457df-07eb-45e0-b6d6-7765202a1b8e None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:52.102519 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b39b7-ba48-4d77-9e1d-14d5150f807a None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:52.189547 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:52.197138 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175457df-07eb-45e0-b6d6-7765202a1b8e None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:52.208881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:52.211635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b39b7-ba48-4d77-9e1d-14d5150f807a None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:52.238052 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c9b0d-0ba2-457f-872f-e6bdc49c2e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:52.238834 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c9b0d-0ba2-457f-872f-e6bdc49c2e6e None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:52.239122 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c9b0d-0ba2-457f-872f-e6bdc49c2e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:52.239504 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626c9b0d-0ba2-457f-872f-e6bdc49c2e6e None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:52.240224 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c9b0d-0ba2-457f-872f-e6bdc49c2e6e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:52.240566 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c9b0d-0ba2-457f-872f-e6bdc49c2e6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:52.242143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626c9b0d-0ba2-457f-872f-e6bdc49c2e6e None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:52.247435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415b454-959c-4ddd-96c1-426ada75996b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:52.248097 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415b454-959c-4ddd-96c1-426ada75996b None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:52.248492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415b454-959c-4ddd-96c1-426ada75996b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:52.248770 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415b454-959c-4ddd-96c1-426ada75996b None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:52.249491 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415b454-959c-4ddd-96c1-426ada75996b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:52.250877 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415b454-959c-4ddd-96c1-426ada75996b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:52.252198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 508/1014] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 19:47:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:52.253055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c9b0d-0ba2-457f-872f-e6bdc49c2e6e None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:52.356379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 507/1015] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:47:52 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:47:52.365733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626c9b0d-0ba2-457f-872f-e6bdc49c2e6e None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:52.412974 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb62e6d-b0dd-4c4b-82d4-06468893ecf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:52.413655 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62e6d-b0dd-4c4b-82d4-06468893ecf5 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:52.413928 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62e6d-b0dd-4c4b-82d4-06468893ecf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:52.414199 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62e6d-b0dd-4c4b-82d4-06468893ecf5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:52.414954 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb62e6d-b0dd-4c4b-82d4-06468893ecf5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:52.416409 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb62e6d-b0dd-4c4b-82d4-06468893ecf5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:52.520456 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 509/1016] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 19:47:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:53.671707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa663e28-44f9-4aa5-9538-fbfcd46fe3b3 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:53.672198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa663e28-44f9-4aa5-9538-fbfcd46fe3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:53.672198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa663e28-44f9-4aa5-9538-fbfcd46fe3b3 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:53.673605 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 508/1017] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:53.679030 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364e426-48aa-4b37-9a8d-3d7393a0ae68 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:53.679417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364e426-48aa-4b37-9a8d-3d7393a0ae68 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:53.679615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364e426-48aa-4b37-9a8d-3d7393a0ae68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:53.696932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-7364e426-48aa-4b37-9a8d-3d7393a0ae68 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:53.698647 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-7364e426-48aa-4b37-9a8d-3d7393a0ae68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:53.720543 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:53.776510 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 510/1018] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:53.801139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa663e28-44f9-4aa5-9538-fbfcd46fe3b3 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:53.919831 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa663e28-44f9-4aa5-9538-fbfcd46fe3b3 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:53.956141 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7bd8f-e660-48a4-adde-aa27e9b71041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:53.956868 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7bd8f-e660-48a4-adde-aa27e9b71041 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:53.957147 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7bd8f-e660-48a4-adde-aa27e9b71041 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:53.957447 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7bd8f-e660-48a4-adde-aa27e9b71041 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:53.958579 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7bd8f-e660-48a4-adde-aa27e9b71041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:53.960016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7bd8f-e660-48a4-adde-aa27e9b71041 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:53.966328 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 509/1019] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:53.967667 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364e426-48aa-4b37-9a8d-3d7393a0ae68 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:54.067620 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364e426-48aa-4b37-9a8d-3d7393a0ae68 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:54.101546 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165ea15-60a9-446c-a401-d8700582b058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:54.102235 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165ea15-60a9-446c-a401-d8700582b058 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:54.102515 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165ea15-60a9-446c-a401-d8700582b058 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:54.102789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165ea15-60a9-446c-a401-d8700582b058 None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:54.103500 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165ea15-60a9-446c-a401-d8700582b058 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:54.105007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165ea15-60a9-446c-a401-d8700582b058 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:54.256269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 511/1020] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:47:53 2022] POST /identity/v3/projects => generated 311 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:54.623834 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de369f4-56ae-4224-89fc-6e19030486e2 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:54.624199 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de369f4-56ae-4224-89fc-6e19030486e2 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:54.624408 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de369f4-56ae-4224-89fc-6e19030486e2 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:54.625915 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 510/1021] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:54.630940 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55585851-3c05-4347-8083-a933e32e420d None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:54.631230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55585851-3c05-4347-8083-a933e32e420d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:54.631505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55585851-3c05-4347-8083-a933e32e420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:54.637628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:54.649542 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-55585851-3c05-4347-8083-a933e32e420d None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:54.657108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-55585851-3c05-4347-8083-a933e32e420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:54.724382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 512/1022] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:54.747242 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de369f4-56ae-4224-89fc-6e19030486e2 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:54.763983 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:54.834929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d95d-3ca8-4716-a1b0-fa4419f1f138 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:54.835304 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d95d-3ca8-4716-a1b0-fa4419f1f138 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:54.835481 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d95d-3ca8-4716-a1b0-fa4419f1f138 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:54.836894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 513/1023] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:47:54.842576 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88a6ca-2765-4cba-acbf-e2ab2847004e None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:54.842865 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88a6ca-2765-4cba-acbf-e2ab2847004e None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:54.843150 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88a6ca-2765-4cba-acbf-e2ab2847004e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:54.858970 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de369f4-56ae-4224-89fc-6e19030486e2 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:54.860707 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-7e88a6ca-2765-4cba-acbf-e2ab2847004e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:54.861885 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-7e88a6ca-2765-4cba-acbf-e2ab2847004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:54.899455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051d0eb-5ddf-4a18-af51-636caee042fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:54.900149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051d0eb-5ddf-4a18-af51-636caee042fe None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:54.901089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051d0eb-5ddf-4a18-af51-636caee042fe None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:54.901372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051d0eb-5ddf-4a18-af51-636caee042fe None admin] PATH_INFO: `/v3/services/image` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:54.902035 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051d0eb-5ddf-4a18-af51-636caee042fe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:54.903435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051d0eb-5ddf-4a18-af51-636caee042fe None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-6051d0eb-5ddf-4a18-af51-636caee042fe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:54.915742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:54.917464 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:54.918395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 511/1024] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 19:47:54 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:54.923503 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051d0eb-5ddf-4a18-af51-636caee042fe None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:54.940760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 514/1025] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:54.964139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88a6ca-2765-4cba-acbf-e2ab2847004e None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:55.008329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:55.030803 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051d0eb-5ddf-4a18-af51-636caee042fe None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:55.032465 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:55.066437 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fb708-7728-4010-ad73-a649afbe4209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:55.067099 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fb708-7728-4010-ad73-a649afbe4209 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:55.067447 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fb708-7728-4010-ad73-a649afbe4209 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:55.067754 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fb708-7728-4010-ad73-a649afbe4209 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:55.068476 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fb708-7728-4010-ad73-a649afbe4209 None admin] RBAC: Adding query filter params (name=image) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:55.068780 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fb708-7728-4010-ad73-a649afbe4209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:55.070184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fb708-7728-4010-ad73-a649afbe4209 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:55.071744 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88a6ca-2765-4cba-acbf-e2ab2847004e None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:55.076103 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 512/1026] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:55.081755 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fb708-7728-4010-ad73-a649afbe4209 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:55.110936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58253b84-2116-4b67-8402-0a54d37ec8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:55.111757 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253b84-2116-4b67-8402-0a54d37ec8aa None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:55.112040 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253b84-2116-4b67-8402-0a54d37ec8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:55.112313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58253b84-2116-4b67-8402-0a54d37ec8aa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:55.123394 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58253b84-2116-4b67-8402-0a54d37ec8aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-58253b84-2116-4b67-8402-0a54d37ec8aa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:55.131654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:55.132679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:55.133539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 515/1027] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 19:47:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:55.139165 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58253b84-2116-4b67-8402-0a54d37ec8aa None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:55.191989 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fb708-7728-4010-ad73-a649afbe4209 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:55.228088 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a997415-731f-4aea-a397-58709ce4b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:55.228805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a997415-731f-4aea-a397-58709ce4b911 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:55.229084 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a997415-731f-4aea-a397-58709ce4b911 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:55.229353 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a997415-731f-4aea-a397-58709ce4b911 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:55.230056 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a997415-731f-4aea-a397-58709ce4b911 None admin] RBAC: Adding query filter params (type=image) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:55.230359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a997415-731f-4aea-a397-58709ce4b911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:55.231733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a997415-731f-4aea-a397-58709ce4b911 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:55.237037 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 513/1028] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:55.242745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a997415-731f-4aea-a397-58709ce4b911 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:55.251733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:55.256663 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58253b84-2116-4b67-8402-0a54d37ec8aa None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:55.288680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:55.302403 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e312d-1f7b-408f-85a1-bc962790f5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:55.303269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e312d-1f7b-408f-85a1-bc962790f5cf None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:55.303879 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e312d-1f7b-408f-85a1-bc962790f5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:55.304583 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e312d-1f7b-408f-85a1-bc962790f5cf None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:55.305672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e312d-1f7b-408f-85a1-bc962790f5cf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:55.305981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e312d-1f7b-408f-85a1-bc962790f5cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:55.308034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e312d-1f7b-408f-85a1-bc962790f5cf None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:55.318965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 516/1029] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 19:47:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:55.325241 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e312d-1f7b-408f-85a1-bc962790f5cf None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:55.352738 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a997415-731f-4aea-a397-58709ce4b911 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:55.391121 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e47203-9c5d-4fb9-8d0d-328f71371f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:55.391795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47203-9c5d-4fb9-8d0d-328f71371f95 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:55.392080 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47203-9c5d-4fb9-8d0d-328f71371f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:55.392375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47203-9c5d-4fb9-8d0d-328f71371f95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:55.393174 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e47203-9c5d-4fb9-8d0d-328f71371f95 None admin] RBAC: Adding query filter params (interface=public, service_id=13cee9c59a134233a820bda637a591bb, region_id=RegionOne) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:55.393483 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e47203-9c5d-4fb9-8d0d-328f71371f95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:55.394852 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e47203-9c5d-4fb9-8d0d-328f71371f95 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:55.401101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 514/1030] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 19:47:55 2022] GET /identity/v3/endpoints?service_id=13cee9c59a134233a820bda637a591bb&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:55.406347 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e47203-9c5d-4fb9-8d0d-328f71371f95 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:55.440802 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e312d-1f7b-408f-85a1-bc962790f5cf None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:55.480153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2531d-4c32-4250-968e-d3d56f31a249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:55.480153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2531d-4c32-4250-968e-d3d56f31a249 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:55.480153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2531d-4c32-4250-968e-d3d56f31a249 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:55.480153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2531d-4c32-4250-968e-d3d56f31a249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:55.480596 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:55.488121 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2531d-4c32-4250-968e-d3d56f31a249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:55.489758 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2531d-4c32-4250-968e-d3d56f31a249 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-82d2531d-4c32-4250-968e-d3d56f31a249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:55.509078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:55.510870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:55.511784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:55.511784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:55.511784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:55.511784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:55.514097 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 517/1031] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:55.517523 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2531d-4c32-4250-968e-d3d56f31a249 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:55.529514 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e47203-9c5d-4fb9-8d0d-328f71371f95 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:55.559997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:55.573988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748bb34-be82-4c70-9f08-58c0330edf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:55.575249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748bb34-be82-4c70-9f08-58c0330edf6c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:55.575722 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748bb34-be82-4c70-9f08-58c0330edf6c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:55.576365 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748bb34-be82-4c70-9f08-58c0330edf6c None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:55.577749 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748bb34-be82-4c70-9f08-58c0330edf6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:55.579579 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748bb34-be82-4c70-9f08-58c0330edf6c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:55.592802 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 515/1032] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 19:47:55 2022] GET /identity/v3/services => generated 1957 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 19:47:55.624694 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2531d-4c32-4250-968e-d3d56f31a249 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:55.662128 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6413b-ea8b-4c04-9ff8-e532ae6f296b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:55.662804 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6413b-ea8b-4c04-9ff8-e532ae6f296b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:55.663091 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6413b-ea8b-4c04-9ff8-e532ae6f296b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:55.663372 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6413b-ea8b-4c04-9ff8-e532ae6f296b None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:55.664111 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6413b-ea8b-4c04-9ff8-e532ae6f296b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:55.664457 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6413b-ea8b-4c04-9ff8-e532ae6f296b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:55.665900 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6413b-ea8b-4c04-9ff8-e532ae6f296b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:55.678902 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 518/1033] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:55.685364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748bb34-be82-4c70-9f08-58c0330edf6c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:55.806116 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748bb34-be82-4c70-9f08-58c0330edf6c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:55.842685 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a58d5-697e-4fe5-aab1-be6ff3bbde41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:55.847926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a58d5-697e-4fe5-aab1-be6ff3bbde41 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:55.847926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a58d5-697e-4fe5-aab1-be6ff3bbde41 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:55.847926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58a58d5-697e-4fe5-aab1-be6ff3bbde41 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:55.852835 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:55.858946 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a58d5-697e-4fe5-aab1-be6ff3bbde41 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:55.860702 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58a58d5-697e-4fe5-aab1-be6ff3bbde41 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-c58a58d5-697e-4fe5-aab1-be6ff3bbde41 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:47:55.876855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:55.877993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:55.879330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:55.879330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:55.879330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:47:55.879330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:47:55.879330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:47:55.879330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:47:55.879330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:47:55.879330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:47:55.879330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:47:55.879330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:55.879330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 516/1034] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 19:47:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:55.884369 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6413b-ea8b-4c04-9ff8-e532ae6f296b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:55.985695 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:55.990104 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6413b-ea8b-4c04-9ff8-e532ae6f296b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:56.024695 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4beaac5-b19a-48c2-9f53-a308d44e773f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:56.025916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beaac5-b19a-48c2-9f53-a308d44e773f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:56.026404 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beaac5-b19a-48c2-9f53-a308d44e773f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:56.026946 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4beaac5-b19a-48c2-9f53-a308d44e773f None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:56.028001 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4beaac5-b19a-48c2-9f53-a308d44e773f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:56.028767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4beaac5-b19a-48c2-9f53-a308d44e773f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:56.030442 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4beaac5-b19a-48c2-9f53-a308d44e773f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:56.053431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 519/1035] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 19:47:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:56.054776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a58d5-697e-4fe5-aab1-be6ff3bbde41 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:56.186002 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58a58d5-697e-4fe5-aab1-be6ff3bbde41 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:56.231949 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84110219-bc84-43e2-b84d-b31e419d693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:56.232640 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84110219-bc84-43e2-b84d-b31e419d693a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:56.232926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84110219-bc84-43e2-b84d-b31e419d693a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:56.233214 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84110219-bc84-43e2-b84d-b31e419d693a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:56.300001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:56.308471 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84110219-bc84-43e2-b84d-b31e419d693a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:56.309936 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84110219-bc84-43e2-b84d-b31e419d693a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:56.430416 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 517/1036] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:47:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:47:56.431701 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4beaac5-b19a-48c2-9f53-a308d44e773f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:56.533384 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4beaac5-b19a-48c2-9f53-a308d44e773f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:56.553344 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35054afc-624c-42c5-a5ff-f04e93f33f7b None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:56.553641 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35054afc-624c-42c5-a5ff-f04e93f33f7b None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:56.553907 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35054afc-624c-42c5-a5ff-f04e93f33f7b None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:56.555414 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 518/1037] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:56.561029 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49cd8-76d7-4887-98f3-41200328e357 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:56.561323 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49cd8-76d7-4887-98f3-41200328e357 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:56.561608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49cd8-76d7-4887-98f3-41200328e357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:56.570492 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941bd2-25dd-477c-b4a0-c2ae0ae8ece5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:56.571601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941bd2-25dd-477c-b4a0-c2ae0ae8ece5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:56.572126 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941bd2-25dd-477c-b4a0-c2ae0ae8ece5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:56.572605 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941bd2-25dd-477c-b4a0-c2ae0ae8ece5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:56.579112 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-3bf49cd8-76d7-4887-98f3-41200328e357 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:56.580203 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-3bf49cd8-76d7-4887-98f3-41200328e357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:56.584662 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941bd2-25dd-477c-b4a0-c2ae0ae8ece5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:56.586258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941bd2-25dd-477c-b4a0-c2ae0ae8ece5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-4b941bd2-25dd-477c-b4a0-c2ae0ae8ece5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:47:56.604730 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:56.605853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:56.606843 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:47:56.606843 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:47:56.606843 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:47:56.606843 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:56.606843 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 520/1038] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:56.611625 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941bd2-25dd-477c-b4a0-c2ae0ae8ece5 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:56.624578 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:56.661316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 519/1039] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:56.685863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf49cd8-76d7-4887-98f3-41200328e357 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:56.715770 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:56.721528 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941bd2-25dd-477c-b4a0-c2ae0ae8ece5 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:56.758258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61067a75-c261-495a-ae42-ebc430807a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:56.758916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61067a75-c261-495a-ae42-ebc430807a25 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:56.759240 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61067a75-c261-495a-ae42-ebc430807a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:56.759512 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61067a75-c261-495a-ae42-ebc430807a25 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:56.760826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61067a75-c261-495a-ae42-ebc430807a25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:56.761134 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61067a75-c261-495a-ae42-ebc430807a25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:56.762526 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61067a75-c261-495a-ae42-ebc430807a25 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:56.776091 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 521/1040] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:56.781890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61067a75-c261-495a-ae42-ebc430807a25 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:56.806657 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf49cd8-76d7-4887-98f3-41200328e357 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:56.848995 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ed605-a862-4e63-a2b3-6f9e05136e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:56.849790 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ed605-a862-4e63-a2b3-6f9e05136e24 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:56.850065 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ed605-a862-4e63-a2b3-6f9e05136e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:56.850325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ed605-a862-4e63-a2b3-6f9e05136e24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:56.851322 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:56.851653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ed605-a862-4e63-a2b3-6f9e05136e24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:56.853187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ed605-a862-4e63-a2b3-6f9e05136e24 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:56.854870 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 520/1041] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:56.860451 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ed605-a862-4e63-a2b3-6f9e05136e24 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:56.890937 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61067a75-c261-495a-ae42-ebc430807a25 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:56.926873 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086bd74-4d67-4872-9153-5ce08d3c182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:56.927683 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086bd74-4d67-4872-9153-5ce08d3c182f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:56.927965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086bd74-4d67-4872-9153-5ce08d3c182f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:56.928241 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086bd74-4d67-4872-9153-5ce08d3c182f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:56.935940 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086bd74-4d67-4872-9153-5ce08d3c182f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:56.937657 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086bd74-4d67-4872-9153-5ce08d3c182f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-f086bd74-4d67-4872-9153-5ce08d3c182f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:47:56.953866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:56.954921 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:56.955845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:47:56.955845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:47:56.955845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:47:56.955845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:47:56.955845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:47:56.955845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:47:56.955845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:47:56.955845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:56.955845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 522/1042] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:47:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:56.961979 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086bd74-4d67-4872-9153-5ce08d3c182f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:56.968861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:56.977323 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ed605-a862-4e63-a2b3-6f9e05136e24 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:57.013284 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ca1a8-7884-41aa-b67f-c96741bd1c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:57.014005 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ca1a8-7884-41aa-b67f-c96741bd1c3f None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:57.014296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ca1a8-7884-41aa-b67f-c96741bd1c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:57.014559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ca1a8-7884-41aa-b67f-c96741bd1c3f None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:57.015257 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ca1a8-7884-41aa-b67f-c96741bd1c3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:57.016719 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ca1a8-7884-41aa-b67f-c96741bd1c3f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:57.068433 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086bd74-4d67-4872-9153-5ce08d3c182f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:57.096494 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:57.104162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21672d8-28bd-446f-b433-061fdf871d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:57.104852 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21672d8-28bd-446f-b433-061fdf871d54 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:57.105135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21672d8-28bd-446f-b433-061fdf871d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:57.105410 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21672d8-28bd-446f-b433-061fdf871d54 None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:57.106191 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21672d8-28bd-446f-b433-061fdf871d54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:57.106503 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21672d8-28bd-446f-b433-061fdf871d54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:57.108040 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21672d8-28bd-446f-b433-061fdf871d54 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:57.119894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 523/1043] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:47:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:57.126220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21672d8-28bd-446f-b433-061fdf871d54 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:57.134424 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 521/1044] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:47:56 2022] POST /identity/v3/users => generated 285 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 19:47:57.237557 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21672d8-28bd-446f-b433-061fdf871d54 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:57.274929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028ec88-cbbb-4fdf-9087-cbaa0a60ed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:57.275639 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028ec88-cbbb-4fdf-9087-cbaa0a60ed12 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:57.275973 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028ec88-cbbb-4fdf-9087-cbaa0a60ed12 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:57.276256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028ec88-cbbb-4fdf-9087-cbaa0a60ed12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:57.277151 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028ec88-cbbb-4fdf-9087-cbaa0a60ed12 None admin] RBAC: Adding query filter params (role.id=f7602c7b55f44757bd1e1a99ee31b24d, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=0c0e71329c4a4470935593636c025257) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:57.277463 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028ec88-cbbb-4fdf-9087-cbaa0a60ed12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:57.278944 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028ec88-cbbb-4fdf-9087-cbaa0a60ed12 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:57.292390 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 524/1045] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:47:57 2022] GET /identity/v3/role_assignments?user.id=0c0e71329c4a4470935593636c025257&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=f7602c7b55f44757bd1e1a99ee31b24d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:57.906805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e13a3-5a25-4153-a2a1-0c88cbbbecab None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:57.907349 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e13a3-5a25-4153-a2a1-0c88cbbbecab None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:57.907349 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e13a3-5a25-4153-a2a1-0c88cbbbecab None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:57.908950 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 522/1046] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:57.914098 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9169c-9a5b-4789-a955-73a925645378 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:57.914527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9169c-9a5b-4789-a955-73a925645378 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:57.914664 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9169c-9a5b-4789-a955-73a925645378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:57.932598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-5ce9169c-9a5b-4789-a955-73a925645378 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:57.933781 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-5ce9169c-9a5b-4789-a955-73a925645378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:57.982400 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:58.004898 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 525/1047] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:47:58.028075 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e13a3-5a25-4153-a2a1-0c88cbbbecab None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:58.134719 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e13a3-5a25-4153-a2a1-0c88cbbbecab None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:58.168808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed97a8-ecb2-4634-9a5d-d196161e133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:58.169520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed97a8-ecb2-4634-9a5d-d196161e133a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:58.169867 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed97a8-ecb2-4634-9a5d-d196161e133a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:58.170146 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feed97a8-ecb2-4634-9a5d-d196161e133a None admin] PATH_INFO: `/v3/services/image` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:58.170817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed97a8-ecb2-4634-9a5d-d196161e133a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:58.172192 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feed97a8-ecb2-4634-9a5d-d196161e133a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-feed97a8-ecb2-4634-9a5d-d196161e133a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:58.184893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:58.185860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 19:47:58.186767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:58.188701 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 523/1048] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 19:47:58 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:47:58.192154 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9169c-9a5b-4789-a955-73a925645378 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:58.293404 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9169c-9a5b-4789-a955-73a925645378 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:58.330706 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32acadb-5554-44f9-b3d0-d197b13ee965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:58.331378 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32acadb-5554-44f9-b3d0-d197b13ee965 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:58.331668 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32acadb-5554-44f9-b3d0-d197b13ee965 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:58.331983 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32acadb-5554-44f9-b3d0-d197b13ee965 None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:58.332765 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32acadb-5554-44f9-b3d0-d197b13ee965 None admin] RBAC: Adding query filter params (name=image) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:58.333109 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32acadb-5554-44f9-b3d0-d197b13ee965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:58.334497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32acadb-5554-44f9-b3d0-d197b13ee965 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:58.345877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 526/1049] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:47:58.347320 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed97a8-ecb2-4634-9a5d-d196161e133a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:58.364084 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:58.460679 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feed97a8-ecb2-4634-9a5d-d196161e133a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:58.498627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39296f0d-1c2e-47e7-964f-076b4e015129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:58.499294 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39296f0d-1c2e-47e7-964f-076b4e015129 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:58.499578 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39296f0d-1c2e-47e7-964f-076b4e015129 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:58.499854 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39296f0d-1c2e-47e7-964f-076b4e015129 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:58.500605 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39296f0d-1c2e-47e7-964f-076b4e015129 None admin] RBAC: Adding query filter params (type=image) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:58.500913 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39296f0d-1c2e-47e7-964f-076b4e015129 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:58.502303 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39296f0d-1c2e-47e7-964f-076b4e015129 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:58.512428 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 524/1050] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:47:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:58.514853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32acadb-5554-44f9-b3d0-d197b13ee965 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:58.564699 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:58.626486 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32acadb-5554-44f9-b3d0-d197b13ee965 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:58.663818 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78457024-a131-463a-885a-80c68bdc4f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:58.664514 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78457024-a131-463a-885a-80c68bdc4f22 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:58.664799 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78457024-a131-463a-885a-80c68bdc4f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:58.665073 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78457024-a131-463a-885a-80c68bdc4f22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:58.665817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78457024-a131-463a-885a-80c68bdc4f22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:58.667172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78457024-a131-463a-885a-80c68bdc4f22 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:58.792429 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 527/1051] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 19:47:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:47:59.498261 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4a583-b12c-4652-a398-a0e7a3a8eddf None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:59.498811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4a583-b12c-4652-a398-a0e7a3a8eddf None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:59.498811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde4a583-b12c-4652-a398-a0e7a3a8eddf None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:59.500370 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 525/1052] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:59.505608 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc66d8-d68a-494a-858d-48bf3e181daf None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:59.506059 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc66d8-d68a-494a-858d-48bf3e181daf None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:59.506145 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc66d8-d68a-494a-858d-48bf3e181daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:59.523987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-efdc66d8-d68a-494a-858d-48bf3e181daf None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:59.525821 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-efdc66d8-d68a-494a-858d-48bf3e181daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:59.539152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ec55-c5d9-4010-a8d1-2c95bce724de None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:59.539431 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ec55-c5d9-4010-a8d1-2c95bce724de None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:59.539705 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166ec55-c5d9-4010-a8d1-2c95bce724de None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:59.541320 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 526/1053] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:47:59.546383 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567cb07-5804-4b53-b9f0-41d7b7990113 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:59.546676 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567cb07-5804-4b53-b9f0-41d7b7990113 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:59.546954 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567cb07-5804-4b53-b9f0-41d7b7990113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:59.565622 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-5567cb07-5804-4b53-b9f0-41d7b7990113 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:47:59.566740 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-5567cb07-5804-4b53-b9f0-41d7b7990113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:47:59.604034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 528/1054] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:59.627535 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc66d8-d68a-494a-858d-48bf3e181daf None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:59.641377 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 527/1055] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:47:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:47:59.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:59.664465 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567cb07-5804-4b53-b9f0-41d7b7990113 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:59.673415 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:59.731699 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc66d8-d68a-494a-858d-48bf3e181daf None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:59.767285 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997c951-0b3a-44dc-8180-2d0b0e4eca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:59.768092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997c951-0b3a-44dc-8180-2d0b0e4eca8e None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:59.768602 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997c951-0b3a-44dc-8180-2d0b0e4eca8e None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:59.769956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997c951-0b3a-44dc-8180-2d0b0e4eca8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:59.774182 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567cb07-5804-4b53-b9f0-41d7b7990113 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:59.781291 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997c951-0b3a-44dc-8180-2d0b0e4eca8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-8997c951-0b3a-44dc-8180-2d0b0e4eca8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:59.788956 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:59.789897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:47:59.790741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 529/1056] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:47:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:59.796155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997c951-0b3a-44dc-8180-2d0b0e4eca8e None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:59.816030 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c457436-00db-44e2-b8fe-675bbcbdff1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:59.816890 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c457436-00db-44e2-b8fe-675bbcbdff1a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:59.817176 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c457436-00db-44e2-b8fe-675bbcbdff1a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:59.817490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c457436-00db-44e2-b8fe-675bbcbdff1a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:59.829256 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c457436-00db-44e2-b8fe-675bbcbdff1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-3c457436-00db-44e2-b8fe-675bbcbdff1a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:47:59.838372 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:47:59.839336 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:47:59.840211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 528/1057] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 19:47:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:47:59.845508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c457436-00db-44e2-b8fe-675bbcbdff1a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:59.887831 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:59.902004 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997c951-0b3a-44dc-8180-2d0b0e4eca8e None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:47:59.917106 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:47:59.940703 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60616603-13a6-48e8-9989-1a9b70c7105b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:47:59.941378 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60616603-13a6-48e8-9989-1a9b70c7105b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:47:59.941658 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60616603-13a6-48e8-9989-1a9b70c7105b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:47:59.942028 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60616603-13a6-48e8-9989-1a9b70c7105b None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:47:59.942877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60616603-13a6-48e8-9989-1a9b70c7105b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:47:59.943177 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60616603-13a6-48e8-9989-1a9b70c7105b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:47:59.944569 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60616603-13a6-48e8-9989-1a9b70c7105b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:47:59.959083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 530/1058] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:47:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:47:59.960200 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60616603-13a6-48e8-9989-1a9b70c7105b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:47:59.962450 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c457436-00db-44e2-b8fe-675bbcbdff1a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:00.002742 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76843310-c4bb-4ab4-bd6b-72bec75ae287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:00.003431 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76843310-c4bb-4ab4-bd6b-72bec75ae287 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:00.003711 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76843310-c4bb-4ab4-bd6b-72bec75ae287 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:00.003987 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76843310-c4bb-4ab4-bd6b-72bec75ae287 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:00.004768 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76843310-c4bb-4ab4-bd6b-72bec75ae287 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:00.005078 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76843310-c4bb-4ab4-bd6b-72bec75ae287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:00.006508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76843310-c4bb-4ab4-bd6b-72bec75ae287 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:00.015816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 529/1059] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 19:47:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:48:00.021521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76843310-c4bb-4ab4-bd6b-72bec75ae287 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:00.071563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60616603-13a6-48e8-9989-1a9b70c7105b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:00.111259 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85c20e-65cc-4f4b-a1a9-ee9888c1e081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:00.112130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85c20e-65cc-4f4b-a1a9-ee9888c1e081 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:00.112440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85c20e-65cc-4f4b-a1a9-ee9888c1e081 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:00.112715 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85c20e-65cc-4f4b-a1a9-ee9888c1e081 None admin] PATH_INFO: `/v3/users/d2590ed658ae47adb0314eb3554a5612` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:00.129065 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85c20e-65cc-4f4b-a1a9-ee9888c1e081 None admin] RBAC: Authorizing `identity:get_user(user_id=d2590ed658ae47adb0314eb3554a5612)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:00.129947 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76843310-c4bb-4ab4-bd6b-72bec75ae287 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:00.130517 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85c20e-65cc-4f4b-a1a9-ee9888c1e081 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:00.138593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 531/1060] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:47:59 2022] GET /identity/v3/users/d2590ed658ae47adb0314eb3554a5612 => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:00.139691 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85c20e-65cc-4f4b-a1a9-ee9888c1e081 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:00.154003 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:00.158417 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:00.166443 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37859473-12b5-4a7d-8894-78006d81042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:00.167246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859473-12b5-4a7d-8894-78006d81042b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:00.167505 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859473-12b5-4a7d-8894-78006d81042b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:00.167773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859473-12b5-4a7d-8894-78006d81042b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:00.179411 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37859473-12b5-4a7d-8894-78006d81042b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:00.181035 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37859473-12b5-4a7d-8894-78006d81042b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-37859473-12b5-4a7d-8894-78006d81042b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:48:00.199510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:00.200833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:00.202504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:48:00.202504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:48:00.202504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:48:00.202504 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:00.205536 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 530/1061] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:48:00.206320 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37859473-12b5-4a7d-8894-78006d81042b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:00.250595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85c20e-65cc-4f4b-a1a9-ee9888c1e081 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:00.287016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c5b96-7a98-4191-a379-045f3d44ecc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:00.287671 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5b96-7a98-4191-a379-045f3d44ecc0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:00.287953 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5b96-7a98-4191-a379-045f3d44ecc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:00.288228 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c5b96-7a98-4191-a379-045f3d44ecc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:00.314926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37859473-12b5-4a7d-8894-78006d81042b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:00.355295 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f07bf1-f276-4fcc-b5aa-64e90477ebcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:00.356049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07bf1-f276-4fcc-b5aa-64e90477ebcd None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:00.356323 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07bf1-f276-4fcc-b5aa-64e90477ebcd None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:00.356868 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f07bf1-f276-4fcc-b5aa-64e90477ebcd None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:00.357344 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f07bf1-f276-4fcc-b5aa-64e90477ebcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:00.357658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f07bf1-f276-4fcc-b5aa-64e90477ebcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:00.359126 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f07bf1-f276-4fcc-b5aa-64e90477ebcd None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:00.366576 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c5b96-7a98-4191-a379-045f3d44ecc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:00.368007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c5b96-7a98-4191-a379-045f3d44ecc0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:00.372389 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 531/1062] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:48:00.377604 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f07bf1-f276-4fcc-b5aa-64e90477ebcd None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:00.383846 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:00.421104 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:00.484669 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 532/1063] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:48:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:48:00.485590 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f07bf1-f276-4fcc-b5aa-64e90477ebcd None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:00.490494 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c5b96-7a98-4191-a379-045f3d44ecc0 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:00.526810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d154605-ac82-46dc-9436-3fe4098f13cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:00.527596 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d154605-ac82-46dc-9436-3fe4098f13cc None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:00.527876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d154605-ac82-46dc-9436-3fe4098f13cc None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:00.528146 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d154605-ac82-46dc-9436-3fe4098f13cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:00.535673 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d154605-ac82-46dc-9436-3fe4098f13cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:00.537158 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d154605-ac82-46dc-9436-3fe4098f13cc None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-5d154605-ac82-46dc-9436-3fe4098f13cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:48:00.553332 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:00.554489 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:00.555453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:00.555453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:00.555453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:48:00.555453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:48:00.555453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:48:00.555453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:48:00.555453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:48:00.555453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:48:00.555453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:48:00.555453 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:00.560124 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 532/1064] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 19:48:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:48:00.561027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d154605-ac82-46dc-9436-3fe4098f13cc None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:00.606154 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c5b96-7a98-4191-a379-045f3d44ecc0 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:00.618360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:00.645220 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b781f8-b6e5-4e15-93fe-2203947c900c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:00.646089 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b781f8-b6e5-4e15-93fe-2203947c900c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:00.646370 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b781f8-b6e5-4e15-93fe-2203947c900c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:00.646751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b781f8-b6e5-4e15-93fe-2203947c900c None admin] PATH_INFO: `/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:00.648110 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b781f8-b6e5-4e15-93fe-2203947c900c None admin] RBAC: Authorizing `identity:get_project(project_id=b6fc64be32a84efabf1c40d0cbc26df9)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:00.649941 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b781f8-b6e5-4e15-93fe-2203947c900c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:00.652932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 533/1065] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:00 2022] GET /identity/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:48:00.661899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b781f8-b6e5-4e15-93fe-2203947c900c None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:00.670205 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:00.673519 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d154605-ac82-46dc-9436-3fe4098f13cc None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:00.709939 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749b3a8-622d-49a3-8134-faa03c53c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:00.710670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749b3a8-622d-49a3-8134-faa03c53c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:00.710956 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749b3a8-622d-49a3-8134-faa03c53c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:00.711236 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749b3a8-622d-49a3-8134-faa03c53c4f2 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:00.712072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749b3a8-622d-49a3-8134-faa03c53c4f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:00.712407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749b3a8-622d-49a3-8134-faa03c53c4f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:00.714409 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749b3a8-622d-49a3-8134-faa03c53c4f2 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:00.732119 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 533/1066] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 19:48:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:00.737539 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749b3a8-622d-49a3-8134-faa03c53c4f2 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:00.773983 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b781f8-b6e5-4e15-93fe-2203947c900c None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:00.812121 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2a026-a6b1-4510-9671-90b82333adb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:00.813106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2a026-a6b1-4510-9671-90b82333adb9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:00.813653 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2a026-a6b1-4510-9671-90b82333adb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:00.814084 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2a026-a6b1-4510-9671-90b82333adb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:00.815210 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2a026-a6b1-4510-9671-90b82333adb9 None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=b6fc64be32a84efabf1c40d0cbc26df9, user.id=d2590ed658ae47adb0314eb3554a5612) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:00.815681 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2a026-a6b1-4510-9671-90b82333adb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:00.817385 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2a026-a6b1-4510-9671-90b82333adb9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:00.823968 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 534/1067] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:48:00 2022] GET /identity/v3/role_assignments?user.id=d2590ed658ae47adb0314eb3554a5612&scope.project.id=b6fc64be32a84efabf1c40d0cbc26df9&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:00.851911 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749b3a8-622d-49a3-8134-faa03c53c4f2 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:00.893054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16458b82-0b89-45e9-b62a-2175af16810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:00.893900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16458b82-0b89-45e9-b62a-2175af16810a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:00.894152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16458b82-0b89-45e9-b62a-2175af16810a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:00.894426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16458b82-0b89-45e9-b62a-2175af16810a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:00.906952 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16458b82-0b89-45e9-b62a-2175af16810a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:00.908501 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16458b82-0b89-45e9-b62a-2175af16810a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-16458b82-0b89-45e9-b62a-2175af16810a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:48:00.928820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:00.929777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:00.930707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:48:00.930707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:48:00.930707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:48:00.930707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:00.930707 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 534/1068] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:00.935917 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2a026-a6b1-4510-9671-90b82333adb9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:00.985019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:01.047131 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03798a39-796c-43ad-a91f-e2f84c445eb0 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:01.047670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03798a39-796c-43ad-a91f-e2f84c445eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:01.047954 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03798a39-796c-43ad-a91f-e2f84c445eb0 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:01.049077 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2a026-a6b1-4510-9671-90b82333adb9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:01.049555 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 535/1069] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:01.055062 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f6866-ef68-41f5-9593-17559bb59269 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:01.055432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f6866-ef68-41f5-9593-17559bb59269 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:01.055780 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f6866-ef68-41f5-9593-17559bb59269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:01.073275 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-192f6866-ef68-41f5-9593-17559bb59269 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:01.074544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-192f6866-ef68-41f5-9593-17559bb59269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:01.078319 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:01.088910 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50104308-d3f0-4d4f-ad01-a059d8581f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:01.089580 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50104308-d3f0-4d4f-ad01-a059d8581f10 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:01.089865 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50104308-d3f0-4d4f-ad01-a059d8581f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:01.090170 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50104308-d3f0-4d4f-ad01-a059d8581f10 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:01.090872 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50104308-d3f0-4d4f-ad01-a059d8581f10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:01.091362 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50104308-d3f0-4d4f-ad01-a059d8581f10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:01.092670 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50104308-d3f0-4d4f-ad01-a059d8581f10 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:01.106480 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 535/1070] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:01.111601 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50104308-d3f0-4d4f-ad01-a059d8581f10 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:01.156385 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 536/1071] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:48:01.176694 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f6866-ef68-41f5-9593-17559bb59269 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:01.216930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:01.219582 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50104308-d3f0-4d4f-ad01-a059d8581f10 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:01.254461 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62280c-0110-4d77-b9c9-d7d857665c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:01.255255 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62280c-0110-4d77-b9c9-d7d857665c07 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:01.255540 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62280c-0110-4d77-b9c9-d7d857665c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:01.255817 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62280c-0110-4d77-b9c9-d7d857665c07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:01.263836 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62280c-0110-4d77-b9c9-d7d857665c07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:01.265663 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62280c-0110-4d77-b9c9-d7d857665c07 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-8e62280c-0110-4d77-b9c9-d7d857665c07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:48:01.280809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:01.281955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:01.283096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:48:01.283096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:48:01.283096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:48:01.283096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:48:01.283096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:48:01.283096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:48:01.283096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:48:01.283096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:48:01.284952 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 536/1072] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:48:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:01.290574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62280c-0110-4d77-b9c9-d7d857665c07 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:01.292779 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f6866-ef68-41f5-9593-17559bb59269 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:01.334733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4acbdc-e7ce-4f66-abe9-38c7fd844370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:01.335414 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4acbdc-e7ce-4f66-abe9-38c7fd844370 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:01.335719 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4acbdc-e7ce-4f66-abe9-38c7fd844370 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:01.335985 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4acbdc-e7ce-4f66-abe9-38c7fd844370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:01.348520 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:01.412192 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62280c-0110-4d77-b9c9-d7d857665c07 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:01.413814 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4acbdc-e7ce-4f66-abe9-38c7fd844370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:01.416058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4acbdc-e7ce-4f66-abe9-38c7fd844370 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:01.459025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c97866-de58-4b73-a09b-ab8b503cce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:01.460239 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c97866-de58-4b73-a09b-ab8b503cce93 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:01.460621 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c97866-de58-4b73-a09b-ab8b503cce93 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:01.460981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c97866-de58-4b73-a09b-ab8b503cce93 None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:01.461775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c97866-de58-4b73-a09b-ab8b503cce93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:01.462089 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c97866-de58-4b73-a09b-ab8b503cce93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:01.463633 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c97866-de58-4b73-a09b-ab8b503cce93 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:01.481567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 537/1073] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:48:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:01.483503 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c97866-de58-4b73-a09b-ab8b503cce93 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:01.506117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:01.537980 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 537/1074] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:48:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:48:01.539491 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4acbdc-e7ce-4f66-abe9-38c7fd844370 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:01.586424 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:01.589812 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c97866-de58-4b73-a09b-ab8b503cce93 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:01.624682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ab3b2-1dfd-40f8-abc8-fcfbced590de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:01.625583 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ab3b2-1dfd-40f8-abc8-fcfbced590de None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:01.625864 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ab3b2-1dfd-40f8-abc8-fcfbced590de None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:01.626139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ab3b2-1dfd-40f8-abc8-fcfbced590de None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/0c0e71329c4a4470935593636c025257/roles/f7602c7b55f44757bd1e1a99ee31b24d` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:01.628417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ab3b2-1dfd-40f8-abc8-fcfbced590de None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=0c0e71329c4a4470935593636c025257, role_id=f7602c7b55f44757bd1e1a99ee31b24d)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:01.629912 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ab3b2-1dfd-40f8-abc8-fcfbced590de None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:01.649955 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4acbdc-e7ce-4f66-abe9-38c7fd844370 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:01.684287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e7c84-edc7-47cf-aeca-c9398f468cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:01.685100 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7c84-edc7-47cf-aeca-c9398f468cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:01.685386 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7c84-edc7-47cf-aeca-c9398f468cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:01.685665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e7c84-edc7-47cf-aeca-c9398f468cf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:01.695695 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e7c84-edc7-47cf-aeca-c9398f468cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:01.697779 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e7c84-edc7-47cf-aeca-c9398f468cf5 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-ca7e7c84-edc7-47cf-aeca-c9398f468cf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:48:01.715541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:01.716744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:01.717833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:48:01.717833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:48:01.717833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:48:01.717833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:01.717833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 538/1075] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:01.722805 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e7c84-edc7-47cf-aeca-c9398f468cf5 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:01.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 538/1076] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:48:01 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/0c0e71329c4a4470935593636c025257/roles/f7602c7b55f44757bd1e1a99ee31b24d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:48:01.836156 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:01.846182 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e7c84-edc7-47cf-aeca-c9398f468cf5 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:01.884833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05930859-bf24-46f9-896e-80f540c9e9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:01.885518 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05930859-bf24-46f9-896e-80f540c9e9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:01.885803 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05930859-bf24-46f9-896e-80f540c9e9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:01.886083 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05930859-bf24-46f9-896e-80f540c9e9c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:01.886828 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05930859-bf24-46f9-896e-80f540c9e9c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:01.887138 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05930859-bf24-46f9-896e-80f540c9e9c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:01.888575 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05930859-bf24-46f9-896e-80f540c9e9c9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:01.908394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 539/1077] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:48:03.066388 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a0a3-4c35-43e4-8154-aec1bf4da997 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:03.066925 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a0a3-4c35-43e4-8154-aec1bf4da997 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:03.066925 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a0a3-4c35-43e4-8154-aec1bf4da997 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:03.068376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 539/1078] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:03.073618 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675129a7-d0f1-428f-b086-433315ef1d8e None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:03.074038 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675129a7-d0f1-428f-b086-433315ef1d8e None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:03.074299 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675129a7-d0f1-428f-b086-433315ef1d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:03.091598 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-675129a7-d0f1-428f-b086-433315ef1d8e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:03.092862 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-675129a7-d0f1-428f-b086-433315ef1d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:03.158631 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 540/1079] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:03.181259 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5a0a3-4c35-43e4-8154-aec1bf4da997 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:03.261324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:03.281298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5a0a3-4c35-43e4-8154-aec1bf4da997 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:03.314815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de00bbb-ebce-472a-bb12-ef55a397e455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:03.315611 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de00bbb-ebce-472a-bb12-ef55a397e455 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:03.315891 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de00bbb-ebce-472a-bb12-ef55a397e455 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:03.316165 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de00bbb-ebce-472a-bb12-ef55a397e455 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:03.327830 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de00bbb-ebce-472a-bb12-ef55a397e455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-3de00bbb-ebce-472a-bb12-ef55a397e455 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:48:03.335519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:03.336800 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:48:03.343336 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 540/1080] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:48:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:03.344049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675129a7-d0f1-428f-b086-433315ef1d8e None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:03.427239 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:03.444998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675129a7-d0f1-428f-b086-433315ef1d8e None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:03.481189 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a466b-e76f-4706-9dbd-a7d91bc4fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:03.481854 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a466b-e76f-4706-9dbd-a7d91bc4fd31 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:03.482136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a466b-e76f-4706-9dbd-a7d91bc4fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:03.482410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a466b-e76f-4706-9dbd-a7d91bc4fd31 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:03.483156 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a466b-e76f-4706-9dbd-a7d91bc4fd31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:03.483461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a466b-e76f-4706-9dbd-a7d91bc4fd31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:03.484870 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a466b-e76f-4706-9dbd-a7d91bc4fd31 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:03.493746 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 541/1081] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:48:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:03.499778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de00bbb-ebce-472a-bb12-ef55a397e455 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:03.599156 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de00bbb-ebce-472a-bb12-ef55a397e455 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:03.632836 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6e4aa-1a27-406b-ad87-1303f825676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:03.633710 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6e4aa-1a27-406b-ad87-1303f825676b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:03.633988 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6e4aa-1a27-406b-ad87-1303f825676b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:03.634261 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6e4aa-1a27-406b-ad87-1303f825676b None admin] PATH_INFO: `/v3/users/d2590ed658ae47adb0314eb3554a5612` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:03.635970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6e4aa-1a27-406b-ad87-1303f825676b None admin] RBAC: Authorizing `identity:get_user(user_id=d2590ed658ae47adb0314eb3554a5612)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:03.637419 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6e4aa-1a27-406b-ad87-1303f825676b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:03.639214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 541/1082] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:03 2022] GET /identity/v3/users/d2590ed658ae47adb0314eb3554a5612 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:03.644860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a466b-e76f-4706-9dbd-a7d91bc4fd31 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:03.745515 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a466b-e76f-4706-9dbd-a7d91bc4fd31 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:03.779090 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702e515-594b-4f46-accd-2b14ee5e90a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:03.780011 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e515-594b-4f46-accd-2b14ee5e90a3 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:03.780297 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e515-594b-4f46-accd-2b14ee5e90a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:03.780682 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e515-594b-4f46-accd-2b14ee5e90a3 None admin] PATH_INFO: `/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:03.781947 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702e515-594b-4f46-accd-2b14ee5e90a3 None admin] RBAC: Authorizing `identity:get_project(project_id=b6fc64be32a84efabf1c40d0cbc26df9)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:03.783461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702e515-594b-4f46-accd-2b14ee5e90a3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:03.785436 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 542/1083] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:03 2022] GET /identity/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:03.792417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6e4aa-1a27-406b-ad87-1303f825676b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:03.805871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:03.893649 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6e4aa-1a27-406b-ad87-1303f825676b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:03.927233 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e298c-d0bd-42d9-9fda-7cb269abc164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:03.928114 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e298c-d0bd-42d9-9fda-7cb269abc164 None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:03.928417 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e298c-d0bd-42d9-9fda-7cb269abc164 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:03.928696 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e298c-d0bd-42d9-9fda-7cb269abc164 None admin] PATH_INFO: `/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9/users/d2590ed658ae47adb0314eb3554a5612/roles/815bd48c3df0405babfd5b53430fd3ea` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:03.930671 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e298c-d0bd-42d9-9fda-7cb269abc164 None admin] RBAC: Authorizing `identity:create_grant(project_id=b6fc64be32a84efabf1c40d0cbc26df9, user_id=d2590ed658ae47adb0314eb3554a5612, role_id=815bd48c3df0405babfd5b53430fd3ea)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:03.932112 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e298c-d0bd-42d9-9fda-7cb269abc164 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:03.975631 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975708ff-f2bb-4d74-b0b0-f6c5f94c35f0 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:03.975917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975708ff-f2bb-4d74-b0b0-f6c5f94c35f0 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:03.976167 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975708ff-f2bb-4d74-b0b0-f6c5f94c35f0 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:03.977591 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 543/1084] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:48:03.983030 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d37842-c9ff-4112-9709-850d05e2aa04 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:03.983330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d37842-c9ff-4112-9709-850d05e2aa04 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:03.983611 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d37842-c9ff-4112-9709-850d05e2aa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:04.000228 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-19d37842-c9ff-4112-9709-850d05e2aa04 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:04.001470 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-19d37842-c9ff-4112-9709-850d05e2aa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:04.008786 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:04.020992 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 542/1085] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:48:03 2022] PUT /identity/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9/users/d2590ed658ae47adb0314eb3554a5612/roles/815bd48c3df0405babfd5b53430fd3ea => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:48:04.075347 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 544/1086] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:48:04.099854 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e298c-d0bd-42d9-9fda-7cb269abc164 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:04.169970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:04.206470 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e298c-d0bd-42d9-9fda-7cb269abc164 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:04.241040 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430feb1-f60a-4ee4-87ee-5a8f286de3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:04.241870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430feb1-f60a-4ee4-87ee-5a8f286de3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:04.242155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430feb1-f60a-4ee4-87ee-5a8f286de3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:04.242434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430feb1-f60a-4ee4-87ee-5a8f286de3f4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:04.252615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430feb1-f60a-4ee4-87ee-5a8f286de3f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-9430feb1-f60a-4ee4-87ee-5a8f286de3f4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:48:04.261488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:04.262423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:48:04.263266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 543/1087] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 19:48:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:04.268881 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d37842-c9ff-4112-9709-850d05e2aa04 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:04.368675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d37842-c9ff-4112-9709-850d05e2aa04 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:04.403170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f9df7-94b9-4f85-ac1f-5158124da9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:04.403836 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f9df7-94b9-4f85-ac1f-5158124da9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:04.404118 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f9df7-94b9-4f85-ac1f-5158124da9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:04.404420 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f9df7-94b9-4f85-ac1f-5158124da9f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:04.405163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f9df7-94b9-4f85-ac1f-5158124da9f4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:04.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f9df7-94b9-4f85-ac1f-5158124da9f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:04.406872 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f9df7-94b9-4f85-ac1f-5158124da9f4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:04.415154 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 545/1088] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 19:48:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:04.421081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430feb1-f60a-4ee4-87ee-5a8f286de3f4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:04.523359 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430feb1-f60a-4ee4-87ee-5a8f286de3f4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:04.558655 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:04.574173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d662de-371f-458e-bc7b-1f58735faab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:04.574974 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d662de-371f-458e-bc7b-1f58735faab0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:04.575254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d662de-371f-458e-bc7b-1f58735faab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:04.575525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d662de-371f-458e-bc7b-1f58735faab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:04.586363 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d662de-371f-458e-bc7b-1f58735faab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:04.588304 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d662de-371f-458e-bc7b-1f58735faab0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-21d662de-371f-458e-bc7b-1f58735faab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:48:04.605855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:04.606945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:04.608292 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:48:04.608292 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:48:04.608292 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:48:04.608292 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:48:04.608292 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 544/1089] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:04.612687 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f9df7-94b9-4f85-ac1f-5158124da9f4 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:04.616689 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:04.710214 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f9df7-94b9-4f85-ac1f-5158124da9f4 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:04.745149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927aa511-73bd-4a60-bd59-ac9ed5a3b4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:04.745802 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927aa511-73bd-4a60-bd59-ac9ed5a3b4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:04.746079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927aa511-73bd-4a60-bd59-ac9ed5a3b4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:04.746349 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927aa511-73bd-4a60-bd59-ac9ed5a3b4d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:04.747062 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927aa511-73bd-4a60-bd59-ac9ed5a3b4d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:04.747361 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927aa511-73bd-4a60-bd59-ac9ed5a3b4d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:04.748789 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927aa511-73bd-4a60-bd59-ac9ed5a3b4d1 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:04.761566 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 546/1090] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:04.767046 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d662de-371f-458e-bc7b-1f58735faab0 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:04.875847 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d662de-371f-458e-bc7b-1f58735faab0 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:04.911894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f13bf7-d2c3-4d07-ac43-3b65da21de44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:04.912727 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f13bf7-d2c3-4d07-ac43-3b65da21de44 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:04.913012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f13bf7-d2c3-4d07-ac43-3b65da21de44 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:04.913286 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f13bf7-d2c3-4d07-ac43-3b65da21de44 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:04.921147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f13bf7-d2c3-4d07-ac43-3b65da21de44 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:04.922578 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f13bf7-d2c3-4d07-ac43-3b65da21de44 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-f5f13bf7-d2c3-4d07-ac43-3b65da21de44 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:48:04.936899 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:04.937887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:04.938860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:04.938860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:04.938860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:48:04.938860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:48:04.938860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:48:04.938860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:48:04.938860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:48:04.938860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:48:04.938860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 19:48:04.938860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:48:04.938860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 545/1091] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 19:48:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:04.943935 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927aa511-73bd-4a60-bd59-ac9ed5a3b4d1 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:05.006902 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:05.044679 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927aa511-73bd-4a60-bd59-ac9ed5a3b4d1 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:05.079238 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147637f0-6c2d-45c1-abbb-b571bd9e9e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:05.079906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147637f0-6c2d-45c1-abbb-b571bd9e9e2e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:05.080163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147637f0-6c2d-45c1-abbb-b571bd9e9e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:05.080465 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147637f0-6c2d-45c1-abbb-b571bd9e9e2e None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:05.081298 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147637f0-6c2d-45c1-abbb-b571bd9e9e2e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:05.081603 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147637f0-6c2d-45c1-abbb-b571bd9e9e2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:05.083035 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147637f0-6c2d-45c1-abbb-b571bd9e9e2e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:05.100362 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 547/1092] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 19:48:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:48:05.106645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f13bf7-d2c3-4d07-ac43-3b65da21de44 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:05.150152 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:05.213906 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f13bf7-d2c3-4d07-ac43-3b65da21de44 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:05.247169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd3f30-7799-4fd0-b1fa-5749cd51083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:05.247814 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd3f30-7799-4fd0-b1fa-5749cd51083a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:05.248102 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd3f30-7799-4fd0-b1fa-5749cd51083a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:05.248411 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd3f30-7799-4fd0-b1fa-5749cd51083a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:05.315794 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd3f30-7799-4fd0-b1fa-5749cd51083a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:05.317290 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd3f30-7799-4fd0-b1fa-5749cd51083a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:05.360823 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:05.418293 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 546/1093] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:48:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:48:05.424014 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147637f0-6c2d-45c1-abbb-b571bd9e9e2e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:05.532867 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147637f0-6c2d-45c1-abbb-b571bd9e9e2e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:05.594019 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:05.605534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17832ff-8d0b-4dcf-9eeb-60e5cf7d52c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:05.606330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17832ff-8d0b-4dcf-9eeb-60e5cf7d52c2 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:05.606613 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17832ff-8d0b-4dcf-9eeb-60e5cf7d52c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:05.606887 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17832ff-8d0b-4dcf-9eeb-60e5cf7d52c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:05.616940 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17832ff-8d0b-4dcf-9eeb-60e5cf7d52c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:05.618395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17832ff-8d0b-4dcf-9eeb-60e5cf7d52c2 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-d17832ff-8d0b-4dcf-9eeb-60e5cf7d52c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:48:05.636394 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:05.637346 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:05.638341 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:48:05.638341 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:48:05.638341 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:48:05.638341 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:05.638341 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 548/1094] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:05.643412 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd3f30-7799-4fd0-b1fa-5749cd51083a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:05.740688 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd3f30-7799-4fd0-b1fa-5749cd51083a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:05.774676 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d6729-f405-4965-849b-15f3995931a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:05.775886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d6729-f405-4965-849b-15f3995931a2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:05.776178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d6729-f405-4965-849b-15f3995931a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:05.776485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d6729-f405-4965-849b-15f3995931a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:05.777201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d6729-f405-4965-849b-15f3995931a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:05.777503 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d6729-f405-4965-849b-15f3995931a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:05.778851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d6729-f405-4965-849b-15f3995931a2 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:05.787517 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:05.795815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 547/1095] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:05.796925 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17832ff-8d0b-4dcf-9eeb-60e5cf7d52c2 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:05.899301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17832ff-8d0b-4dcf-9eeb-60e5cf7d52c2 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:05.934234 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef3e0-9a24-421a-9686-b9d39bb86a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:05.935024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef3e0-9a24-421a-9686-b9d39bb86a54 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:05.935304 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef3e0-9a24-421a-9686-b9d39bb86a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:05.935574 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecef3e0-9a24-421a-9686-b9d39bb86a54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:05.942195 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecef3e0-9a24-421a-9686-b9d39bb86a54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:05.943636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecef3e0-9a24-421a-9686-b9d39bb86a54 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-3ecef3e0-9a24-421a-9686-b9d39bb86a54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:48:05.957906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:05.958893 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:05.959972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:48:05.959972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:48:05.959972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:48:05.959972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:48:05.959972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:48:05.959972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:48:05.959972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:48:05.959972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:05.959972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 549/1096] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:05.964976 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d6729-f405-4965-849b-15f3995931a2 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:06.066383 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d6729-f405-4965-849b-15f3995931a2 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:06.102051 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0229c5-e0a9-4931-924d-d5ad1af286ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:06.102707 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0229c5-e0a9-4931-924d-d5ad1af286ed None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:06.102990 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0229c5-e0a9-4931-924d-d5ad1af286ed None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:06.103262 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0229c5-e0a9-4931-924d-d5ad1af286ed None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:06.104079 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0229c5-e0a9-4931-924d-d5ad1af286ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:06.104418 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0229c5-e0a9-4931-924d-d5ad1af286ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:06.105835 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0229c5-e0a9-4931-924d-d5ad1af286ed None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:06.117945 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 548/1097] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:06.124512 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef3e0-9a24-421a-9686-b9d39bb86a54 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:06.159431 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:06.225782 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecef3e0-9a24-421a-9686-b9d39bb86a54 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:06.241708 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a56cf-b52c-4ef6-86a6-2e87c824a284 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:06.241995 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a56cf-b52c-4ef6-86a6-2e87c824a284 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:06.242271 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a56cf-b52c-4ef6-86a6-2e87c824a284 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:06.243634 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 549/1098] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:48:06.249325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87ff46-b86a-4264-a834-5dc0023e9a6e None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:06.249616 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87ff46-b86a-4264-a834-5dc0023e9a6e None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:06.249891 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87ff46-b86a-4264-a834-5dc0023e9a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:06.261193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6e386-3d96-43b2-a9f8-25b007826ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:06.261837 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e386-3d96-43b2-a9f8-25b007826ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:06.262153 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e386-3d96-43b2-a9f8-25b007826ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:06.262425 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e386-3d96-43b2-a9f8-25b007826ca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:06.263249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6e386-3d96-43b2-a9f8-25b007826ca6 None admin] RBAC: Adding query filter params (role.id=f7602c7b55f44757bd1e1a99ee31b24d, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=0c0e71329c4a4470935593636c025257) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:06.263556 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6e386-3d96-43b2-a9f8-25b007826ca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:06.265017 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6e386-3d96-43b2-a9f8-25b007826ca6 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:06.266670 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-fb87ff46-b86a-4264-a834-5dc0023e9a6e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:06.267854 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-fb87ff46-b86a-4264-a834-5dc0023e9a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:06.270861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 550/1099] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:48:06 2022] GET /identity/v3/role_assignments?user.id=0c0e71329c4a4470935593636c025257&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=f7602c7b55f44757bd1e1a99ee31b24d => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:06.336260 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:06.350113 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 550/1100] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:06.374176 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6e386-3d96-43b2-a9f8-25b007826ca6 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:06.482751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:06.487496 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6e386-3d96-43b2-a9f8-25b007826ca6 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:06.522512 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bc569-0867-441f-bcef-0bb50d7af68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:06.523302 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bc569-0867-441f-bcef-0bb50d7af68b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:06.523583 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bc569-0867-441f-bcef-0bb50d7af68b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:06.523859 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bc569-0867-441f-bcef-0bb50d7af68b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:06.534832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bc569-0867-441f-bcef-0bb50d7af68b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-3c2bc569-0867-441f-bcef-0bb50d7af68b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:48:06.542615 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:06.543752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:06.544988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 551/1101] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:48:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:06.550235 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87ff46-b86a-4264-a834-5dc0023e9a6e None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:06.653397 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87ff46-b86a-4264-a834-5dc0023e9a6e None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:06.688582 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd411f74-b795-4e04-be72-8baec2d0ef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:06.689263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd411f74-b795-4e04-be72-8baec2d0ef23 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:06.689541 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd411f74-b795-4e04-be72-8baec2d0ef23 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:06.689818 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd411f74-b795-4e04-be72-8baec2d0ef23 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:06.690567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd411f74-b795-4e04-be72-8baec2d0ef23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:06.690869 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd411f74-b795-4e04-be72-8baec2d0ef23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:06.692230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd411f74-b795-4e04-be72-8baec2d0ef23 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:06.701740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 551/1102] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:48:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:06.707508 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bc569-0867-441f-bcef-0bb50d7af68b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:06.813215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bc569-0867-441f-bcef-0bb50d7af68b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:06.847593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64575c1-a445-4f0f-bdbc-1d2861aacdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:06.848499 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64575c1-a445-4f0f-bdbc-1d2861aacdad None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:06.848781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64575c1-a445-4f0f-bdbc-1d2861aacdad None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:06.849062 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64575c1-a445-4f0f-bdbc-1d2861aacdad None admin] PATH_INFO: `/v3/users/d2590ed658ae47adb0314eb3554a5612` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:06.850246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64575c1-a445-4f0f-bdbc-1d2861aacdad None admin] RBAC: Authorizing `identity:get_user(user_id=d2590ed658ae47adb0314eb3554a5612)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:06.851686 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64575c1-a445-4f0f-bdbc-1d2861aacdad None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:06.853383 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 552/1103] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:06 2022] GET /identity/v3/users/d2590ed658ae47adb0314eb3554a5612 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:48:06.859161 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd411f74-b795-4e04-be72-8baec2d0ef23 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:06.914458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:06.967454 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd411f74-b795-4e04-be72-8baec2d0ef23 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:07.002734 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8602d-d5de-426a-b8bb-e4f8d3d0fb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:07.003387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8602d-d5de-426a-b8bb-e4f8d3d0fb8b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:07.003668 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8602d-d5de-426a-b8bb-e4f8d3d0fb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:07.003943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8602d-d5de-426a-b8bb-e4f8d3d0fb8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:07.077636 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8602d-d5de-426a-b8bb-e4f8d3d0fb8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:07.079010 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8602d-d5de-426a-b8bb-e4f8d3d0fb8b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:07.131155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:07.180071 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 552/1104] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:48:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:48:07.187012 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64575c1-a445-4f0f-bdbc-1d2861aacdad None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:07.214813 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:07.287079 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64575c1-a445-4f0f-bdbc-1d2861aacdad None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:07.321085 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e40c9-d2c4-4fe4-9c0a-b8718b12c497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:07.321930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e40c9-d2c4-4fe4-9c0a-b8718b12c497 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:07.322744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e40c9-d2c4-4fe4-9c0a-b8718b12c497 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:07.323019 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e40c9-d2c4-4fe4-9c0a-b8718b12c497 None admin] PATH_INFO: `/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:07.324198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e40c9-d2c4-4fe4-9c0a-b8718b12c497 None admin] RBAC: Authorizing `identity:get_project(project_id=b6fc64be32a84efabf1c40d0cbc26df9)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:07.325667 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e40c9-d2c4-4fe4-9c0a-b8718b12c497 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:07.327371 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 553/1105] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:07 2022] GET /identity/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:07.333644 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8602d-d5de-426a-b8bb-e4f8d3d0fb8b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:07.429603 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8602d-d5de-426a-b8bb-e4f8d3d0fb8b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:07.462718 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011530f3-14d0-43cb-9969-de5d565bfec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:07.463397 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011530f3-14d0-43cb-9969-de5d565bfec3 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:07.463680 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011530f3-14d0-43cb-9969-de5d565bfec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:07.463953 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011530f3-14d0-43cb-9969-de5d565bfec3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:07.464820 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011530f3-14d0-43cb-9969-de5d565bfec3 None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=b6fc64be32a84efabf1c40d0cbc26df9, user.id=d2590ed658ae47adb0314eb3554a5612) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:07.465133 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011530f3-14d0-43cb-9969-de5d565bfec3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:07.466577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011530f3-14d0-43cb-9969-de5d565bfec3 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:07.472278 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 553/1106] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:48:07 2022] GET /identity/v3/role_assignments?user.id=d2590ed658ae47adb0314eb3554a5612&scope.project.id=b6fc64be32a84efabf1c40d0cbc26df9&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:09.726452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6553cd-246d-403f-aa39-29511d423d44 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:09.726955 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6553cd-246d-403f-aa39-29511d423d44 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:09.726955 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6553cd-246d-403f-aa39-29511d423d44 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:09.728303 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 554/1107] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:09.733682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4efce7-eab8-4481-b32d-af9f4a4fc1d4 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:09.734094 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4efce7-eab8-4481-b32d-af9f4a4fc1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:09.734235 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4efce7-eab8-4481-b32d-af9f4a4fc1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:09.751640 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-ab4efce7-eab8-4481-b32d-af9f4a4fc1d4 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:09.752858 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-ab4efce7-eab8-4481-b32d-af9f4a4fc1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:09.763968 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:09.821263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 554/1108] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:48:09.846722 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6553cd-246d-403f-aa39-29511d423d44 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:09.940262 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:09.948525 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6553cd-246d-403f-aa39-29511d423d44 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:09.982232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c11bd40-ab59-4f6d-b17e-6eeff3ecb45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:09.982930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11bd40-ab59-4f6d-b17e-6eeff3ecb45b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:09.983207 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11bd40-ab59-4f6d-b17e-6eeff3ecb45b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:09.983473 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11bd40-ab59-4f6d-b17e-6eeff3ecb45b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:09.984654 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11bd40-ab59-4f6d-b17e-6eeff3ecb45b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:09.986111 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c11bd40-ab59-4f6d-b17e-6eeff3ecb45b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:09.987723 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 555/1109] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:48:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:48:09.993567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4efce7-eab8-4481-b32d-af9f4a4fc1d4 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:10.092225 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4efce7-eab8-4481-b32d-af9f4a4fc1d4 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:10.126846 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fcceb5-ed1a-406a-a0cc-ebab16cb96f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:10.127508 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcceb5-ed1a-406a-a0cc-ebab16cb96f5 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:10.127787 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcceb5-ed1a-406a-a0cc-ebab16cb96f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:10.128058 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fcceb5-ed1a-406a-a0cc-ebab16cb96f5 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:10.128794 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcceb5-ed1a-406a-a0cc-ebab16cb96f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:10.130208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fcceb5-ed1a-406a-a0cc-ebab16cb96f5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:10.241764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 555/1110] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:48:09 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:48:12.493810 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ddaef-9085-45dc-b9bc-3d096803f93a None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:12.494340 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ddaef-9085-45dc-b9bc-3d096803f93a None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:12.494426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ddaef-9085-45dc-b9bc-3d096803f93a None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:12.496287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 556/1111] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:12.501520 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f17ca-1bdb-4d33-a21a-d1dad6fa15b8 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:12.501941 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f17ca-1bdb-4d33-a21a-d1dad6fa15b8 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:12.502082 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f17ca-1bdb-4d33-a21a-d1dad6fa15b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:12.507250 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:12.519264 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-ee2f17ca-1bdb-4d33-a21a-d1dad6fa15b8 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:12.520482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-ee2f17ca-1bdb-4d33-a21a-d1dad6fa15b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:12.587352 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 556/1112] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:12.610870 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ddaef-9085-45dc-b9bc-3d096803f93a None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:12.712292 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ddaef-9085-45dc-b9bc-3d096803f93a None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:12.745874 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967538a5-e659-43ba-b154-bd77eb3ced99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:12.746735 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967538a5-e659-43ba-b154-bd77eb3ced99 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:12.747016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967538a5-e659-43ba-b154-bd77eb3ced99 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:12.747288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967538a5-e659-43ba-b154-bd77eb3ced99 None admin] PATH_INFO: `/v3/roles/cb116a5614cf4a48ad99840d95f96bb3` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:12.748781 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967538a5-e659-43ba-b154-bd77eb3ced99 None admin] RBAC: Authorizing `identity:get_role(role_id=cb116a5614cf4a48ad99840d95f96bb3)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:12.750689 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967538a5-e659-43ba-b154-bd77eb3ced99 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:12.752060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 557/1113] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:12 2022] GET /identity/v3/roles/cb116a5614cf4a48ad99840d95f96bb3 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:12.757961 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f17ca-1bdb-4d33-a21a-d1dad6fa15b8 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:12.854401 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f17ca-1bdb-4d33-a21a-d1dad6fa15b8 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:12.882929 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:12.887805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794bb70-77aa-46a7-9585-539db5b23bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:12.888696 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794bb70-77aa-46a7-9585-539db5b23bbf None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:12.888977 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794bb70-77aa-46a7-9585-539db5b23bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:12.889251 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794bb70-77aa-46a7-9585-539db5b23bbf None admin] PATH_INFO: `/v3/users/c414b1ef68934f54afa91ea5e301b34c` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:12.902664 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794bb70-77aa-46a7-9585-539db5b23bbf None admin] RBAC: Authorizing `identity:get_user(user_id=c414b1ef68934f54afa91ea5e301b34c)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:12.904538 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794bb70-77aa-46a7-9585-539db5b23bbf None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:12.906190 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 557/1114] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:12 2022] GET /identity/v3/users/c414b1ef68934f54afa91ea5e301b34c => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:12.911961 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967538a5-e659-43ba-b154-bd77eb3ced99 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:12.942974 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:13.012775 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967538a5-e659-43ba-b154-bd77eb3ced99 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:13.046536 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff2f42-7be8-4fd7-87b9-609b5d3c840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:13.047176 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff2f42-7be8-4fd7-87b9-609b5d3c840e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:13.047511 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff2f42-7be8-4fd7-87b9-609b5d3c840e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:13.047792 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff2f42-7be8-4fd7-87b9-609b5d3c840e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:13.116097 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff2f42-7be8-4fd7-87b9-609b5d3c840e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:13.117682 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff2f42-7be8-4fd7-87b9-609b5d3c840e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:13.146418 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:13.215855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 558/1115] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:48:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:48:13.222759 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794bb70-77aa-46a7-9585-539db5b23bbf None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:13.322818 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794bb70-77aa-46a7-9585-539db5b23bbf None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:13.357097 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630924f-01e6-4c45-b1fa-3723d22170a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:13.357955 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630924f-01e6-4c45-b1fa-3723d22170a0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:13.358242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630924f-01e6-4c45-b1fa-3723d22170a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:13.358521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630924f-01e6-4c45-b1fa-3723d22170a0 None admin] PATH_INFO: `/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:13.359695 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630924f-01e6-4c45-b1fa-3723d22170a0 None admin] RBAC: Authorizing `identity:get_project(project_id=b6fc64be32a84efabf1c40d0cbc26df9)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:13.361144 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630924f-01e6-4c45-b1fa-3723d22170a0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:13.362830 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 558/1116] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:13 2022] GET /identity/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:13.368809 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff2f42-7be8-4fd7-87b9-609b5d3c840e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:13.466714 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff2f42-7be8-4fd7-87b9-609b5d3c840e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:13.493640 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:13.499886 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef135c4d-9707-40b1-aa4b-2520ccd74458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:13.500560 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef135c4d-9707-40b1-aa4b-2520ccd74458 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:13.500848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef135c4d-9707-40b1-aa4b-2520ccd74458 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:13.501122 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef135c4d-9707-40b1-aa4b-2520ccd74458 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:13.501941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef135c4d-9707-40b1-aa4b-2520ccd74458 None admin] RBAC: Adding query filter params (role.id=cb116a5614cf4a48ad99840d95f96bb3, scope.project.id=b6fc64be32a84efabf1c40d0cbc26df9, user.id=c414b1ef68934f54afa91ea5e301b34c) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:13.502249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef135c4d-9707-40b1-aa4b-2520ccd74458 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:13.503696 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef135c4d-9707-40b1-aa4b-2520ccd74458 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:13.509009 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 559/1117] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:48:13 2022] GET /identity/v3/role_assignments?user.id=c414b1ef68934f54afa91ea5e301b34c&scope.project.id=b6fc64be32a84efabf1c40d0cbc26df9&role.id=cb116a5614cf4a48ad99840d95f96bb3 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:15.689545 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebb929-09bc-48c6-accc-24c2014322cf None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:15.690065 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebb929-09bc-48c6-accc-24c2014322cf None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:15.690065 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebb929-09bc-48c6-accc-24c2014322cf None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:15.691359 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 559/1118] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:15.696430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33923d11-6a2c-4333-b68a-237a4a852d61 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:15.696827 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33923d11-6a2c-4333-b68a-237a4a852d61 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:15.696992 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33923d11-6a2c-4333-b68a-237a4a852d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:15.714951 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-33923d11-6a2c-4333-b68a-237a4a852d61 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:15.716134 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-33923d11-6a2c-4333-b68a-237a4a852d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:15.781852 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 560/1119] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:15.805015 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebb929-09bc-48c6-accc-24c2014322cf None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:15.863152 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:15.902941 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebb929-09bc-48c6-accc-24c2014322cf None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:15.936422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e01486-864d-44bf-987b-8a0c2ff6cae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:15.937254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e01486-864d-44bf-987b-8a0c2ff6cae4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:15.937536 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e01486-864d-44bf-987b-8a0c2ff6cae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:15.937809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e01486-864d-44bf-987b-8a0c2ff6cae4 None admin] PATH_INFO: `/v3/roles/cb116a5614cf4a48ad99840d95f96bb3` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:15.939281 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e01486-864d-44bf-987b-8a0c2ff6cae4 None admin] RBAC: Authorizing `identity:get_role(role_id=cb116a5614cf4a48ad99840d95f96bb3)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:15.940716 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e01486-864d-44bf-987b-8a0c2ff6cae4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:15.942019 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 560/1120] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:15 2022] GET /identity/v3/roles/cb116a5614cf4a48ad99840d95f96bb3 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:15.948025 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33923d11-6a2c-4333-b68a-237a4a852d61 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:16.043678 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33923d11-6a2c-4333-b68a-237a4a852d61 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:16.066225 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:16.076833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa2948-2fa9-4d57-99ac-091ec701a1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:16.080706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa2948-2fa9-4d57-99ac-091ec701a1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:16.080706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa2948-2fa9-4d57-99ac-091ec701a1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:16.080706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa2948-2fa9-4d57-99ac-091ec701a1e9 None admin] PATH_INFO: `/v3/users/c414b1ef68934f54afa91ea5e301b34c` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:16.080706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa2948-2fa9-4d57-99ac-091ec701a1e9 None admin] RBAC: Authorizing `identity:get_user(user_id=c414b1ef68934f54afa91ea5e301b34c)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:16.081063 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa2948-2fa9-4d57-99ac-091ec701a1e9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:16.082433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 561/1121] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:15 2022] GET /identity/v3/users/c414b1ef68934f54afa91ea5e301b34c => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:16.089064 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e01486-864d-44bf-987b-8a0c2ff6cae4 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:16.186177 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e01486-864d-44bf-987b-8a0c2ff6cae4 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:16.214845 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:16.222788 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4390ec2-aa8e-4849-81a4-b8c69bf2221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:16.223645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4390ec2-aa8e-4849-81a4-b8c69bf2221f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:16.229740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4390ec2-aa8e-4849-81a4-b8c69bf2221f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:16.229740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4390ec2-aa8e-4849-81a4-b8c69bf2221f None admin] PATH_INFO: `/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:16.229740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4390ec2-aa8e-4849-81a4-b8c69bf2221f None admin] RBAC: Authorizing `identity:get_project(project_id=b6fc64be32a84efabf1c40d0cbc26df9)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:16.229740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4390ec2-aa8e-4849-81a4-b8c69bf2221f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:16.229740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 561/1122] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:16 2022] GET /identity/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:16.234470 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa2948-2fa9-4d57-99ac-091ec701a1e9 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:16.332660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa2948-2fa9-4d57-99ac-091ec701a1e9 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:16.366736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedcc107-c1f7-449a-9a08-fd0ce0b9364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:16.367612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcc107-c1f7-449a-9a08-fd0ce0b9364d None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:16.367891 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcc107-c1f7-449a-9a08-fd0ce0b9364d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:16.368162 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedcc107-c1f7-449a-9a08-fd0ce0b9364d None admin] PATH_INFO: `/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9/users/c414b1ef68934f54afa91ea5e301b34c/roles/cb116a5614cf4a48ad99840d95f96bb3` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:16.370413 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcc107-c1f7-449a-9a08-fd0ce0b9364d None admin] RBAC: Authorizing `identity:create_grant(project_id=b6fc64be32a84efabf1c40d0cbc26df9, user_id=c414b1ef68934f54afa91ea5e301b34c, role_id=cb116a5614cf4a48ad99840d95f96bb3)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:16.371848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedcc107-c1f7-449a-9a08-fd0ce0b9364d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:16.470936 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 562/1123] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:48:16 2022] PUT /identity/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9/users/c414b1ef68934f54afa91ea5e301b34c/roles/cb116a5614cf4a48ad99840d95f96bb3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:48:18.685703 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226415e3-f47c-4d9b-ab7f-ebc8a64db2b7 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:18.686226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226415e3-f47c-4d9b-ab7f-ebc8a64db2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:18.686226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226415e3-f47c-4d9b-ab7f-ebc8a64db2b7 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:18.687527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 562/1124] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:18.692993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8e06e-e75f-4909-95ba-204c65fdb57a None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:18.693400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8e06e-e75f-4909-95ba-204c65fdb57a None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:18.693546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8e06e-e75f-4909-95ba-204c65fdb57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:18.709992 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-b2d8e06e-e75f-4909-95ba-204c65fdb57a None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:18.711160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-b2d8e06e-e75f-4909-95ba-204c65fdb57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:18.757289 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:18.776277 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 563/1125] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:48:18.799995 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226415e3-f47c-4d9b-ab7f-ebc8a64db2b7 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:18.898632 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226415e3-f47c-4d9b-ab7f-ebc8a64db2b7 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:18.931037 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa31864-17db-4037-abde-4f49941151ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:18.932436 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa31864-17db-4037-abde-4f49941151ee None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:18.932722 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa31864-17db-4037-abde-4f49941151ee None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:18.932997 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa31864-17db-4037-abde-4f49941151ee None admin] PATH_INFO: `/v3/roles/cb116a5614cf4a48ad99840d95f96bb3` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:18.934470 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa31864-17db-4037-abde-4f49941151ee None admin] RBAC: Authorizing `identity:get_role(role_id=cb116a5614cf4a48ad99840d95f96bb3)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:18.935875 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa31864-17db-4037-abde-4f49941151ee None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:18.937207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 563/1126] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:18 2022] GET /identity/v3/roles/cb116a5614cf4a48ad99840d95f96bb3 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:18.943147 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8e06e-e75f-4909-95ba-204c65fdb57a None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:19.039759 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8e06e-e75f-4909-95ba-204c65fdb57a None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:19.074327 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8180e-4b6c-4cdd-a157-d7bb25c4efbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:19.075191 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8180e-4b6c-4cdd-a157-d7bb25c4efbf None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:19.075522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8180e-4b6c-4cdd-a157-d7bb25c4efbf None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:19.075797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8180e-4b6c-4cdd-a157-d7bb25c4efbf None admin] PATH_INFO: `/v3/users/c414b1ef68934f54afa91ea5e301b34c` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:19.077099 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8180e-4b6c-4cdd-a157-d7bb25c4efbf None admin] RBAC: Authorizing `identity:get_user(user_id=c414b1ef68934f54afa91ea5e301b34c)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:19.078554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8180e-4b6c-4cdd-a157-d7bb25c4efbf None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:19.080206 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 564/1127] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:18 2022] GET /identity/v3/users/c414b1ef68934f54afa91ea5e301b34c => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:19.086111 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa31864-17db-4037-abde-4f49941151ee None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:19.149221 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:19.192188 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa31864-17db-4037-abde-4f49941151ee None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:19.225529 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a106e-9210-46f4-9821-3bc72c3f33ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:19.226175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a106e-9210-46f4-9821-3bc72c3f33ec None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:19.226461 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a106e-9210-46f4-9821-3bc72c3f33ec None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:19.226737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a106e-9210-46f4-9821-3bc72c3f33ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:19.299150 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a106e-9210-46f4-9821-3bc72c3f33ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:19.300871 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a106e-9210-46f4-9821-3bc72c3f33ec None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:19.361760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:19.401911 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 564/1128] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:48:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:48:19.407915 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8180e-4b6c-4cdd-a157-d7bb25c4efbf None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:19.457327 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:19.510462 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8180e-4b6c-4cdd-a157-d7bb25c4efbf None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:19.544651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52760a60-ec30-4b94-bec0-f17496fb17ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:19.545604 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52760a60-ec30-4b94-bec0-f17496fb17ea None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:19.545884 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52760a60-ec30-4b94-bec0-f17496fb17ea None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:19.546152 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52760a60-ec30-4b94-bec0-f17496fb17ea None admin] PATH_INFO: `/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:19.547307 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52760a60-ec30-4b94-bec0-f17496fb17ea None admin] RBAC: Authorizing `identity:get_project(project_id=b6fc64be32a84efabf1c40d0cbc26df9)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:19.548783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52760a60-ec30-4b94-bec0-f17496fb17ea None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:19.550754 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 565/1129] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:19 2022] GET /identity/v3/projects/b6fc64be32a84efabf1c40d0cbc26df9 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:19.556913 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a106e-9210-46f4-9821-3bc72c3f33ec None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:19.651599 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a106e-9210-46f4-9821-3bc72c3f33ec None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:19.686431 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3fd986-9a16-4a9d-8f2a-c95df2b6dc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:19.687063 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fd986-9a16-4a9d-8f2a-c95df2b6dc00 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:19.687339 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fd986-9a16-4a9d-8f2a-c95df2b6dc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:19.687610 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3fd986-9a16-4a9d-8f2a-c95df2b6dc00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:19.688461 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3fd986-9a16-4a9d-8f2a-c95df2b6dc00 None admin] RBAC: Adding query filter params (role.id=cb116a5614cf4a48ad99840d95f96bb3, scope.project.id=b6fc64be32a84efabf1c40d0cbc26df9, user.id=c414b1ef68934f54afa91ea5e301b34c) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:19.688772 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3fd986-9a16-4a9d-8f2a-c95df2b6dc00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:19.690188 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3fd986-9a16-4a9d-8f2a-c95df2b6dc00 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:19.695837 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 565/1130] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:48:19 2022] GET /identity/v3/role_assignments?user.id=c414b1ef68934f54afa91ea5e301b34c&scope.project.id=b6fc64be32a84efabf1c40d0cbc26df9&role.id=cb116a5614cf4a48ad99840d95f96bb3 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:21.948290 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f8f91-59c2-4b88-82c2-79208cf3f72c None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:21.948858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f8f91-59c2-4b88-82c2-79208cf3f72c None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:21.948858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f8f91-59c2-4b88-82c2-79208cf3f72c None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:21.950253 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 566/1131] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:21.955463 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bbe43-fc64-4af5-9543-a69c1a14e451 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:21.955838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bbe43-fc64-4af5-9543-a69c1a14e451 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:21.956017 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bbe43-fc64-4af5-9543-a69c1a14e451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:21.973041 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-aa2bbe43-fc64-4af5-9543-a69c1a14e451 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:21.974219 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-aa2bbe43-fc64-4af5-9543-a69c1a14e451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:21.994086 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:22.039197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 566/1132] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:22.062401 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f8f91-59c2-4b88-82c2-79208cf3f72c None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:22.163729 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f8f91-59c2-4b88-82c2-79208cf3f72c None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:22.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:22.202991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b44f94-17e4-4f76-b418-05307c5ddedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:22.202991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b44f94-17e4-4f76-b418-05307c5ddedf None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:22.202991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b44f94-17e4-4f76-b418-05307c5ddedf None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:22.202991 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b44f94-17e4-4f76-b418-05307c5ddedf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:22.203420 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b44f94-17e4-4f76-b418-05307c5ddedf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:22.203420 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b44f94-17e4-4f76-b418-05307c5ddedf None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:22.204423 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 567/1133] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:48:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:22.210327 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bbe43-fc64-4af5-9543-a69c1a14e451 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:22.306049 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bbe43-fc64-4af5-9543-a69c1a14e451 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:22.339529 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed38b5-536f-4d78-8371-26a44faf613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:22.340205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed38b5-536f-4d78-8371-26a44faf613b None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:22.340520 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed38b5-536f-4d78-8371-26a44faf613b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:22.340796 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed38b5-536f-4d78-8371-26a44faf613b None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:22.341502 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed38b5-536f-4d78-8371-26a44faf613b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:22.342915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed38b5-536f-4d78-8371-26a44faf613b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:22.433092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 567/1134] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:48:22 2022] POST /identity/v3/projects => generated 311 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:48:24.644601 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71add5-d3e2-47c3-adc3-65e626923582 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:24.645205 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71add5-d3e2-47c3-adc3-65e626923582 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:24.645205 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71add5-d3e2-47c3-adc3-65e626923582 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:24.646439 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 568/1135] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:48:24.652093 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b9b99-5145-4b0d-93f8-9e15c824f015 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:24.652556 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b9b99-5145-4b0d-93f8-9e15c824f015 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:24.652668 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b9b99-5145-4b0d-93f8-9e15c824f015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:24.669714 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-e52b9b99-5145-4b0d-93f8-9e15c824f015 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:24.670942 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-e52b9b99-5145-4b0d-93f8-9e15c824f015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:24.672893 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:24.736283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 568/1136] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:24.760581 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71add5-d3e2-47c3-adc3-65e626923582 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:24.862149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71add5-d3e2-47c3-adc3-65e626923582 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:24.894925 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bdc459-9256-4444-a44b-ecc7f7f1e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:24.895634 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdc459-9256-4444-a44b-ecc7f7f1e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:24.895913 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdc459-9256-4444-a44b-ecc7f7f1e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:24.896184 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bdc459-9256-4444-a44b-ecc7f7f1e7a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:24.897363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdc459-9256-4444-a44b-ecc7f7f1e7a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:24.898883 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bdc459-9256-4444-a44b-ecc7f7f1e7a8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:24.901031 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 569/1137] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:48:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:48:24.906943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b9b99-5145-4b0d-93f8-9e15c824f015 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:25.006852 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b9b99-5145-4b0d-93f8-9e15c824f015 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:25.040218 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccd4f0-3067-4832-a652-e9059aa25634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:25.040924 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccd4f0-3067-4832-a652-e9059aa25634 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:25.041208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccd4f0-3067-4832-a652-e9059aa25634 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:25.041481 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccd4f0-3067-4832-a652-e9059aa25634 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:25.042182 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccd4f0-3067-4832-a652-e9059aa25634 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:25.043596 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccd4f0-3067-4832-a652-e9059aa25634 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:25.052198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:25.220777 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 569/1138] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:48:24 2022] POST /identity/v3/users => generated 286 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:48:27.491947 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b0a8f-c51a-4aeb-a6f2-e49e3ae1e887 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:27.492506 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b0a8f-c51a-4aeb-a6f2-e49e3ae1e887 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:27.492506 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b0a8f-c51a-4aeb-a6f2-e49e3ae1e887 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:27.493901 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 570/1139] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:27.499117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14858291-4ded-4002-8acb-e82935080bf9 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:27.499544 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14858291-4ded-4002-8acb-e82935080bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:27.499672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14858291-4ded-4002-8acb-e82935080bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:27.516778 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-14858291-4ded-4002-8acb-e82935080bf9 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:27.517963 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-14858291-4ded-4002-8acb-e82935080bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:27.584698 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 570/1140] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:27.608978 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b0a8f-c51a-4aeb-a6f2-e49e3ae1e887 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:27.667503 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:27.713656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b0a8f-c51a-4aeb-a6f2-e49e3ae1e887 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:27.746814 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cdac6-8d78-44d7-bc54-f742e14cd781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:27.747586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cdac6-8d78-44d7-bc54-f742e14cd781 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:27.747861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cdac6-8d78-44d7-bc54-f742e14cd781 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:27.748133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cdac6-8d78-44d7-bc54-f742e14cd781 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:27.757955 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88cdac6-8d78-44d7-bc54-f742e14cd781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-b88cdac6-8d78-44d7-bc54-f742e14cd781 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:48:27.767288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:27.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:27.769745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 571/1141] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:48:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:27.774641 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14858291-4ded-4002-8acb-e82935080bf9 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:27.872140 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14858291-4ded-4002-8acb-e82935080bf9 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:27.895462 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:27.906424 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2ba53-529a-40dd-9caf-df1c9b7389d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:27.916811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2ba53-529a-40dd-9caf-df1c9b7389d6 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:27.916811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2ba53-529a-40dd-9caf-df1c9b7389d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:27.916811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2ba53-529a-40dd-9caf-df1c9b7389d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:27.916811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2ba53-529a-40dd-9caf-df1c9b7389d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:27.916811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2ba53-529a-40dd-9caf-df1c9b7389d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:27.916811 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2ba53-529a-40dd-9caf-df1c9b7389d6 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:27.918247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 571/1142] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:48:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:27.924204 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cdac6-8d78-44d7-bc54-f742e14cd781 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:28.024780 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cdac6-8d78-44d7-bc54-f742e14cd781 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:28.038101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:28.058916 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bba46e-89dd-451c-85d2-e3650cf8d09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:28.059785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba46e-89dd-451c-85d2-e3650cf8d09f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:28.060101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba46e-89dd-451c-85d2-e3650cf8d09f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:28.060422 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bba46e-89dd-451c-85d2-e3650cf8d09f None admin] PATH_INFO: `/v3/users/ca001a0a84b84830a758fac93e5eeba1` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:28.073887 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bba46e-89dd-451c-85d2-e3650cf8d09f None admin] RBAC: Authorizing `identity:get_user(user_id=ca001a0a84b84830a758fac93e5eeba1)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:28.075339 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bba46e-89dd-451c-85d2-e3650cf8d09f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:28.077038 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 572/1143] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:27 2022] GET /identity/v3/users/ca001a0a84b84830a758fac93e5eeba1 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:28.082801 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2ba53-529a-40dd-9caf-df1c9b7389d6 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:28.180694 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2ba53-529a-40dd-9caf-df1c9b7389d6 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:28.214598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e3ff2-cb73-4f36-a9fd-2a168a100943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:28.215240 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3ff2-cb73-4f36-a9fd-2a168a100943 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:28.215523 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3ff2-cb73-4f36-a9fd-2a168a100943 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:28.215798 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3ff2-cb73-4f36-a9fd-2a168a100943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:28.268613 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:28.281887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e3ff2-cb73-4f36-a9fd-2a168a100943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:28.283331 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e3ff2-cb73-4f36-a9fd-2a168a100943 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:28.380228 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 572/1144] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:48:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:48:28.386541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bba46e-89dd-451c-85d2-e3650cf8d09f None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:28.483172 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bba46e-89dd-451c-85d2-e3650cf8d09f None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:28.516739 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794db29-42b4-45e5-9e8d-ce1b89e4d72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:28.517579 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794db29-42b4-45e5-9e8d-ce1b89e4d72f None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:28.517860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794db29-42b4-45e5-9e8d-ce1b89e4d72f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:28.518136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794db29-42b4-45e5-9e8d-ce1b89e4d72f None admin] PATH_INFO: `/v3/projects/5c18ffa35d8d4101ad1052b67f34db42` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:28.519306 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794db29-42b4-45e5-9e8d-ce1b89e4d72f None admin] RBAC: Authorizing `identity:get_project(project_id=5c18ffa35d8d4101ad1052b67f34db42)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:28.520771 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794db29-42b4-45e5-9e8d-ce1b89e4d72f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:28.522452 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 573/1145] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:28 2022] GET /identity/v3/projects/5c18ffa35d8d4101ad1052b67f34db42 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:28.528677 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e3ff2-cb73-4f36-a9fd-2a168a100943 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:28.610172 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:28.624685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e3ff2-cb73-4f36-a9fd-2a168a100943 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:28.657208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab67b7-99c7-4a17-97e8-0e3dcf4a9980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:28.657870 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab67b7-99c7-4a17-97e8-0e3dcf4a9980 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:28.658126 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab67b7-99c7-4a17-97e8-0e3dcf4a9980 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:28.658402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab67b7-99c7-4a17-97e8-0e3dcf4a9980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:28.659232 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab67b7-99c7-4a17-97e8-0e3dcf4a9980 None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=5c18ffa35d8d4101ad1052b67f34db42, user.id=ca001a0a84b84830a758fac93e5eeba1) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:28.659752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab67b7-99c7-4a17-97e8-0e3dcf4a9980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:28.661783 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab67b7-99c7-4a17-97e8-0e3dcf4a9980 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:28.667177 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 573/1146] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:48:28 2022] GET /identity/v3/role_assignments?user.id=ca001a0a84b84830a758fac93e5eeba1&scope.project.id=5c18ffa35d8d4101ad1052b67f34db42&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:30.852624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111778b-0d1b-4de4-ad4a-0d76a2d3451d None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:30.853205 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111778b-0d1b-4de4-ad4a-0d76a2d3451d None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:30.853205 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111778b-0d1b-4de4-ad4a-0d76a2d3451d None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:30.854599 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 574/1147] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:30.860510 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281a943-f6ae-4830-851f-2e9d19e7fb5e None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:30.860897 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281a943-f6ae-4830-851f-2e9d19e7fb5e None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:30.861075 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281a943-f6ae-4830-851f-2e9d19e7fb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:30.877615 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-4281a943-f6ae-4830-851f-2e9d19e7fb5e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:30.878799 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-4281a943-f6ae-4830-851f-2e9d19e7fb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:30.947303 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 574/1148] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:30.970875 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111778b-0d1b-4de4-ad4a-0d76a2d3451d None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:31.014108 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:31.071408 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111778b-0d1b-4de4-ad4a-0d76a2d3451d None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:31.105792 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e62458-94d9-41c3-98f8-21cfc459bc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:31.106487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e62458-94d9-41c3-98f8-21cfc459bc12 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:31.106766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e62458-94d9-41c3-98f8-21cfc459bc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:31.107038 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e62458-94d9-41c3-98f8-21cfc459bc12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:31.116859 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e62458-94d9-41c3-98f8-21cfc459bc12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-b8e62458-94d9-41c3-98f8-21cfc459bc12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:48:31.124454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:31.125582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:31.126626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 575/1149] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:48:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:31.131562 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281a943-f6ae-4830-851f-2e9d19e7fb5e None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:31.209207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:31.227631 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281a943-f6ae-4830-851f-2e9d19e7fb5e None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:31.260490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebae73-a939-4cee-bbfc-4f370d9ec54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:31.261149 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebae73-a939-4cee-bbfc-4f370d9ec54b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:31.261421 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebae73-a939-4cee-bbfc-4f370d9ec54b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:31.261729 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebae73-a939-4cee-bbfc-4f370d9ec54b None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:31.262582 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebae73-a939-4cee-bbfc-4f370d9ec54b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:31.263122 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebae73-a939-4cee-bbfc-4f370d9ec54b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:31.265060 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebae73-a939-4cee-bbfc-4f370d9ec54b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:31.273117 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 575/1150] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:31.279664 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e62458-94d9-41c3-98f8-21cfc459bc12 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:31.376034 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:31.378914 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e62458-94d9-41c3-98f8-21cfc459bc12 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:31.413581 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145705a3-5cbe-4707-8620-5c4345e4e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:31.414434 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145705a3-5cbe-4707-8620-5c4345e4e33a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:31.414717 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145705a3-5cbe-4707-8620-5c4345e4e33a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:31.414988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145705a3-5cbe-4707-8620-5c4345e4e33a None admin] PATH_INFO: `/v3/users/ca001a0a84b84830a758fac93e5eeba1` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:31.416196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145705a3-5cbe-4707-8620-5c4345e4e33a None admin] RBAC: Authorizing `identity:get_user(user_id=ca001a0a84b84830a758fac93e5eeba1)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:31.417662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145705a3-5cbe-4707-8620-5c4345e4e33a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:31.419271 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 576/1151] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:31 2022] GET /identity/v3/users/ca001a0a84b84830a758fac93e5eeba1 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:31.425032 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebae73-a939-4cee-bbfc-4f370d9ec54b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:31.520972 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebae73-a939-4cee-bbfc-4f370d9ec54b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:31.552518 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74965081-5552-4b15-a5d8-565c6ea6a0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:31.553365 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74965081-5552-4b15-a5d8-565c6ea6a0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:31.553642 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74965081-5552-4b15-a5d8-565c6ea6a0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:31.553909 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74965081-5552-4b15-a5d8-565c6ea6a0c0 None admin] PATH_INFO: `/v3/projects/5c18ffa35d8d4101ad1052b67f34db42` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:31.555065 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74965081-5552-4b15-a5d8-565c6ea6a0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=5c18ffa35d8d4101ad1052b67f34db42)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:31.557013 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74965081-5552-4b15-a5d8-565c6ea6a0c0 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:31.558681 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 576/1152] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:31 2022] GET /identity/v3/projects/5c18ffa35d8d4101ad1052b67f34db42 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:31.564637 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145705a3-5cbe-4707-8620-5c4345e4e33a None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:31.661396 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145705a3-5cbe-4707-8620-5c4345e4e33a None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:31.695020 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27be24-dcd5-46cb-8d51-6cfed3134471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:31.695885 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27be24-dcd5-46cb-8d51-6cfed3134471 None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:31.696161 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27be24-dcd5-46cb-8d51-6cfed3134471 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:31.696465 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27be24-dcd5-46cb-8d51-6cfed3134471 None admin] PATH_INFO: `/v3/projects/5c18ffa35d8d4101ad1052b67f34db42/users/ca001a0a84b84830a758fac93e5eeba1/roles/815bd48c3df0405babfd5b53430fd3ea` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:31.698908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27be24-dcd5-46cb-8d51-6cfed3134471 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c18ffa35d8d4101ad1052b67f34db42, user_id=ca001a0a84b84830a758fac93e5eeba1, role_id=815bd48c3df0405babfd5b53430fd3ea)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:31.700326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27be24-dcd5-46cb-8d51-6cfed3134471 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:31.819941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 577/1153] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:48:31 2022] PUT /identity/v3/projects/5c18ffa35d8d4101ad1052b67f34db42/users/ca001a0a84b84830a758fac93e5eeba1/roles/815bd48c3df0405babfd5b53430fd3ea => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:48:34.046192 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fe8cd-b77e-4ccd-9116-1daa1cddabc4 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:34.046785 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fe8cd-b77e-4ccd-9116-1daa1cddabc4 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:34.046785 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fe8cd-b77e-4ccd-9116-1daa1cddabc4 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:34.048084 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 577/1154] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:34.053552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b024b34-a321-4f92-a5d0-2a17b8e3d4bb None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:34.053948 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b024b34-a321-4f92-a5d0-2a17b8e3d4bb None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:34.054097 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b024b34-a321-4f92-a5d0-2a17b8e3d4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:34.070541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-5b024b34-a321-4f92-a5d0-2a17b8e3d4bb None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:34.071704 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-5b024b34-a321-4f92-a5d0-2a17b8e3d4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:34.094973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:34.136718 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 578/1155] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:34.159577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fe8cd-b77e-4ccd-9116-1daa1cddabc4 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:34.172221 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:34.258981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fe8cd-b77e-4ccd-9116-1daa1cddabc4 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:34.292435 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6463aba-022d-42eb-9993-77f72b3eb73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:34.293218 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6463aba-022d-42eb-9993-77f72b3eb73a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:34.293505 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6463aba-022d-42eb-9993-77f72b3eb73a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:34.293786 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6463aba-022d-42eb-9993-77f72b3eb73a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:34.304020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6463aba-022d-42eb-9993-77f72b3eb73a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-f6463aba-022d-42eb-9993-77f72b3eb73a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:48:34.311568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:34.312825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:48:34.313748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 578/1156] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:48:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:34.319123 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b024b34-a321-4f92-a5d0-2a17b8e3d4bb None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:34.416282 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b024b34-a321-4f92-a5d0-2a17b8e3d4bb None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:34.450249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e14079-f7e6-43da-9a43-db7018162b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:34.450905 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14079-f7e6-43da-9a43-db7018162b20 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:34.451186 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14079-f7e6-43da-9a43-db7018162b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:34.451461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e14079-f7e6-43da-9a43-db7018162b20 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:34.452200 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e14079-f7e6-43da-9a43-db7018162b20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:34.452534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e14079-f7e6-43da-9a43-db7018162b20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:34.453904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e14079-f7e6-43da-9a43-db7018162b20 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:34.461887 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 579/1157] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:48:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:34.467766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6463aba-022d-42eb-9993-77f72b3eb73a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:34.536029 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:34.566185 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6463aba-022d-42eb-9993-77f72b3eb73a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:34.598846 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b942d46-8ed7-4163-b4bd-9f07a5725b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:34.599702 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b942d46-8ed7-4163-b4bd-9f07a5725b92 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:34.599981 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b942d46-8ed7-4163-b4bd-9f07a5725b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:34.600249 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b942d46-8ed7-4163-b4bd-9f07a5725b92 None admin] PATH_INFO: `/v3/users/ca001a0a84b84830a758fac93e5eeba1` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:34.601483 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b942d46-8ed7-4163-b4bd-9f07a5725b92 None admin] RBAC: Authorizing `identity:get_user(user_id=ca001a0a84b84830a758fac93e5eeba1)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:34.602889 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b942d46-8ed7-4163-b4bd-9f07a5725b92 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:34.604678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 579/1158] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:34 2022] GET /identity/v3/users/ca001a0a84b84830a758fac93e5eeba1 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:34.610592 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e14079-f7e6-43da-9a43-db7018162b20 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:34.632674 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:34.709126 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e14079-f7e6-43da-9a43-db7018162b20 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:34.742545 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c78227-0fe1-4822-a6ed-1fbc4636d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:34.743173 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c78227-0fe1-4822-a6ed-1fbc4636d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:34.743448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c78227-0fe1-4822-a6ed-1fbc4636d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:34.743717 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c78227-0fe1-4822-a6ed-1fbc4636d4b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:34.809884 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c78227-0fe1-4822-a6ed-1fbc4636d4b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:34.811319 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c78227-0fe1-4822-a6ed-1fbc4636d4b3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:34.831329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:34.907328 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 580/1159] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:48:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:48:34.913177 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b942d46-8ed7-4163-b4bd-9f07a5725b92 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:35.011427 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b942d46-8ed7-4163-b4bd-9f07a5725b92 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:35.043766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385cd56-32f7-4739-ba8e-f827b27d82c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:35.044676 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385cd56-32f7-4739-ba8e-f827b27d82c5 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:35.044964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385cd56-32f7-4739-ba8e-f827b27d82c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:35.045238 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385cd56-32f7-4739-ba8e-f827b27d82c5 None admin] PATH_INFO: `/v3/projects/5c18ffa35d8d4101ad1052b67f34db42` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:35.046413 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385cd56-32f7-4739-ba8e-f827b27d82c5 None admin] RBAC: Authorizing `identity:get_project(project_id=5c18ffa35d8d4101ad1052b67f34db42)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:35.047823 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385cd56-32f7-4739-ba8e-f827b27d82c5 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:35.049551 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 580/1160] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:34 2022] GET /identity/v3/projects/5c18ffa35d8d4101ad1052b67f34db42 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:35.056009 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c78227-0fe1-4822-a6ed-1fbc4636d4b3 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:35.153470 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c78227-0fe1-4822-a6ed-1fbc4636d4b3 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:35.172903 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:35.186503 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4db8564-484d-4144-8bee-9137d0aebf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:35.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db8564-484d-4144-8bee-9137d0aebf43 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:35.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db8564-484d-4144-8bee-9137d0aebf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:35.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db8564-484d-4144-8bee-9137d0aebf43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:35.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db8564-484d-4144-8bee-9137d0aebf43 None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=5c18ffa35d8d4101ad1052b67f34db42, user.id=ca001a0a84b84830a758fac93e5eeba1) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:35.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db8564-484d-4144-8bee-9137d0aebf43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:35.192945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4db8564-484d-4144-8bee-9137d0aebf43 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:35.196136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 581/1161] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:48:35 2022] GET /identity/v3/role_assignments?user.id=ca001a0a84b84830a758fac93e5eeba1&scope.project.id=5c18ffa35d8d4101ad1052b67f34db42&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:37.416679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21463f5-ed84-44c5-b3cf-fcef1817963b None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:37.417275 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21463f5-ed84-44c5-b3cf-fcef1817963b None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:37.417275 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21463f5-ed84-44c5-b3cf-fcef1817963b None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:37.418634 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 581/1162] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:37.423718 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814d12-5e48-4178-a004-6a72ff1ce17f None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:37.424139 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814d12-5e48-4178-a004-6a72ff1ce17f None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:37.424263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9814d12-5e48-4178-a004-6a72ff1ce17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:37.441851 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-b9814d12-5e48-4178-a004-6a72ff1ce17f None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:37.443029 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-b9814d12-5e48-4178-a004-6a72ff1ce17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:37.508883 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 582/1163] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:37.532119 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21463f5-ed84-44c5-b3cf-fcef1817963b None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:37.542108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:37.630776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21463f5-ed84-44c5-b3cf-fcef1817963b None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:37.663476 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0b172-7912-4460-8782-de3017b9cda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:37.664160 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0b172-7912-4460-8782-de3017b9cda9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:37.664470 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0b172-7912-4460-8782-de3017b9cda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:37.664745 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a0b172-7912-4460-8782-de3017b9cda9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:37.729685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0b172-7912-4460-8782-de3017b9cda9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:37.731109 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a0b172-7912-4460-8782-de3017b9cda9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:37.738468 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:37.827023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 582/1164] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:48:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:48:37.833130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9814d12-5e48-4178-a004-6a72ff1ce17f None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:37.936562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9814d12-5e48-4178-a004-6a72ff1ce17f None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:37.950097 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:37.969069 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9bef3f-7dd0-4455-858e-d98846e64a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:37.969859 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bef3f-7dd0-4455-858e-d98846e64a7c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:37.970140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bef3f-7dd0-4455-858e-d98846e64a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:37.970411 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9bef3f-7dd0-4455-858e-d98846e64a7c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:37.977165 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9bef3f-7dd0-4455-858e-d98846e64a7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:37.979204 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9bef3f-7dd0-4455-858e-d98846e64a7c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-de9bef3f-7dd0-4455-858e-d98846e64a7c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:37.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:37.995554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:37.996801 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 583/1165] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 19:48:37 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:38.001541 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0b172-7912-4460-8782-de3017b9cda9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:38.097096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a0b172-7912-4460-8782-de3017b9cda9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:38.104828 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:38.130514 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eed03e-b003-43e0-b1df-7002f421e654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:38.131158 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eed03e-b003-43e0-b1df-7002f421e654 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:38.131437 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eed03e-b003-43e0-b1df-7002f421e654 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:38.131708 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eed03e-b003-43e0-b1df-7002f421e654 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:38.132445 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eed03e-b003-43e0-b1df-7002f421e654 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:38.132748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eed03e-b003-43e0-b1df-7002f421e654 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:38.134163 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eed03e-b003-43e0-b1df-7002f421e654 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:38.142277 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 583/1166] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 19:48:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:40.319430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130953b-2a89-42d5-98c8-13be0665fbb4 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:40.320010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130953b-2a89-42d5-98c8-13be0665fbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:40.320010 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130953b-2a89-42d5-98c8-13be0665fbb4 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:40.321369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 584/1167] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:40.327198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b6ea3-4a85-4d26-a183-3f30770e4adc None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:40.327640 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b6ea3-4a85-4d26-a183-3f30770e4adc None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:40.327749 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b6ea3-4a85-4d26-a183-3f30770e4adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:40.346988 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-286b6ea3-4a85-4d26-a183-3f30770e4adc None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:40.348168 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-286b6ea3-4a85-4d26-a183-3f30770e4adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:40.416413 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 584/1168] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:40.439432 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130953b-2a89-42d5-98c8-13be0665fbb4 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:40.538530 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130953b-2a89-42d5-98c8-13be0665fbb4 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:40.574094 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a0eba-47b8-4350-8214-f3e3d150772e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:40.574676 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a0eba-47b8-4350-8214-f3e3d150772e None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:40.574958 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a0eba-47b8-4350-8214-f3e3d150772e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:40.575233 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a0eba-47b8-4350-8214-f3e3d150772e None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:40.575975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a0eba-47b8-4350-8214-f3e3d150772e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:40.577392 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a0eba-47b8-4350-8214-f3e3d150772e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:40.690359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 585/1169] 10.222.0.88 () {60 vars in 1138 bytes} [Thu May 19 19:48:40 2022] POST /identity/v3/domains => generated 267 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:48:42.934532 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a2490-b792-4062-9c81-20b286e11f08 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:42.935135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a2490-b792-4062-9c81-20b286e11f08 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:42.935135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a2490-b792-4062-9c81-20b286e11f08 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:42.936487 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 585/1170] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:42.941846 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483075-3988-4b7f-89a3-e81e92d13fc5 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:42.942296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483075-3988-4b7f-89a3-e81e92d13fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:42.942435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40483075-3988-4b7f-89a3-e81e92d13fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:42.958876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-40483075-3988-4b7f-89a3-e81e92d13fc5 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:42.960044 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-40483075-3988-4b7f-89a3-e81e92d13fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:42.986811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:43.026426 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 586/1171] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:43.057889 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a2490-b792-4062-9c81-20b286e11f08 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:43.145596 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:43.149419 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a2490-b792-4062-9c81-20b286e11f08 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:43.184188 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fa5df-1c65-4242-bcac-b8459e87829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:43.185131 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fa5df-1c65-4242-bcac-b8459e87829f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:43.185369 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fa5df-1c65-4242-bcac-b8459e87829f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:43.185646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fa5df-1c65-4242-bcac-b8459e87829f None admin] PATH_INFO: `/v3/domains/0ca4865ed314438c8a292e4a34eda658` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:43.196133 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fa5df-1c65-4242-bcac-b8459e87829f None admin] RBAC: Authorizing `identity:get_domain(domain_id=0ca4865ed314438c8a292e4a34eda658)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:43.198154 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fa5df-1c65-4242-bcac-b8459e87829f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:43.199808 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 586/1172] 10.222.0.88 () {56 vars in 1183 bytes} [Thu May 19 19:48:43 2022] GET /identity/v3/domains/0ca4865ed314438c8a292e4a34eda658 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:43.207187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40483075-3988-4b7f-89a3-e81e92d13fc5 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:43.305221 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40483075-3988-4b7f-89a3-e81e92d13fc5 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:43.338874 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769fc01-e583-47e6-9587-ae9a66d3e9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:43.339602 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769fc01-e583-47e6-9587-ae9a66d3e9e3 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:43.339887 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769fc01-e583-47e6-9587-ae9a66d3e9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:43.340157 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769fc01-e583-47e6-9587-ae9a66d3e9e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:43.340896 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769fc01-e583-47e6-9587-ae9a66d3e9e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:43.342333 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769fc01-e583-47e6-9587-ae9a66d3e9e3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:43.456286 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 587/1173] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:48:43 2022] POST /identity/v3/projects => generated 361 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:48:45.694396 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c824234-1aba-4784-b91d-3fbfb2d23e9c None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:45.694896 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c824234-1aba-4784-b91d-3fbfb2d23e9c None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:45.694896 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c824234-1aba-4784-b91d-3fbfb2d23e9c None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:45.696254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 587/1174] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:45.702217 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84705168-0d28-4aa0-b2c7-e3ca4d00b1e3 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:45.702624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84705168-0d28-4aa0-b2c7-e3ca4d00b1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:45.702766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84705168-0d28-4aa0-b2c7-e3ca4d00b1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:45.719759 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-84705168-0d28-4aa0-b2c7-e3ca4d00b1e3 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:45.720963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-84705168-0d28-4aa0-b2c7-e3ca4d00b1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:45.731449 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:45.786640 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 588/1175] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:45.812273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c824234-1aba-4784-b91d-3fbfb2d23e9c None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:45.910260 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c824234-1aba-4784-b91d-3fbfb2d23e9c None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:45.943025 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f85e1-01bc-4deb-895c-deb375fdabd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:45.943890 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f85e1-01bc-4deb-895c-deb375fdabd4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:45.944167 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f85e1-01bc-4deb-895c-deb375fdabd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:45.944524 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f85e1-01bc-4deb-895c-deb375fdabd4 None admin] PATH_INFO: `/v3/domains/0ca4865ed314438c8a292e4a34eda658` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:45.945649 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f85e1-01bc-4deb-895c-deb375fdabd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0ca4865ed314438c8a292e4a34eda658)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:45.947120 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f85e1-01bc-4deb-895c-deb375fdabd4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:45.949335 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 588/1176] 10.222.0.88 () {56 vars in 1183 bytes} [Thu May 19 19:48:45 2022] GET /identity/v3/domains/0ca4865ed314438c8a292e4a34eda658 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:45.955239 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84705168-0d28-4aa0-b2c7-e3ca4d00b1e3 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:46.052074 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84705168-0d28-4aa0-b2c7-e3ca4d00b1e3 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:46.084883 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc37ee1-8b8a-4582-aa34-4968f523ebc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:46.085540 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc37ee1-8b8a-4582-aa34-4968f523ebc4 None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:46.085821 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc37ee1-8b8a-4582-aa34-4968f523ebc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:46.086095 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc37ee1-8b8a-4582-aa34-4968f523ebc4 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:46.086791 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc37ee1-8b8a-4582-aa34-4968f523ebc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:46.088324 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc37ee1-8b8a-4582-aa34-4968f523ebc4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:46.174869 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 589/1177] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:48:45 2022] POST /identity/v3/users => generated 311 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:48:48.422162 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81924595-fd5f-4036-8b4a-22338f69e56f None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:48.422685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81924595-fd5f-4036-8b4a-22338f69e56f None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:48.422685 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81924595-fd5f-4036-8b4a-22338f69e56f None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:48.424063 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 589/1178] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:48.429534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcc7c7-894d-4b85-8efe-9c30cda2d207 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:48.429953 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcc7c7-894d-4b85-8efe-9c30cda2d207 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:48.430088 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcc7c7-894d-4b85-8efe-9c30cda2d207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:48.435384 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:48.447739 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-38fcc7c7-894d-4b85-8efe-9c30cda2d207 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:48.448957 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-38fcc7c7-894d-4b85-8efe-9c30cda2d207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:48.514583 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 590/1179] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:48.538297 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81924595-fd5f-4036-8b4a-22338f69e56f None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:48.571211 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:48.648678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81924595-fd5f-4036-8b4a-22338f69e56f None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:48.682363 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4d32f-1725-4ec3-b6e5-fb70517f98c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:48.683139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4d32f-1725-4ec3-b6e5-fb70517f98c7 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:48.683421 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4d32f-1725-4ec3-b6e5-fb70517f98c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:48.683697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4d32f-1725-4ec3-b6e5-fb70517f98c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:48.693719 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4d32f-1725-4ec3-b6e5-fb70517f98c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-ffd4d32f-1725-4ec3-b6e5-fb70517f98c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:48:48.701936 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:48.703055 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:48:48.704130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 590/1180] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:48:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:48.709265 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcc7c7-894d-4b85-8efe-9c30cda2d207 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:48.805650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcc7c7-894d-4b85-8efe-9c30cda2d207 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:48.834049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:48.839397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f57a0e-85e8-464e-b8c7-12506520895d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:48.840047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f57a0e-85e8-464e-b8c7-12506520895d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:48.840325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f57a0e-85e8-464e-b8c7-12506520895d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:48.844367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f57a0e-85e8-464e-b8c7-12506520895d None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:48.844367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f57a0e-85e8-464e-b8c7-12506520895d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:48.844367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f57a0e-85e8-464e-b8c7-12506520895d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:48.844367 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f57a0e-85e8-464e-b8c7-12506520895d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:48.851085 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 591/1181] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:48:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:48.857710 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4d32f-1725-4ec3-b6e5-fb70517f98c7 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:48.950203 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:48.961793 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4d32f-1725-4ec3-b6e5-fb70517f98c7 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:48.994988 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1910df-4e9c-41b9-b954-e23779c1b520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:48.995859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1910df-4e9c-41b9-b954-e23779c1b520 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:48.996137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1910df-4e9c-41b9-b954-e23779c1b520 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:48.996438 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1910df-4e9c-41b9-b954-e23779c1b520 None admin] PATH_INFO: `/v3/users/615348dde6874afca87dcbe16ec15f2d` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:49.009118 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1910df-4e9c-41b9-b954-e23779c1b520 None admin] RBAC: Authorizing `identity:get_user(user_id=615348dde6874afca87dcbe16ec15f2d)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:49.011263 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1910df-4e9c-41b9-b954-e23779c1b520 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:49.012923 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 591/1182] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:48 2022] GET /identity/v3/users/615348dde6874afca87dcbe16ec15f2d => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:49.018650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f57a0e-85e8-464e-b8c7-12506520895d None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:49.116467 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f57a0e-85e8-464e-b8c7-12506520895d None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:49.149007 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2991f12-48bd-406c-8ea8-c40124be9ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:49.149641 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2991f12-48bd-406c-8ea8-c40124be9ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:49.149924 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2991f12-48bd-406c-8ea8-c40124be9ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:49.150239 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2991f12-48bd-406c-8ea8-c40124be9ea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:49.208539 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:49.215607 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2991f12-48bd-406c-8ea8-c40124be9ea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:49.217170 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2991f12-48bd-406c-8ea8-c40124be9ea7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:49.313774 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 592/1183] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:48:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:48:49.319697 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1910df-4e9c-41b9-b954-e23779c1b520 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:49.418583 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1910df-4e9c-41b9-b954-e23779c1b520 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:49.454912 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f1de3-c4cc-48f9-9916-824bf825cc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:49.455766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f1de3-c4cc-48f9-9916-824bf825cc8a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:49.456049 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f1de3-c4cc-48f9-9916-824bf825cc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:49.456324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f1de3-c4cc-48f9-9916-824bf825cc8a None admin] PATH_INFO: `/v3/projects/de32aa5e2bc84271be2c53d4bf0b8d7c` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:49.457534 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f1de3-c4cc-48f9-9916-824bf825cc8a None admin] RBAC: Authorizing `identity:get_project(project_id=de32aa5e2bc84271be2c53d4bf0b8d7c)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:49.459005 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f1de3-c4cc-48f9-9916-824bf825cc8a None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:49.460689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 592/1184] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:49 2022] GET /identity/v3/projects/de32aa5e2bc84271be2c53d4bf0b8d7c => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:49.467298 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2991f12-48bd-406c-8ea8-c40124be9ea7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:49.558316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:49.564795 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2991f12-48bd-406c-8ea8-c40124be9ea7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:49.597994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367add4-9f55-4dc7-810b-85eef2e3f22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:49.598635 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367add4-9f55-4dc7-810b-85eef2e3f22a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:49.598918 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367add4-9f55-4dc7-810b-85eef2e3f22a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:49.599195 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367add4-9f55-4dc7-810b-85eef2e3f22a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:49.600018 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367add4-9f55-4dc7-810b-85eef2e3f22a None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=de32aa5e2bc84271be2c53d4bf0b8d7c, user.id=615348dde6874afca87dcbe16ec15f2d) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:49.600429 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367add4-9f55-4dc7-810b-85eef2e3f22a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:49.601871 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367add4-9f55-4dc7-810b-85eef2e3f22a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:49.608096 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 593/1185] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:48:49 2022] GET /identity/v3/role_assignments?user.id=615348dde6874afca87dcbe16ec15f2d&scope.project.id=de32aa5e2bc84271be2c53d4bf0b8d7c&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:51.820140 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf572a6-8172-427a-a16e-e045d2e4a2fe None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:51.820729 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf572a6-8172-427a-a16e-e045d2e4a2fe None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:51.820729 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf572a6-8172-427a-a16e-e045d2e4a2fe None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:51.822083 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 593/1186] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:51.827501 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d1d9-baab-4183-bffb-66ca018141cf None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:51.827928 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d1d9-baab-4183-bffb-66ca018141cf None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:51.828049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d1d9-baab-4183-bffb-66ca018141cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:51.845160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-4032d1d9-baab-4183-bffb-66ca018141cf None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:51.846353 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-4032d1d9-baab-4183-bffb-66ca018141cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:51.912397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 594/1187] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:51.936073 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf572a6-8172-427a-a16e-e045d2e4a2fe None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:51.952295 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:52.037933 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf572a6-8172-427a-a16e-e045d2e4a2fe None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:52.072266 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528e70a-ae1d-472f-945f-8cef9bacf5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:52.072998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528e70a-ae1d-472f-945f-8cef9bacf5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:52.073283 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528e70a-ae1d-472f-945f-8cef9bacf5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:52.073561 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528e70a-ae1d-472f-945f-8cef9bacf5f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:52.083535 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528e70a-ae1d-472f-945f-8cef9bacf5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-4528e70a-ae1d-472f-945f-8cef9bacf5f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:48:52.091364 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:52.092598 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:48:52.093712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 594/1188] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:48:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:52.102179 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d1d9-baab-4183-bffb-66ca018141cf None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:52.188251 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:52.199752 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d1d9-baab-4183-bffb-66ca018141cf None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:52.233362 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080f356-87ea-469b-a9e9-bfb0f6df1218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:52.240605 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080f356-87ea-469b-a9e9-bfb0f6df1218 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:52.240605 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080f356-87ea-469b-a9e9-bfb0f6df1218 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:52.240605 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080f356-87ea-469b-a9e9-bfb0f6df1218 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:52.240605 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080f356-87ea-469b-a9e9-bfb0f6df1218 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:52.240605 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080f356-87ea-469b-a9e9-bfb0f6df1218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:52.240605 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0080f356-87ea-469b-a9e9-bfb0f6df1218 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:52.245529 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 595/1189] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:48:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:52.251395 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528e70a-ae1d-472f-945f-8cef9bacf5f7 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:52.326102 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:52.350190 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528e70a-ae1d-472f-945f-8cef9bacf5f7 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:52.383966 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f74a70-6615-4aea-8f4a-81768901abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:52.384859 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f74a70-6615-4aea-8f4a-81768901abf9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:52.385136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f74a70-6615-4aea-8f4a-81768901abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:52.385405 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f74a70-6615-4aea-8f4a-81768901abf9 None admin] PATH_INFO: `/v3/users/615348dde6874afca87dcbe16ec15f2d` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:52.386583 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f74a70-6615-4aea-8f4a-81768901abf9 None admin] RBAC: Authorizing `identity:get_user(user_id=615348dde6874afca87dcbe16ec15f2d)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:52.388002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f74a70-6615-4aea-8f4a-81768901abf9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:52.389684 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 595/1190] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:52 2022] GET /identity/v3/users/615348dde6874afca87dcbe16ec15f2d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:52.396885 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080f356-87ea-469b-a9e9-bfb0f6df1218 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:52.496798 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0080f356-87ea-469b-a9e9-bfb0f6df1218 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:52.530169 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25429561-30be-4831-99a2-fb89c19fb02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:52.531014 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25429561-30be-4831-99a2-fb89c19fb02e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:52.531288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25429561-30be-4831-99a2-fb89c19fb02e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:52.531555 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25429561-30be-4831-99a2-fb89c19fb02e None admin] PATH_INFO: `/v3/projects/de32aa5e2bc84271be2c53d4bf0b8d7c` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:52.532753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25429561-30be-4831-99a2-fb89c19fb02e None admin] RBAC: Authorizing `identity:get_project(project_id=de32aa5e2bc84271be2c53d4bf0b8d7c)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:52.534189 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25429561-30be-4831-99a2-fb89c19fb02e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:52.535861 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 596/1191] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:52 2022] GET /identity/v3/projects/de32aa5e2bc84271be2c53d4bf0b8d7c => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:52.542319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f74a70-6615-4aea-8f4a-81768901abf9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:52.637862 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f74a70-6615-4aea-8f4a-81768901abf9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:52.670502 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf8aa6-519f-4c20-9ed5-41e7632ad981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:52.671368 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf8aa6-519f-4c20-9ed5-41e7632ad981 None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:52.671648 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf8aa6-519f-4c20-9ed5-41e7632ad981 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:52.671920 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf8aa6-519f-4c20-9ed5-41e7632ad981 None admin] PATH_INFO: `/v3/projects/de32aa5e2bc84271be2c53d4bf0b8d7c/users/615348dde6874afca87dcbe16ec15f2d/roles/815bd48c3df0405babfd5b53430fd3ea` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:52.673930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf8aa6-519f-4c20-9ed5-41e7632ad981 None admin] RBAC: Authorizing `identity:create_grant(project_id=de32aa5e2bc84271be2c53d4bf0b8d7c, user_id=615348dde6874afca87dcbe16ec15f2d, role_id=815bd48c3df0405babfd5b53430fd3ea)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:52.675343 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf8aa6-519f-4c20-9ed5-41e7632ad981 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:52.757065 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 596/1192] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:48:52 2022] PUT /identity/v3/projects/de32aa5e2bc84271be2c53d4bf0b8d7c/users/615348dde6874afca87dcbe16ec15f2d/roles/815bd48c3df0405babfd5b53430fd3ea => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:48:54.936840 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935f2ab-1d14-4dee-af47-e8359e4603fb None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:54.937393 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935f2ab-1d14-4dee-af47-e8359e4603fb None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:54.937393 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935f2ab-1d14-4dee-af47-e8359e4603fb None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:54.938831 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 597/1193] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:48:54.943943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f2ed-ce05-412c-a2af-426bdab9f4db None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:54.944392 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f2ed-ce05-412c-a2af-426bdab9f4db None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:54.944538 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f2ed-ce05-412c-a2af-426bdab9f4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:54.950270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:54.962178 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-38b9f2ed-ce05-412c-a2af-426bdab9f4db None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:48:54.963355 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-38b9f2ed-ce05-412c-a2af-426bdab9f4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:48:55.029049 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 597/1194] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:48:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:48:55.052915 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935f2ab-1d14-4dee-af47-e8359e4603fb None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:55.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:55.154047 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935f2ab-1d14-4dee-af47-e8359e4603fb None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:55.188484 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895916f-68b2-4b3d-8dd8-5157f4bd3794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:55.189194 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895916f-68b2-4b3d-8dd8-5157f4bd3794 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:55.189474 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895916f-68b2-4b3d-8dd8-5157f4bd3794 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:55.189746 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895916f-68b2-4b3d-8dd8-5157f4bd3794 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:55.199780 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895916f-68b2-4b3d-8dd8-5157f4bd3794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-c895916f-68b2-4b3d-8dd8-5157f4bd3794 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:48:55.207378 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:48:55.208534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:48:55.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 598/1195] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:48:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:48:55.214836 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9f2ed-ce05-412c-a2af-426bdab9f4db None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:55.313789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9f2ed-ce05-412c-a2af-426bdab9f4db None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:55.343954 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:55.348267 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172f634-b6ef-454a-a6b0-67d50adae25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:55.352416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172f634-b6ef-454a-a6b0-67d50adae25d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:55.352416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172f634-b6ef-454a-a6b0-67d50adae25d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:55.352416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172f634-b6ef-454a-a6b0-67d50adae25d None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:55.352416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172f634-b6ef-454a-a6b0-67d50adae25d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:55.352416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172f634-b6ef-454a-a6b0-67d50adae25d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:55.352416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172f634-b6ef-454a-a6b0-67d50adae25d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:55.359840 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 598/1196] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:48:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:55.365559 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895916f-68b2-4b3d-8dd8-5157f4bd3794 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:55.444519 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:55.466016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895916f-68b2-4b3d-8dd8-5157f4bd3794 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:55.498713 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4d73a-c713-4386-b480-12600adb7165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:55.499572 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4d73a-c713-4386-b480-12600adb7165 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:55.499855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4d73a-c713-4386-b480-12600adb7165 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:55.500126 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4d73a-c713-4386-b480-12600adb7165 None admin] PATH_INFO: `/v3/users/615348dde6874afca87dcbe16ec15f2d` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:55.501638 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4d73a-c713-4386-b480-12600adb7165 None admin] RBAC: Authorizing `identity:get_user(user_id=615348dde6874afca87dcbe16ec15f2d)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:55.503337 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4d73a-c713-4386-b480-12600adb7165 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:55.504993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 599/1197] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 19:48:55 2022] GET /identity/v3/users/615348dde6874afca87dcbe16ec15f2d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:55.511190 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172f634-b6ef-454a-a6b0-67d50adae25d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:55.609585 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172f634-b6ef-454a-a6b0-67d50adae25d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:55.642751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476902b-b78f-49b6-919b-793436a25556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:55.643381 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476902b-b78f-49b6-919b-793436a25556 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:55.643659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476902b-b78f-49b6-919b-793436a25556 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:55.643931 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476902b-b78f-49b6-919b-793436a25556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:55.702207 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:55.709307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476902b-b78f-49b6-919b-793436a25556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:55.710783 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476902b-b78f-49b6-919b-793436a25556 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:55.805881 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 599/1198] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:48:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:48:55.812326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4d73a-c713-4386-b480-12600adb7165 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:55.909940 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4d73a-c713-4386-b480-12600adb7165 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:55.943375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc3a64-a4da-4398-baf3-14e0bd3041c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:55.944260 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc3a64-a4da-4398-baf3-14e0bd3041c4 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:55.944562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc3a64-a4da-4398-baf3-14e0bd3041c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:55.944833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc3a64-a4da-4398-baf3-14e0bd3041c4 None admin] PATH_INFO: `/v3/projects/de32aa5e2bc84271be2c53d4bf0b8d7c` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:55.945988 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc3a64-a4da-4398-baf3-14e0bd3041c4 None admin] RBAC: Authorizing `identity:get_project(project_id=de32aa5e2bc84271be2c53d4bf0b8d7c)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:55.947459 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc3a64-a4da-4398-baf3-14e0bd3041c4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:55.949159 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 600/1199] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 19:48:55 2022] GET /identity/v3/projects/de32aa5e2bc84271be2c53d4bf0b8d7c => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:48:55.955273 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476902b-b78f-49b6-919b-793436a25556 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:48:56.050153 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:48:56.057412 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476902b-b78f-49b6-919b-793436a25556 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:48:56.089081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61092c-b68f-4fcc-a757-e76c77e50151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:48:56.089869 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61092c-b68f-4fcc-a757-e76c77e50151 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:48:56.090479 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61092c-b68f-4fcc-a757-e76c77e50151 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:48:56.090750 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61092c-b68f-4fcc-a757-e76c77e50151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:48:56.091571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61092c-b68f-4fcc-a757-e76c77e50151 None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=de32aa5e2bc84271be2c53d4bf0b8d7c, user.id=615348dde6874afca87dcbe16ec15f2d) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:48:56.091963 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61092c-b68f-4fcc-a757-e76c77e50151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:48:56.093439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61092c-b68f-4fcc-a757-e76c77e50151 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:48:56.099176 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 600/1200] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:48:55 2022] GET /identity/v3/role_assignments?user.id=615348dde6874afca87dcbe16ec15f2d&scope.project.id=de32aa5e2bc84271be2c53d4bf0b8d7c&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:49:05.939285 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=93 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:49:06.096588 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=21 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:15.364454 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704175b-31bf-4982-b93d-dbfb6a2afcd0 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:15.365006 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704175b-31bf-4982-b93d-dbfb6a2afcd0 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:15.365006 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704175b-31bf-4982-b93d-dbfb6a2afcd0 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:15.366428 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 601/1201] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:15.371676 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfe52b-63d8-4d32-8644-055e4abff96d None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:15.372664 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfe52b-63d8-4d32-8644-055e4abff96d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:15.372917 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfe52b-63d8-4d32-8644-055e4abff96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:15.389900 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-e8bfe52b-63d8-4d32-8644-055e4abff96d None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:15.391323 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-e8bfe52b-63d8-4d32-8644-055e4abff96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:15.458574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 601/1202] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:50:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:50:15.483387 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704175b-31bf-4982-b93d-dbfb6a2afcd0 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:15.585565 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704175b-31bf-4982-b93d-dbfb6a2afcd0 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:15.619206 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070127a-4f60-43c2-ae80-84c8c53cf61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:15.619921 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070127a-4f60-43c2-ae80-84c8c53cf61c None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:15.620201 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070127a-4f60-43c2-ae80-84c8c53cf61c None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:15.620503 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070127a-4f60-43c2-ae80-84c8c53cf61c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:15.630390 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070127a-4f60-43c2-ae80-84c8c53cf61c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:15.631879 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070127a-4f60-43c2-ae80-84c8c53cf61c None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-2070127a-4f60-43c2-ae80-84c8c53cf61c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:15.650858 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:15.652128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:15.654039 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:15.654039 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:15.654039 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:15.654039 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:15.654039 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 602/1203] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:15.659086 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfe52b-63d8-4d32-8644-055e4abff96d None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:15.755036 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfe52b-63d8-4d32-8644-055e4abff96d None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:15.782889 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:15.788571 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a51a23-fc56-467b-bcf7-553ac0ddef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:15.789229 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a51a23-fc56-467b-bcf7-553ac0ddef7d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:15.789510 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a51a23-fc56-467b-bcf7-553ac0ddef7d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:15.789789 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a51a23-fc56-467b-bcf7-553ac0ddef7d None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:15.790508 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a51a23-fc56-467b-bcf7-553ac0ddef7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:15.790809 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a51a23-fc56-467b-bcf7-553ac0ddef7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:15.792167 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a51a23-fc56-467b-bcf7-553ac0ddef7d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:15.804130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 602/1204] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:15.809855 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070127a-4f60-43c2-ae80-84c8c53cf61c None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:15.862847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:15.912672 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070127a-4f60-43c2-ae80-84c8c53cf61c None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:15.947591 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57072-6c51-448b-8521-b74862e0d02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:15.948262 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57072-6c51-448b-8521-b74862e0d02a None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:15.948585 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57072-6c51-448b-8521-b74862e0d02a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:15.948860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57072-6c51-448b-8521-b74862e0d02a None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:15.949563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57072-6c51-448b-8521-b74862e0d02a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:15.951061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57072-6c51-448b-8521-b74862e0d02a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:16.135491 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 603/1205] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 19:50:15 2022] POST /identity/v3/users => generated 251 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:18.387504 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71bb4d-3789-4f91-826e-a2f44ba65a4a None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:18.388007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71bb4d-3789-4f91-826e-a2f44ba65a4a None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:18.388099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71bb4d-3789-4f91-826e-a2f44ba65a4a None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:18.389449 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 603/1206] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:18.394734 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37881e04-e422-450f-a7c0-8edd1493c9f2 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:18.395160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37881e04-e422-450f-a7c0-8edd1493c9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:18.395300 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37881e04-e422-450f-a7c0-8edd1493c9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:18.411816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-37881e04-e422-450f-a7c0-8edd1493c9f2 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:18.413023 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-37881e04-e422-450f-a7c0-8edd1493c9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:18.478994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 604/1207] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:50:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:50:18.502857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71bb4d-3789-4f91-826e-a2f44ba65a4a None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:18.604034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71bb4d-3789-4f91-826e-a2f44ba65a4a None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:18.638627 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76102bc-69f5-4da3-9267-5c11dc5e2d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:18.639416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76102bc-69f5-4da3-9267-5c11dc5e2d47 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:18.639694 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76102bc-69f5-4da3-9267-5c11dc5e2d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:18.640012 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76102bc-69f5-4da3-9267-5c11dc5e2d47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:18.649829 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76102bc-69f5-4da3-9267-5c11dc5e2d47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-b76102bc-69f5-4da3-9267-5c11dc5e2d47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:18.657638 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:18.658761 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:18.659805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 604/1208] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:50:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:18.665236 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37881e04-e422-450f-a7c0-8edd1493c9f2 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:18.704324 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:18.760945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37881e04-e422-450f-a7c0-8edd1493c9f2 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:18.793593 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c93315-614e-4eac-a7be-4f4b72cbfcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:18.794311 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c93315-614e-4eac-a7be-4f4b72cbfcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:18.794831 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c93315-614e-4eac-a7be-4f4b72cbfcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:18.795233 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c93315-614e-4eac-a7be-4f4b72cbfcb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:18.796030 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c93315-614e-4eac-a7be-4f4b72cbfcb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:18.796759 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c93315-614e-4eac-a7be-4f4b72cbfcb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:18.798145 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c93315-614e-4eac-a7be-4f4b72cbfcb8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:18.806249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 605/1209] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:50:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:18.812792 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76102bc-69f5-4da3-9267-5c11dc5e2d47 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:18.862028 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:18.910637 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76102bc-69f5-4da3-9267-5c11dc5e2d47 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:18.943611 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53767841-85aa-473a-8f15-eec9e2dfe25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:18.957047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53767841-85aa-473a-8f15-eec9e2dfe25b None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:18.957047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53767841-85aa-473a-8f15-eec9e2dfe25b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:18.957047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53767841-85aa-473a-8f15-eec9e2dfe25b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:18.957047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53767841-85aa-473a-8f15-eec9e2dfe25b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:18.957421 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53767841-85aa-473a-8f15-eec9e2dfe25b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-53767841-85aa-473a-8f15-eec9e2dfe25b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:18.973930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:18.975127 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:18.976231 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:18.976231 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:18.976231 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:18.976231 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:18.976231 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 605/1210] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:18.980908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c93315-614e-4eac-a7be-4f4b72cbfcb8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:19.079586 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:19.082624 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c93315-614e-4eac-a7be-4f4b72cbfcb8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:19.117089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4e7b4-6fd5-4aaa-b560-0bf877c21153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:19.117089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4e7b4-6fd5-4aaa-b560-0bf877c21153 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.117089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4e7b4-6fd5-4aaa-b560-0bf877c21153 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.117425 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4e7b4-6fd5-4aaa-b560-0bf877c21153 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.118061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4e7b4-6fd5-4aaa-b560-0bf877c21153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:19.118364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4e7b4-6fd5-4aaa-b560-0bf877c21153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:19.119870 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4e7b4-6fd5-4aaa-b560-0bf877c21153 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:19.131821 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 606/1211] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:19.137759 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53767841-85aa-473a-8f15-eec9e2dfe25b None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:19.237140 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53767841-85aa-473a-8f15-eec9e2dfe25b None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:19.238659 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:19.269978 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea40e1c-5990-48ca-b497-62848bc462d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:19.270768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea40e1c-5990-48ca-b497-62848bc462d1 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.271050 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea40e1c-5990-48ca-b497-62848bc462d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.271322 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea40e1c-5990-48ca-b497-62848bc462d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.278589 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea40e1c-5990-48ca-b497-62848bc462d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:19.280049 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea40e1c-5990-48ca-b497-62848bc462d1 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-4ea40e1c-5990-48ca-b497-62848bc462d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:19.296482 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:19.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:19.298810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:19.298810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:19.298810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:19.298810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:19.298810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:19.298810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:19.298810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:19.298810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:19.298810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:19.298810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:19.298810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 606/1212] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:19.303918 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4e7b4-6fd5-4aaa-b560-0bf877c21153 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:19.407656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4e7b4-6fd5-4aaa-b560-0bf877c21153 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:19.441259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aaa60-cb7b-423f-ae21-0d9764de761e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:19.441926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aaa60-cb7b-423f-ae21-0d9764de761e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.442215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aaa60-cb7b-423f-ae21-0d9764de761e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.442493 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8aaa60-cb7b-423f-ae21-0d9764de761e None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.443327 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8aaa60-cb7b-423f-ae21-0d9764de761e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:19.443640 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8aaa60-cb7b-423f-ae21-0d9764de761e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:19.445101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8aaa60-cb7b-423f-ae21-0d9764de761e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:19.462806 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 607/1213] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 19:50:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:19.469347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea40e1c-5990-48ca-b497-62848bc462d1 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:19.572901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea40e1c-5990-48ca-b497-62848bc462d1 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:19.609382 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d94924-7dba-4197-827e-3dc88ee2294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:19.610028 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d94924-7dba-4197-827e-3dc88ee2294d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.610308 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d94924-7dba-4197-827e-3dc88ee2294d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.610581 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d94924-7dba-4197-827e-3dc88ee2294d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.640175 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:19.678100 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d94924-7dba-4197-827e-3dc88ee2294d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:19.679717 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d94924-7dba-4197-827e-3dc88ee2294d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:19.784984 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 607/1214] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:50:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:50:19.790716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aaa60-cb7b-423f-ae21-0d9764de761e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:19.809552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:19.886572 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8aaa60-cb7b-423f-ae21-0d9764de761e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:19.919751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f831de-8929-49e9-84c0-07bb239067f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:19.920554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f831de-8929-49e9-84c0-07bb239067f6 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:19.920949 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f831de-8929-49e9-84c0-07bb239067f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:19.921219 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f831de-8929-49e9-84c0-07bb239067f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:19.931480 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f831de-8929-49e9-84c0-07bb239067f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:19.932926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f831de-8929-49e9-84c0-07bb239067f6 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-46f831de-8929-49e9-84c0-07bb239067f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:19.950666 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:19.961815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:19.962959 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:19.962959 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:19.962959 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:19.962959 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:19.962959 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 608/1215] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:19.963318 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d94924-7dba-4197-827e-3dc88ee2294d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:20.017152 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:20.054044 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d94924-7dba-4197-827e-3dc88ee2294d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:20.086703 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c672d6-0938-4780-a7d9-156c7ddc9556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:20.091677 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c672d6-0938-4780-a7d9-156c7ddc9556 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:20.091677 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c672d6-0938-4780-a7d9-156c7ddc9556 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:20.091677 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c672d6-0938-4780-a7d9-156c7ddc9556 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:20.091677 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c672d6-0938-4780-a7d9-156c7ddc9556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:20.091677 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c672d6-0938-4780-a7d9-156c7ddc9556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:20.091677 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c672d6-0938-4780-a7d9-156c7ddc9556 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:20.101498 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 608/1216] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:20.107204 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f831de-8929-49e9-84c0-07bb239067f6 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:20.175433 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:20.209525 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f831de-8929-49e9-84c0-07bb239067f6 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:20.243985 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51021c21-d258-4114-be8c-8212ec84a07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:20.244797 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51021c21-d258-4114-be8c-8212ec84a07e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:20.245080 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51021c21-d258-4114-be8c-8212ec84a07e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:20.245354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51021c21-d258-4114-be8c-8212ec84a07e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:20.252608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51021c21-d258-4114-be8c-8212ec84a07e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:20.254054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51021c21-d258-4114-be8c-8212ec84a07e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-51021c21-d258-4114-be8c-8212ec84a07e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:20.268900 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:20.274917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:20.276060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:20.276060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:20.276060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:20.276060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:20.276060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:20.276060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:20.276060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:20.276060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:20.276060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 609/1217] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:20.276715 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c672d6-0938-4780-a7d9-156c7ddc9556 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:20.373861 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c672d6-0938-4780-a7d9-156c7ddc9556 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:20.396390 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:20.407506 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af7851-bf8a-42f8-a0bc-db7d1909a19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:20.408158 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7851-bf8a-42f8-a0bc-db7d1909a19f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:20.408474 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7851-bf8a-42f8-a0bc-db7d1909a19f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:20.408751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7851-bf8a-42f8-a0bc-db7d1909a19f None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:20.409534 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af7851-bf8a-42f8-a0bc-db7d1909a19f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:20.409869 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af7851-bf8a-42f8-a0bc-db7d1909a19f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:20.411284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af7851-bf8a-42f8-a0bc-db7d1909a19f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:20.423126 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 609/1218] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:50:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:20.429691 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51021c21-d258-4114-be8c-8212ec84a07e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:20.529198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51021c21-d258-4114-be8c-8212ec84a07e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:20.557121 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:20.563334 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802e7c5-750d-4e56-af21-a8eaf19e67bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:20.563975 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802e7c5-750d-4e56-af21-a8eaf19e67bc None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:20.564259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802e7c5-750d-4e56-af21-a8eaf19e67bc None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:20.564562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802e7c5-750d-4e56-af21-a8eaf19e67bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:20.565392 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802e7c5-750d-4e56-af21-a8eaf19e67bc None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=28997ce3cd2e41feaf22b9742ba5e510) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:20.565702 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802e7c5-750d-4e56-af21-a8eaf19e67bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:20.567134 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802e7c5-750d-4e56-af21-a8eaf19e67bc None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:20.572461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 610/1219] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:50:20 2022] GET /identity/v3/role_assignments?user.id=28997ce3cd2e41feaf22b9742ba5e510&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:22.768535 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afbdf6-a600-4fc0-9c09-585695605cc0 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:22.769046 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afbdf6-a600-4fc0-9c09-585695605cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:22.769046 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afbdf6-a600-4fc0-9c09-585695605cc0 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:22.770377 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 610/1220] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:22.775712 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cb1c4-9575-4b2f-9aa2-b5d324d0630c None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:22.776134 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cb1c4-9575-4b2f-9aa2-b5d324d0630c None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:22.776269 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cb1c4-9575-4b2f-9aa2-b5d324d0630c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:22.793399 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-bd8cb1c4-9575-4b2f-9aa2-b5d324d0630c None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:22.794599 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-bd8cb1c4-9575-4b2f-9aa2-b5d324d0630c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:22.860355 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 611/1221] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:50:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:50:22.883148 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afbdf6-a600-4fc0-9c09-585695605cc0 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:22.984576 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afbdf6-a600-4fc0-9c09-585695605cc0 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:23.018791 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc55bf-0dfe-4e15-99c0-9e0e1344b05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:23.019506 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc55bf-0dfe-4e15-99c0-9e0e1344b05a None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.019780 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc55bf-0dfe-4e15-99c0-9e0e1344b05a None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.020051 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc55bf-0dfe-4e15-99c0-9e0e1344b05a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.030700 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc55bf-0dfe-4e15-99c0-9e0e1344b05a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-94dc55bf-0dfe-4e15-99c0-9e0e1344b05a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:23.038458 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:23.039558 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:23.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 611/1222] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:50:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:23.045749 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cb1c4-9575-4b2f-9aa2-b5d324d0630c None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:23.145677 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cb1c4-9575-4b2f-9aa2-b5d324d0630c None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:23.168718 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:23.179351 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf01dc-e91f-4c6b-82df-1eaefb23210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:23.180016 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf01dc-e91f-4c6b-82df-1eaefb23210d None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.180298 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf01dc-e91f-4c6b-82df-1eaefb23210d None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.180606 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf01dc-e91f-4c6b-82df-1eaefb23210d None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.181350 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf01dc-e91f-4c6b-82df-1eaefb23210d None admin] RBAC: Adding query filter params (name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:23.181655 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf01dc-e91f-4c6b-82df-1eaefb23210d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:23.183039 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf01dc-e91f-4c6b-82df-1eaefb23210d None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:23.191518 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 612/1223] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:50:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:23.197333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc55bf-0dfe-4e15-99c0-9e0e1344b05a None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:23.241380 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:23.296046 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc55bf-0dfe-4e15-99c0-9e0e1344b05a None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:23.329874 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a34eb-444c-4063-a286-3c1cf6c3ff1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:23.330656 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a34eb-444c-4063-a286-3c1cf6c3ff1d None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.330932 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a34eb-444c-4063-a286-3c1cf6c3ff1d None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.331202 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a34eb-444c-4063-a286-3c1cf6c3ff1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.343239 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a34eb-444c-4063-a286-3c1cf6c3ff1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:23.344711 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a34eb-444c-4063-a286-3c1cf6c3ff1d None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-704a34eb-444c-4063-a286-3c1cf6c3ff1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:23.361987 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:23.363760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:23.365503 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:23.365503 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:23.365503 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:23.365503 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:23.365503 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 612/1224] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:23.369138 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf01dc-e91f-4c6b-82df-1eaefb23210d None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:23.470069 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf01dc-e91f-4c6b-82df-1eaefb23210d None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:23.503264 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7457780-d24c-484d-9b30-ca865b3998fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:23.503912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7457780-d24c-484d-9b30-ca865b3998fe None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.504192 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7457780-d24c-484d-9b30-ca865b3998fe None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.504494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7457780-d24c-484d-9b30-ca865b3998fe None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.505258 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7457780-d24c-484d-9b30-ca865b3998fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:23.505554 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7457780-d24c-484d-9b30-ca865b3998fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:23.506936 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7457780-d24c-484d-9b30-ca865b3998fe None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:23.518661 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 613/1225] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:23.524457 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a34eb-444c-4063-a286-3c1cf6c3ff1d None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:23.617485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:23.623900 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a34eb-444c-4063-a286-3c1cf6c3ff1d None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:23.658281 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de264ff5-d6f9-410f-8cdf-097caa738cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:23.659155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de264ff5-d6f9-410f-8cdf-097caa738cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.659525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de264ff5-d6f9-410f-8cdf-097caa738cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.659801 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de264ff5-d6f9-410f-8cdf-097caa738cb9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.667085 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de264ff5-d6f9-410f-8cdf-097caa738cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:23.668917 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de264ff5-d6f9-410f-8cdf-097caa738cb9 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-de264ff5-d6f9-410f-8cdf-097caa738cb9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:23.683851 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:23.685440 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:23.686586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:23.686586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:23.686586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:23.686586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:23.686586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:23.686586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:23.686586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:23.686586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:23.686586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:23.686586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:23.686586 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 613/1226] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:23.693465 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7457780-d24c-484d-9b30-ca865b3998fe None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:23.739753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:23.790638 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7457780-d24c-484d-9b30-ca865b3998fe None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:23.823793 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd6918-acc0-46ae-a0f3-65710af6cfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:23.824502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6918-acc0-46ae-a0f3-65710af6cfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.825400 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6918-acc0-46ae-a0f3-65710af6cfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.825694 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6918-acc0-46ae-a0f3-65710af6cfc5 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.826736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd6918-acc0-46ae-a0f3-65710af6cfc5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:23.827211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd6918-acc0-46ae-a0f3-65710af6cfc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:23.829240 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd6918-acc0-46ae-a0f3-65710af6cfc5 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:23.845999 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 614/1227] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 19:50:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:23.851691 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de264ff5-d6f9-410f-8cdf-097caa738cb9 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:23.948923 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de264ff5-d6f9-410f-8cdf-097caa738cb9 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:23.982654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3217fa0-6a3b-45f4-816e-2928a46b63d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:23.983438 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3217fa0-6a3b-45f4-816e-2928a46b63d2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:23.983723 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3217fa0-6a3b-45f4-816e-2928a46b63d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:23.984075 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3217fa0-6a3b-45f4-816e-2928a46b63d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:23.993840 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3217fa0-6a3b-45f4-816e-2928a46b63d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:23.995331 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3217fa0-6a3b-45f4-816e-2928a46b63d2 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-e3217fa0-6a3b-45f4-816e-2928a46b63d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:24.013768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:24.014827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:24.015943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:24.015943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:24.015943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:24.015943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:24.015943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 614/1228] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:24.020665 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd6918-acc0-46ae-a0f3-65710af6cfc5 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:24.118815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd6918-acc0-46ae-a0f3-65710af6cfc5 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:24.121189 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:24.152769 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ad4de-96fa-4bf5-8bb1-4798bd8a1c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:24.153745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ad4de-96fa-4bf5-8bb1-4798bd8a1c46 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.154024 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ad4de-96fa-4bf5-8bb1-4798bd8a1c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.154297 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ad4de-96fa-4bf5-8bb1-4798bd8a1c46 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.155012 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ad4de-96fa-4bf5-8bb1-4798bd8a1c46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:24.155320 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ad4de-96fa-4bf5-8bb1-4798bd8a1c46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:24.156751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ad4de-96fa-4bf5-8bb1-4798bd8a1c46 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:24.167986 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 615/1229] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:24.174209 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3217fa0-6a3b-45f4-816e-2928a46b63d2 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:24.183484 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:24.273776 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3217fa0-6a3b-45f4-816e-2928a46b63d2 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:24.307750 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5b1be-7b68-4f9a-be72-71537b7f784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:24.308564 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5b1be-7b68-4f9a-be72-71537b7f784f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.308852 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5b1be-7b68-4f9a-be72-71537b7f784f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.315784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5b1be-7b68-4f9a-be72-71537b7f784f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.316258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5b1be-7b68-4f9a-be72-71537b7f784f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:24.317787 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e5b1be-7b68-4f9a-be72-71537b7f784f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-99e5b1be-7b68-4f9a-be72-71537b7f784f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:24.331490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:24.332604 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:24.333731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:24.333731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:24.333731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:24.333731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:24.333731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:24.333731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:24.333731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:24.333731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:24.333731 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 615/1230] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:50:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:24.339093 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ad4de-96fa-4bf5-8bb1-4798bd8a1c46 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:24.440571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ad4de-96fa-4bf5-8bb1-4798bd8a1c46 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:24.473081 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb4b88-eb46-4027-a9ec-acd9b70248e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:24.473732 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb4b88-eb46-4027-a9ec-acd9b70248e3 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.474029 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb4b88-eb46-4027-a9ec-acd9b70248e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.474296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb4b88-eb46-4027-a9ec-acd9b70248e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.475082 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb4b88-eb46-4027-a9ec-acd9b70248e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:24.475390 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb4b88-eb46-4027-a9ec-acd9b70248e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:24.476870 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb4b88-eb46-4027-a9ec-acd9b70248e3 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:24.489145 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 616/1231] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:50:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:24.496092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5b1be-7b68-4f9a-be72-71537b7f784f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:24.576557 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:24.610569 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e5b1be-7b68-4f9a-be72-71537b7f784f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:24.642661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fca63-a3cd-42c3-bcfe-d8c8757b055b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:24.643540 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fca63-a3cd-42c3-bcfe-d8c8757b055b None admin] REQUEST_METHOD: `PUT` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:24.643823 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fca63-a3cd-42c3-bcfe-d8c8757b055b None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:24.644098 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fca63-a3cd-42c3-bcfe-d8c8757b055b None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/28997ce3cd2e41feaf22b9742ba5e510/roles/a615c7d83a0c46b1b26642bcc6e6ecef` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:24.658821 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fca63-a3cd-42c3-bcfe-d8c8757b055b None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=28997ce3cd2e41feaf22b9742ba5e510, role_id=a615c7d83a0c46b1b26642bcc6e6ecef)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:24.660270 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fca63-a3cd-42c3-bcfe-d8c8757b055b None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:24.739740 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 616/1232] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:50:24 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/28997ce3cd2e41feaf22b9742ba5e510/roles/a615c7d83a0c46b1b26642bcc6e6ecef => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 19:50:26.924418 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4642-5f13-46f1-9eb5-6b0df9d81b11 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:26.924993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4642-5f13-46f1-9eb5-6b0df9d81b11 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:26.924993 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4642-5f13-46f1-9eb5-6b0df9d81b11 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:26.926377 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 617/1233] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:26.931887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ed8bf-f3d4-4920-992b-ce31c6bdeb9b None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:26.932284 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ed8bf-f3d4-4920-992b-ce31c6bdeb9b None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:26.932479 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ed8bf-f3d4-4920-992b-ce31c6bdeb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:26.950679 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-666ed8bf-f3d4-4920-992b-ce31c6bdeb9b None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:26.951935 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-666ed8bf-f3d4-4920-992b-ce31c6bdeb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:27.018099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 617/1234] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:50:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:50:27.040904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4642-5f13-46f1-9eb5-6b0df9d81b11 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.066856 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:27.141645 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4642-5f13-46f1-9eb5-6b0df9d81b11 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.178385 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf2e12-3d08-4f16-9413-996e98a57cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.178385 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf2e12-3d08-4f16-9413-996e98a57cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.178385 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf2e12-3d08-4f16-9413-996e98a57cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.178385 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf2e12-3d08-4f16-9413-996e98a57cb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.187803 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf2e12-3d08-4f16-9413-996e98a57cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-ebdf2e12-3d08-4f16-9413-996e98a57cb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:27.195477 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:27.196617 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:27.197847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 618/1235] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:27.203231 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ed8bf-f3d4-4920-992b-ce31c6bdeb9b None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.249400 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:27.302771 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ed8bf-f3d4-4920-992b-ce31c6bdeb9b None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.337470 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b37df-49f7-4c29-a5bb-7263006fb5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.338139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b37df-49f7-4c29-a5bb-7263006fb5ec None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.338426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b37df-49f7-4c29-a5bb-7263006fb5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.338703 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b37df-49f7-4c29-a5bb-7263006fb5ec None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.339520 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b37df-49f7-4c29-a5bb-7263006fb5ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:27.339827 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b37df-49f7-4c29-a5bb-7263006fb5ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.341810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b37df-49f7-4c29-a5bb-7263006fb5ec None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.350648 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 618/1236] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:27.358142 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf2e12-3d08-4f16-9413-996e98a57cb0 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.473983 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:27.473983 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf2e12-3d08-4f16-9413-996e98a57cb0 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.490838 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18522c46-e229-4838-8e53-92766f928a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.491661 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522c46-e229-4838-8e53-92766f928a80 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.491941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522c46-e229-4838-8e53-92766f928a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.492216 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522c46-e229-4838-8e53-92766f928a80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.502239 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18522c46-e229-4838-8e53-92766f928a80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.503675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18522c46-e229-4838-8e53-92766f928a80 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-18522c46-e229-4838-8e53-92766f928a80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:27.521101 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:27.522220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:27.523259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:27.523259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:27.523259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:27.523259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:27.523259 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 619/1237] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:27.528759 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b37df-49f7-4c29-a5bb-7263006fb5ec None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.626688 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b37df-49f7-4c29-a5bb-7263006fb5ec None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.635494 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:27.660099 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c0de1-b258-4d4b-a1d6-3427bf83aa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.660799 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c0de1-b258-4d4b-a1d6-3427bf83aa96 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.661085 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c0de1-b258-4d4b-a1d6-3427bf83aa96 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.661358 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c0de1-b258-4d4b-a1d6-3427bf83aa96 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.662103 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c0de1-b258-4d4b-a1d6-3427bf83aa96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:27.662378 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c0de1-b258-4d4b-a1d6-3427bf83aa96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.663737 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c0de1-b258-4d4b-a1d6-3427bf83aa96 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.675620 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 619/1238] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:27.681546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18522c46-e229-4838-8e53-92766f928a80 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.779148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18522c46-e229-4838-8e53-92766f928a80 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.813069 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87ceaf-bb91-47bf-adaa-f73818bfe664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.813860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87ceaf-bb91-47bf-adaa-f73818bfe664 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.814140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87ceaf-bb91-47bf-adaa-f73818bfe664 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.814411 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87ceaf-bb91-47bf-adaa-f73818bfe664 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.817157 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:27.821636 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87ceaf-bb91-47bf-adaa-f73818bfe664 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.823078 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87ceaf-bb91-47bf-adaa-f73818bfe664 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-ea87ceaf-bb91-47bf-adaa-f73818bfe664 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:27.838263 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:27.839542 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:27.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:27.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:27.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:27.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:27.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:27.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:27.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:27.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:27.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:27.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:27.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 620/1239] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:27.845889 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c0de1-b258-4d4b-a1d6-3427bf83aa96 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:27.945430 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c0de1-b258-4d4b-a1d6-3427bf83aa96 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:27.979958 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abf126-7ee9-437a-b4fc-d16dc0ad890f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:27.980646 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abf126-7ee9-437a-b4fc-d16dc0ad890f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:27.981007 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abf126-7ee9-437a-b4fc-d16dc0ad890f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:27.981285 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abf126-7ee9-437a-b4fc-d16dc0ad890f None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:27.982231 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abf126-7ee9-437a-b4fc-d16dc0ad890f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:27.982543 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abf126-7ee9-437a-b4fc-d16dc0ad890f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:27.983946 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abf126-7ee9-437a-b4fc-d16dc0ad890f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:28.000792 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 620/1240] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 19:50:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:28.006541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87ceaf-bb91-47bf-adaa-f73818bfe664 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:28.104575 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87ceaf-bb91-47bf-adaa-f73818bfe664 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:28.137852 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282fc26-b189-4b84-9cdd-07db26aa9338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:28.138492 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282fc26-b189-4b84-9cdd-07db26aa9338 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:28.143220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282fc26-b189-4b84-9cdd-07db26aa9338 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:28.143220 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282fc26-b189-4b84-9cdd-07db26aa9338 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:28.197802 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:28.204653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282fc26-b189-4b84-9cdd-07db26aa9338 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:28.206136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2282fc26-b189-4b84-9cdd-07db26aa9338 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:28.305390 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 621/1241] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:50:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:50:28.311795 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abf126-7ee9-437a-b4fc-d16dc0ad890f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:28.340059 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:28.411027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abf126-7ee9-437a-b4fc-d16dc0ad890f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:28.446269 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50278d-fcee-4ca6-9038-27f189836e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:28.447057 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50278d-fcee-4ca6-9038-27f189836e87 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:28.447340 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50278d-fcee-4ca6-9038-27f189836e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:28.447649 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50278d-fcee-4ca6-9038-27f189836e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:28.458011 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50278d-fcee-4ca6-9038-27f189836e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:28.459446 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c50278d-fcee-4ca6-9038-27f189836e87 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-0c50278d-fcee-4ca6-9038-27f189836e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:28.477904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:28.479087 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:28.480531 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:28.480531 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:28.480531 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:28.480531 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:28.480531 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 621/1242] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:28.484939 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282fc26-b189-4b84-9cdd-07db26aa9338 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:28.578743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:28.584258 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282fc26-b189-4b84-9cdd-07db26aa9338 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:28.616563 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57205aee-1600-473a-94f2-9201588b10d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:28.617212 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57205aee-1600-473a-94f2-9201588b10d4 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:28.617496 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57205aee-1600-473a-94f2-9201588b10d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:28.617772 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57205aee-1600-473a-94f2-9201588b10d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:28.618482 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57205aee-1600-473a-94f2-9201588b10d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:28.618785 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57205aee-1600-473a-94f2-9201588b10d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:28.620151 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57205aee-1600-473a-94f2-9201588b10d4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:28.631435 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 622/1243] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:28.637030 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50278d-fcee-4ca6-9038-27f189836e87 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:28.714507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:28.735201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50278d-fcee-4ca6-9038-27f189836e87 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:28.767307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de36a1-dd08-4dad-a072-0760925c3bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:28.769784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de36a1-dd08-4dad-a072-0760925c3bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:28.769784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de36a1-dd08-4dad-a072-0760925c3bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:28.769784 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de36a1-dd08-4dad-a072-0760925c3bb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:28.774927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de36a1-dd08-4dad-a072-0760925c3bb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:28.776360 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de36a1-dd08-4dad-a072-0760925c3bb6 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-c7de36a1-dd08-4dad-a072-0760925c3bb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:28.789967 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:28.791033 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:28.792143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:28.792143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:28.792143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:28.792143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:28.792143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:28.792143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:28.792143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:28.792143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:28.792143 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 622/1244] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:50:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:28.797570 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57205aee-1600-473a-94f2-9201588b10d4 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:28.897094 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57205aee-1600-473a-94f2-9201588b10d4 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:28.930804 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f883e8-c38b-4fed-a4da-3426bcbb95af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:28.931458 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f883e8-c38b-4fed-a4da-3426bcbb95af None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:28.931744 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f883e8-c38b-4fed-a4da-3426bcbb95af None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:28.932021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f883e8-c38b-4fed-a4da-3426bcbb95af None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:28.932841 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f883e8-c38b-4fed-a4da-3426bcbb95af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:28.933151 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f883e8-c38b-4fed-a4da-3426bcbb95af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:28.935210 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f883e8-c38b-4fed-a4da-3426bcbb95af None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:28.947121 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 623/1245] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:50:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:28.953248 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de36a1-dd08-4dad-a072-0760925c3bb6 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:29.050791 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de36a1-dd08-4dad-a072-0760925c3bb6 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:29.083324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47519069-0c7a-43c9-842a-63c60b38ae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:29.083966 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47519069-0c7a-43c9-842a-63c60b38ae63 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:29.084251 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47519069-0c7a-43c9-842a-63c60b38ae63 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:29.084550 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47519069-0c7a-43c9-842a-63c60b38ae63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:29.085380 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47519069-0c7a-43c9-842a-63c60b38ae63 None admin] RBAC: Adding query filter params (role.id=a615c7d83a0c46b1b26642bcc6e6ecef, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=28997ce3cd2e41feaf22b9742ba5e510) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:29.085768 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47519069-0c7a-43c9-842a-63c60b38ae63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:29.087174 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47519069-0c7a-43c9-842a-63c60b38ae63 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:29.093380 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 623/1246] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:50:28 2022] GET /identity/v3/role_assignments?user.id=28997ce3cd2e41feaf22b9742ba5e510&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=a615c7d83a0c46b1b26642bcc6e6ecef => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:31.377970 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1b271-4a66-4a63-8bb4-c7ef06e84e9c None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.378490 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1b271-4a66-4a63-8bb4-c7ef06e84e9c None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.378582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1b271-4a66-4a63-8bb4-c7ef06e84e9c None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.379948 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 624/1247] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:31.385079 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d83dc5-f595-49e3-890d-bf348ef1f5f5 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.385506 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d83dc5-f595-49e3-890d-bf348ef1f5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.385639 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d83dc5-f595-49e3-890d-bf348ef1f5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.390630 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:31.402411 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-c8d83dc5-f595-49e3-890d-bf348ef1f5f5 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:31.404090 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-c8d83dc5-f595-49e3-890d-bf348ef1f5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:31.468281 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 624/1248] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:50:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:50:31.491840 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1b271-4a66-4a63-8bb4-c7ef06e84e9c None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.510274 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:31.592224 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1b271-4a66-4a63-8bb4-c7ef06e84e9c None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.626873 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7ecdf-5e3f-4847-97bc-df76d4bcd529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.627602 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7ecdf-5e3f-4847-97bc-df76d4bcd529 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.627886 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7ecdf-5e3f-4847-97bc-df76d4bcd529 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.628160 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7ecdf-5e3f-4847-97bc-df76d4bcd529 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.640305 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7ecdf-5e3f-4847-97bc-df76d4bcd529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-52c7ecdf-5e3f-4847-97bc-df76d4bcd529 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:31.648187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:31.649363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:31.650589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 625/1249] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:31.655507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d83dc5-f595-49e3-890d-bf348ef1f5f5 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.755582 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d83dc5-f595-49e3-890d-bf348ef1f5f5 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.784142 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:31.788919 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abd139-33dc-4335-9a08-027bf23046f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.789588 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abd139-33dc-4335-9a08-027bf23046f2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.789874 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abd139-33dc-4335-9a08-027bf23046f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.790152 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abd139-33dc-4335-9a08-027bf23046f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.790970 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06abd139-33dc-4335-9a08-027bf23046f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:31.791282 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06abd139-33dc-4335-9a08-027bf23046f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:31.792660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06abd139-33dc-4335-9a08-027bf23046f2 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:31.801108 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 625/1250] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:31.807248 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7ecdf-5e3f-4847-97bc-df76d4bcd529 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:31.885777 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:31.907538 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7ecdf-5e3f-4847-97bc-df76d4bcd529 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:31.940817 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10982424-90bc-43da-ac36-b84c3eb28e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:31.941592 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982424-90bc-43da-ac36-b84c3eb28e88 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:31.941889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982424-90bc-43da-ac36-b84c3eb28e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:31.942138 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10982424-90bc-43da-ac36-b84c3eb28e88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:31.953728 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10982424-90bc-43da-ac36-b84c3eb28e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:31.958072 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10982424-90bc-43da-ac36-b84c3eb28e88 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-10982424-90bc-43da-ac36-b84c3eb28e88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:31.972055 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:31.973193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:31.974140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:31.974140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:31.974140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:31.974140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:31.974140 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 626/1251] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:31.981439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abd139-33dc-4335-9a08-027bf23046f2 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:32.094289 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abd139-33dc-4335-9a08-027bf23046f2 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:32.127347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b0ee3-4c9a-4dfc-ae45-1f28d459e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:32.128016 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b0ee3-4c9a-4dfc-ae45-1f28d459e5ed None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.128299 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b0ee3-4c9a-4dfc-ae45-1f28d459e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.128619 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b0ee3-4c9a-4dfc-ae45-1f28d459e5ed None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:32.129355 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b0ee3-4c9a-4dfc-ae45-1f28d459e5ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:32.129661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b0ee3-4c9a-4dfc-ae45-1f28d459e5ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:32.131018 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b0ee3-4c9a-4dfc-ae45-1f28d459e5ed None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:32.142861 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 626/1252] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:32.148821 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10982424-90bc-43da-ac36-b84c3eb28e88 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:32.259273 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10982424-90bc-43da-ac36-b84c3eb28e88 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:32.288316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:32.297442 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715922b2-9a6d-4c9c-9400-13ce35ac0625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:32.298272 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715922b2-9a6d-4c9c-9400-13ce35ac0625 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.298551 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715922b2-9a6d-4c9c-9400-13ce35ac0625 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.298831 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715922b2-9a6d-4c9c-9400-13ce35ac0625 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:32.307924 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715922b2-9a6d-4c9c-9400-13ce35ac0625 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:32.310087 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715922b2-9a6d-4c9c-9400-13ce35ac0625 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-715922b2-9a6d-4c9c-9400-13ce35ac0625 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:32.326301 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:32.327808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:32.335369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:32.335369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:32.335369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:32.335369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:32.335369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:32.335369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:32.335369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:32.335369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:32.335369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:32.335369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:32.335369 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 627/1253] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:32.336164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b0ee3-4c9a-4dfc-ae45-1f28d459e5ed None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:32.399490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:32.443820 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b0ee3-4c9a-4dfc-ae45-1f28d459e5ed None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:32.480276 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e176b1-5ec6-41e1-894d-b19f7c396c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:32.481486 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e176b1-5ec6-41e1-894d-b19f7c396c88 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.481763 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e176b1-5ec6-41e1-894d-b19f7c396c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.482034 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e176b1-5ec6-41e1-894d-b19f7c396c88 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:32.482884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e176b1-5ec6-41e1-894d-b19f7c396c88 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:32.483186 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e176b1-5ec6-41e1-894d-b19f7c396c88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:32.484660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e176b1-5ec6-41e1-894d-b19f7c396c88 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:32.504413 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 627/1254] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 19:50:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:32.510323 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715922b2-9a6d-4c9c-9400-13ce35ac0625 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:32.615556 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715922b2-9a6d-4c9c-9400-13ce35ac0625 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:32.668091 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e79dde-33b7-4b97-8f11-2752167e8e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:32.669208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e79dde-33b7-4b97-8f11-2752167e8e0e None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.669488 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e79dde-33b7-4b97-8f11-2752167e8e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.669764 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e79dde-33b7-4b97-8f11-2752167e8e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:32.727225 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:32.739990 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e79dde-33b7-4b97-8f11-2752167e8e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:32.741502 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e79dde-33b7-4b97-8f11-2752167e8e0e None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:32.844137 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 628/1255] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:50:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 19:50:32.851767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e176b1-5ec6-41e1-894d-b19f7c396c88 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:32.954173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e176b1-5ec6-41e1-894d-b19f7c396c88 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:32.970806 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:32.992689 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4be5a-b319-422a-8e80-b7634dcd3451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:32.993499 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4be5a-b319-422a-8e80-b7634dcd3451 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:32.993777 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4be5a-b319-422a-8e80-b7634dcd3451 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:32.994049 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4be5a-b319-422a-8e80-b7634dcd3451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:33.004617 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4be5a-b319-422a-8e80-b7634dcd3451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:33.006115 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4be5a-b319-422a-8e80-b7634dcd3451 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-28c4be5a-b319-422a-8e80-b7634dcd3451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:33.023916 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:33.025595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:33.027338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:33.027338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:33.027338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:33.027338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:33.028533 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 628/1256] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:33.031506 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e79dde-33b7-4b97-8f11-2752167e8e0e None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:33.120626 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:33.134863 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e79dde-33b7-4b97-8f11-2752167e8e0e None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:33.169969 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2558d2-bda4-4a80-852c-0bbdeea5d865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:33.170661 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2558d2-bda4-4a80-852c-0bbdeea5d865 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:33.171126 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2558d2-bda4-4a80-852c-0bbdeea5d865 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:33.171775 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2558d2-bda4-4a80-852c-0bbdeea5d865 None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:33.172541 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2558d2-bda4-4a80-852c-0bbdeea5d865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:33.172847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2558d2-bda4-4a80-852c-0bbdeea5d865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:33.174235 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2558d2-bda4-4a80-852c-0bbdeea5d865 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:33.186552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 629/1257] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:33.194071 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4be5a-b319-422a-8e80-b7634dcd3451 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:33.295135 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4be5a-b319-422a-8e80-b7634dcd3451 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:33.330675 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b66592-ada1-4bfb-b041-0e2744a6878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:33.331525 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b66592-ada1-4bfb-b041-0e2744a6878f None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:33.331781 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b66592-ada1-4bfb-b041-0e2744a6878f None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:33.332048 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b66592-ada1-4bfb-b041-0e2744a6878f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:33.341464 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b66592-ada1-4bfb-b041-0e2744a6878f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:33.343198 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b66592-ada1-4bfb-b041-0e2744a6878f None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-c2b66592-ada1-4bfb-b041-0e2744a6878f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:33.358268 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:33.359567 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:33.365319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:33.365319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:33.365319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:33.365319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:33.365319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:33.365319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:33.365319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:33.365319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:33.365319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 629/1258] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:33.365920 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2558d2-bda4-4a80-852c-0bbdeea5d865 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:33.470058 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2558d2-bda4-4a80-852c-0bbdeea5d865 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:33.504000 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ddc0e-576d-448d-bb7b-a5cea1b7e91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:33.504684 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ddc0e-576d-448d-bb7b-a5cea1b7e91a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:33.504963 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ddc0e-576d-448d-bb7b-a5cea1b7e91a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:33.505234 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ddc0e-576d-448d-bb7b-a5cea1b7e91a None admin] PATH_INFO: `/v3/projects` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:33.506009 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ddc0e-576d-448d-bb7b-a5cea1b7e91a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:33.506314 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ddc0e-576d-448d-bb7b-a5cea1b7e91a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:33.507743 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084ddc0e-576d-448d-bb7b-a5cea1b7e91a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:33.519808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 630/1259] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:33.525792 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b66592-ada1-4bfb-b041-0e2744a6878f None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:33.555733 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:33.630886 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b66592-ada1-4bfb-b041-0e2744a6878f None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:33.666147 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17610f-fb83-460a-8523-7b62c288d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:33.666810 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17610f-fb83-460a-8523-7b62c288d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:33.667096 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17610f-fb83-460a-8523-7b62c288d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:33.667372 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17610f-fb83-460a-8523-7b62c288d9c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:33.668252 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17610f-fb83-460a-8523-7b62c288d9c4 None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=28997ce3cd2e41feaf22b9742ba5e510) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:33.669142 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17610f-fb83-460a-8523-7b62c288d9c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:33.670719 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17610f-fb83-460a-8523-7b62c288d9c4 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:33.676570 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 630/1260] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:50:33 2022] GET /identity/v3/role_assignments?user.id=28997ce3cd2e41feaf22b9742ba5e510&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:35.931558 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7bf5a-0f9a-478f-bdc5-965d43e8ca19 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.932108 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7bf5a-0f9a-478f-bdc5-965d43e8ca19 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.932108 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7bf5a-0f9a-478f-bdc5-965d43e8ca19 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.933540 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 631/1261] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:35.939050 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208efef-4f7f-4e70-bba8-68093bfdc5e9 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:35.939521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208efef-4f7f-4e70-bba8-68093bfdc5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:35.939628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208efef-4f7f-4e70-bba8-68093bfdc5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:35.956888 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-4208efef-4f7f-4e70-bba8-68093bfdc5e9 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:35.958092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-4208efef-4f7f-4e70-bba8-68093bfdc5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:36.027492 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 631/1262] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:50:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:50:36.051754 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7bf5a-0f9a-478f-bdc5-965d43e8ca19 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:36.063756 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:36.160098 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7bf5a-0f9a-478f-bdc5-965d43e8ca19 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.197102 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ffe3c-8f44-43ea-a861-ac99036f9835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.197825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ffe3c-8f44-43ea-a861-ac99036f9835 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.198105 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ffe3c-8f44-43ea-a861-ac99036f9835 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.198377 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ffe3c-8f44-43ea-a861-ac99036f9835 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.209218 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ffe3c-8f44-43ea-a861-ac99036f9835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-788ffe3c-8f44-43ea-a861-ac99036f9835 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:36.216998 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:36.218796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:36.220448 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 632/1263] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:36.228975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208efef-4f7f-4e70-bba8-68093bfdc5e9 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:36.248647 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:36.337078 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208efef-4f7f-4e70-bba8-68093bfdc5e9 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.378426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fc573-de09-48e1-bd57-7cb13460d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.379103 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fc573-de09-48e1-bd57-7cb13460d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.379386 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fc573-de09-48e1-bd57-7cb13460d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.379663 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fc573-de09-48e1-bd57-7cb13460d6f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.380442 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fc573-de09-48e1-bd57-7cb13460d6f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:36.380750 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fc573-de09-48e1-bd57-7cb13460d6f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.382115 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fc573-de09-48e1-bd57-7cb13460d6f2 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.390786 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 632/1264] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:36.398144 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ffe3c-8f44-43ea-a861-ac99036f9835 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:36.479883 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:36.515190 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ffe3c-8f44-43ea-a861-ac99036f9835 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.550773 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098609bd-528b-4e20-a1e8-c28b4fee83e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.551571 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098609bd-528b-4e20-a1e8-c28b4fee83e8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.551856 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098609bd-528b-4e20-a1e8-c28b4fee83e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.552128 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098609bd-528b-4e20-a1e8-c28b4fee83e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.563653 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098609bd-528b-4e20-a1e8-c28b4fee83e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.565163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098609bd-528b-4e20-a1e8-c28b4fee83e8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-098609bd-528b-4e20-a1e8-c28b4fee83e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:36.582495 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:36.584149 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:36.585868 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:36.585868 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:36.585868 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:36.585868 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:36.585868 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 633/1265] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:36.589593 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fc573-de09-48e1-bd57-7cb13460d6f2 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:36.673793 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:36.692706 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fc573-de09-48e1-bd57-7cb13460d6f2 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.726197 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7612a-6be5-431a-b3ce-eeae5d416f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.726849 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7612a-6be5-431a-b3ce-eeae5d416f48 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.727130 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7612a-6be5-431a-b3ce-eeae5d416f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.727406 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7612a-6be5-431a-b3ce-eeae5d416f48 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.728120 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7612a-6be5-431a-b3ce-eeae5d416f48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:36.728451 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7612a-6be5-431a-b3ce-eeae5d416f48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.729807 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7612a-6be5-431a-b3ce-eeae5d416f48 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.743056 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 633/1266] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:36.747884 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098609bd-528b-4e20-a1e8-c28b4fee83e8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:36.857474 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098609bd-528b-4e20-a1e8-c28b4fee83e8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:36.879650 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:36.895115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b57914-c058-4202-84da-8963cf9bd1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:36.895923 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b57914-c058-4202-84da-8963cf9bd1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:36.896208 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b57914-c058-4202-84da-8963cf9bd1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:36.902158 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b57914-c058-4202-84da-8963cf9bd1a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:36.905129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b57914-c058-4202-84da-8963cf9bd1a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:36.906589 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b57914-c058-4202-84da-8963cf9bd1a4 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-93b57914-c058-4202-84da-8963cf9bd1a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:36.921651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:36.923120 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:36.924061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:36.924061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:36.924061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:36.924061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:36.924061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:36.924061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:36.924061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:36.924061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:36.924061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:36.924061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:36.924061 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 634/1267] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:36.929376 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7612a-6be5-431a-b3ce-eeae5d416f48 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:37.033384 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7612a-6be5-431a-b3ce-eeae5d416f48 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:37.070819 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63f1db-c765-4f3f-a013-ef9ae73a1f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:37.071488 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63f1db-c765-4f3f-a013-ef9ae73a1f10 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:37.071765 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63f1db-c765-4f3f-a013-ef9ae73a1f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:37.072036 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63f1db-c765-4f3f-a013-ef9ae73a1f10 None admin] PATH_INFO: `/v3/users` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:37.072924 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63f1db-c765-4f3f-a013-ef9ae73a1f10 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:37.073226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63f1db-c765-4f3f-a013-ef9ae73a1f10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:37.075318 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63f1db-c765-4f3f-a013-ef9ae73a1f10 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:37.086721 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:37.098226 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 634/1268] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 19:50:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:37.104398 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b57914-c058-4202-84da-8963cf9bd1a4 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:37.231555 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b57914-c058-4202-84da-8963cf9bd1a4 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:37.266534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b011fb-0bc0-40a5-bceb-72ddd9d60eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:37.267317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b011fb-0bc0-40a5-bceb-72ddd9d60eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:37.267603 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b011fb-0bc0-40a5-bceb-72ddd9d60eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:37.267876 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b011fb-0bc0-40a5-bceb-72ddd9d60eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:37.285562 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b011fb-0bc0-40a5-bceb-72ddd9d60eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:37.287115 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b011fb-0bc0-40a5-bceb-72ddd9d60eb8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-a1b011fb-0bc0-40a5-bceb-72ddd9d60eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:37.305889 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:37.310316 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:37.311544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:37.311544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:37.311544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:37.311544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:37.311544 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 635/1269] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:37.313758 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63f1db-c765-4f3f-a013-ef9ae73a1f10 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:37.460998 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63f1db-c765-4f3f-a013-ef9ae73a1f10 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:37.498352 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352996c9-0765-4af5-b61f-00cb05696675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:37.499051 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352996c9-0765-4af5-b61f-00cb05696675 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:37.499332 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352996c9-0765-4af5-b61f-00cb05696675 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:37.499610 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352996c9-0765-4af5-b61f-00cb05696675 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:37.500553 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352996c9-0765-4af5-b61f-00cb05696675 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:37.500694 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352996c9-0765-4af5-b61f-00cb05696675 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:37.502150 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352996c9-0765-4af5-b61f-00cb05696675 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:37.518154 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 635/1270] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:37.524664 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b011fb-0bc0-40a5-bceb-72ddd9d60eb8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:37.568232 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:37.647782 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b011fb-0bc0-40a5-bceb-72ddd9d60eb8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:37.684724 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238aa154-261a-44c8-9696-6c63eecd93a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:37.685523 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa154-261a-44c8-9696-6c63eecd93a8 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:37.685809 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa154-261a-44c8-9696-6c63eecd93a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:37.686088 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238aa154-261a-44c8-9696-6c63eecd93a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:37.693354 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238aa154-261a-44c8-9696-6c63eecd93a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:37.694850 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238aa154-261a-44c8-9696-6c63eecd93a8 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-238aa154-261a-44c8-9696-6c63eecd93a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:37.709066 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:37.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:37.712879 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:37.712879 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:37.712879 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:37.712879 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:37.712879 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:37.712879 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:37.712879 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:37.712879 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:37.712879 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 636/1271] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:50:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:37.718071 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352996c9-0765-4af5-b61f-00cb05696675 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:37.784136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:37.831119 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352996c9-0765-4af5-b61f-00cb05696675 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:37.867280 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6889b16-bfd8-4756-8369-84cb324c6703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:37.867948 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6889b16-bfd8-4756-8369-84cb324c6703 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:37.868234 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6889b16-bfd8-4756-8369-84cb324c6703 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:37.868543 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6889b16-bfd8-4756-8369-84cb324c6703 None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:37.869341 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6889b16-bfd8-4756-8369-84cb324c6703 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:37.869654 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6889b16-bfd8-4756-8369-84cb324c6703 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:37.871139 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6889b16-bfd8-4756-8369-84cb324c6703 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:37.885061 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 636/1272] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:50:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:37.891183 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238aa154-261a-44c8-9696-6c63eecd93a8 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:37.990385 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:38.000284 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238aa154-261a-44c8-9696-6c63eecd93a8 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:38.039196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1e4c1-8a91-4c35-bc42-7f2f6a77373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:38.040203 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e4c1-8a91-4c35-bc42-7f2f6a77373f None admin] REQUEST_METHOD: `PUT` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:38.040630 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e4c1-8a91-4c35-bc42-7f2f6a77373f None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:38.041000 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e4c1-8a91-4c35-bc42-7f2f6a77373f None admin] PATH_INFO: `/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/28997ce3cd2e41feaf22b9742ba5e510/roles/815bd48c3df0405babfd5b53430fd3ea` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:38.043981 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1e4c1-8a91-4c35-bc42-7f2f6a77373f None admin] RBAC: Authorizing `identity:create_grant(project_id=b14a16a022894289a30e8697bef1fcf0, user_id=28997ce3cd2e41feaf22b9742ba5e510, role_id=815bd48c3df0405babfd5b53430fd3ea)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:38.045522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1e4c1-8a91-4c35-bc42-7f2f6a77373f None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:38.181789 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 637/1273] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 19:50:37 2022] PUT /identity/v3/projects/b14a16a022894289a30e8697bef1fcf0/users/28997ce3cd2e41feaf22b9742ba5e510/roles/815bd48c3df0405babfd5b53430fd3ea => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 19:50:40.410155 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e1931-2781-44c2-adab-5e3be79084c6 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.410718 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e1931-2781-44c2-adab-5e3be79084c6 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.410718 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e1931-2781-44c2-adab-5e3be79084c6 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.412510 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 637/1274] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:40.417932 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201f84b-373a-4432-b897-727b1cde028a None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.418319 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201f84b-373a-4432-b897-727b1cde028a None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.418515 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201f84b-373a-4432-b897-727b1cde028a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.435972 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-1201f84b-373a-4432-b897-727b1cde028a None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:40.437215 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-1201f84b-373a-4432-b897-727b1cde028a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:40.503246 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 638/1275] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:50:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 19:50:40.526887 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e1931-2781-44c2-adab-5e3be79084c6 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:40.626874 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e1931-2781-44c2-adab-5e3be79084c6 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:40.643094 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:40.659436 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96242c24-898d-4204-b24a-7428eecfbd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:40.660214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242c24-898d-4204-b24a-7428eecfbd91 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.660523 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242c24-898d-4204-b24a-7428eecfbd91 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.660793 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242c24-898d-4204-b24a-7428eecfbd91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.671256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96242c24-898d-4204-b24a-7428eecfbd91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-96242c24-898d-4204-b24a-7428eecfbd91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 19:50:40.679402 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:40.680559 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:40.681439 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 638/1276] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 19:50:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:40.686721 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201f84b-373a-4432-b897-727b1cde028a None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:40.767634 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:40.785384 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201f84b-373a-4432-b897-727b1cde028a None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:40.818779 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cba565-8b20-4435-bf53-ed783879d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:40.819438 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cba565-8b20-4435-bf53-ed783879d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.819723 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cba565-8b20-4435-bf53-ed783879d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.819997 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cba565-8b20-4435-bf53-ed783879d4f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.820779 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cba565-8b20-4435-bf53-ed783879d4f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:40.821086 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cba565-8b20-4435-bf53-ed783879d4f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:40.822461 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cba565-8b20-4435-bf53-ed783879d4f7 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:40.833077 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 639/1277] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 19:50:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:40.838930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96242c24-898d-4204-b24a-7428eecfbd91 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:40.938366 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96242c24-898d-4204-b24a-7428eecfbd91 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:40.972071 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad597f-842c-442f-920b-892a381990b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:40.972891 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad597f-842c-442f-920b-892a381990b1 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:40.973248 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad597f-842c-442f-920b-892a381990b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:40.973529 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad597f-842c-442f-920b-892a381990b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:40.983835 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad597f-842c-442f-920b-892a381990b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:40.985421 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad597f-842c-442f-920b-892a381990b1 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-3dad597f-842c-442f-920b-892a381990b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:41.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:41.003422 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:41.004751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:41.004751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:41.004751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:41.004751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:41.004751 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 639/1278] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:41.009257 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cba565-8b20-4435-bf53-ed783879d4f7 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:41.110670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cba565-8b20-4435-bf53-ed783879d4f7 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.145083 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce7fd4-82bf-4ac0-ba43-0a89f19f3e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.145731 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce7fd4-82bf-4ac0-ba43-0a89f19f3e5b None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.146018 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce7fd4-82bf-4ac0-ba43-0a89f19f3e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.146297 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce7fd4-82bf-4ac0-ba43-0a89f19f3e5b None admin] PATH_INFO: `/v3/domains` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.147018 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce7fd4-82bf-4ac0-ba43-0a89f19f3e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:41.147326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce7fd4-82bf-4ac0-ba43-0a89f19f3e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:41.148808 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce7fd4-82bf-4ac0-ba43-0a89f19f3e5b None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.160826 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 640/1279] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:41.166411 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad597f-842c-442f-920b-892a381990b1 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:41.197748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:41.268640 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad597f-842c-442f-920b-892a381990b1 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.302106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8468d-6b6a-4ca9-9a65-07719c4309cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.302969 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8468d-6b6a-4ca9-9a65-07719c4309cc None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.303247 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8468d-6b6a-4ca9-9a65-07719c4309cc None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.303521 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf8468d-6b6a-4ca9-9a65-07719c4309cc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.311467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8468d-6b6a-4ca9-9a65-07719c4309cc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:41.312951 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf8468d-6b6a-4ca9-9a65-07719c4309cc None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-baf8468d-6b6a-4ca9-9a65-07719c4309cc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 19:50:41.327867 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:41.329092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:41.330214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:41.330214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:41.330214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 19:50:41.330214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 19:50:41.330214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 19:50:41.330214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 19:50:41.330214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 19:50:41.330214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 19:50:41.330214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 19:50:41.330214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:41.330214 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 640/1280] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:41.335014 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce7fd4-82bf-4ac0-ba43-0a89f19f3e5b None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:41.339073 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:41.435258 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce7fd4-82bf-4ac0-ba43-0a89f19f3e5b None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.469555 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566e17e-e9d7-4dca-994e-ac3a20c68fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.470144 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566e17e-e9d7-4dca-994e-ac3a20c68fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.470424 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566e17e-e9d7-4dca-994e-ac3a20c68fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.470697 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566e17e-e9d7-4dca-994e-ac3a20c68fb5 None admin] PATH_INFO: `/v3/users` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.471527 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566e17e-e9d7-4dca-994e-ac3a20c68fb5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:41.471830 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566e17e-e9d7-4dca-994e-ac3a20c68fb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:41.473380 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566e17e-e9d7-4dca-994e-ac3a20c68fb5 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.490757 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 641/1281] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 19:50:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:41.497821 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8468d-6b6a-4ca9-9a65-07719c4309cc None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:41.586137 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:41.595429 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf8468d-6b6a-4ca9-9a65-07719c4309cc None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.630129 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4397d-6259-409d-967e-bf331d96ea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.630764 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4397d-6259-409d-967e-bf331d96ea60 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.631042 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4397d-6259-409d-967e-bf331d96ea60 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.631315 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4397d-6259-409d-967e-bf331d96ea60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.698320 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4397d-6259-409d-967e-bf331d96ea60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:41.699766 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4397d-6259-409d-967e-bf331d96ea60 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.785379 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:41.795914 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 641/1282] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 19:50:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 19:50:41.801753 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566e17e-e9d7-4dca-994e-ac3a20c68fb5 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:41.862837 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:41.899926 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566e17e-e9d7-4dca-994e-ac3a20c68fb5 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:41.935392 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57843a-e687-42f3-90bc-d2765147f5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:41.936176 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57843a-e687-42f3-90bc-d2765147f5ef None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:41.936486 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57843a-e687-42f3-90bc-d2765147f5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:41.936757 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57843a-e687-42f3-90bc-d2765147f5ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:41.946522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57843a-e687-42f3-90bc-d2765147f5ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:41.947973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57843a-e687-42f3-90bc-d2765147f5ef None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-fa57843a-e687-42f3-90bc-d2765147f5ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 19:50:41.965733 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:41.966844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:41.968317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 19:50:41.968317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 19:50:41.968317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 19:50:41.968317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:41.968317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 642/1283] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 19:50:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:41.972682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4397d-6259-409d-967e-bf331d96ea60 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:42.070122 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4397d-6259-409d-967e-bf331d96ea60 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:42.103389 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93536b-be96-4148-8dc6-ac4e9c2e5d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:42.104040 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93536b-be96-4148-8dc6-ac4e9c2e5d20 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:42.104324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93536b-be96-4148-8dc6-ac4e9c2e5d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:42.104629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93536b-be96-4148-8dc6-ac4e9c2e5d20 None admin] PATH_INFO: `/v3/domains` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:42.105404 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93536b-be96-4148-8dc6-ac4e9c2e5d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:42.105760 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93536b-be96-4148-8dc6-ac4e9c2e5d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:42.107620 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93536b-be96-4148-8dc6-ac4e9c2e5d20 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:42.118645 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 642/1284] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 19:50:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:42.124534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57843a-e687-42f3-90bc-d2765147f5ef None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:42.224548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57843a-e687-42f3-90bc-d2765147f5ef None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:42.238314 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:42.257510 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1060ed-4a7d-4bc9-a109-7a257bc6ffff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:42.258296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1060ed-4a7d-4bc9-a109-7a257bc6ffff None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:42.258577 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1060ed-4a7d-4bc9-a109-7a257bc6ffff None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:42.258853 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1060ed-4a7d-4bc9-a109-7a257bc6ffff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:42.266044 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1060ed-4a7d-4bc9-a109-7a257bc6ffff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:42.267620 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1060ed-4a7d-4bc9-a109-7a257bc6ffff None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application [None req-ef1060ed-4a7d-4bc9-a109-7a257bc6ffff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 19:50:42.282136 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application with Lock( May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:42.284130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:42.285908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 19:50:42.285908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 19:50:42.285908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 19:50:42.285908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 19:50:42.285908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 19:50:42.285908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 19:50:42.285908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 19:50:42.285908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: ERROR keystone.server.flask.application May 19 19:50:42.285908 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 643/1285] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 19:50:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:42.289569 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93536b-be96-4148-8dc6-ac4e9c2e5d20 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:42.343097 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:42.392527 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93536b-be96-4148-8dc6-ac4e9c2e5d20 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:42.426199 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f4c65-523a-4750-960e-37fb5ebd342c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:42.426846 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f4c65-523a-4750-960e-37fb5ebd342c None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:42.427335 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f4c65-523a-4750-960e-37fb5ebd342c None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:42.428136 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f4c65-523a-4750-960e-37fb5ebd342c None admin] PATH_INFO: `/v3/projects` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:42.429326 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f4c65-523a-4750-960e-37fb5ebd342c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:42.429643 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f4c65-523a-4750-960e-37fb5ebd342c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:42.431125 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f4c65-523a-4750-960e-37fb5ebd342c None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:42.442826 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 643/1286] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 19:50:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 19:50:42.449014 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1060ed-4a7d-4bc9-a109-7a257bc6ffff None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:42.547379 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1060ed-4a7d-4bc9-a109-7a257bc6ffff None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:42.581486 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727d265-47ac-4c77-adb7-b89d2a9d51aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:42.582125 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727d265-47ac-4c77-adb7-b89d2a9d51aa None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:42.582407 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727d265-47ac-4c77-adb7-b89d2a9d51aa None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:42.582688 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727d265-47ac-4c77-adb7-b89d2a9d51aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:42.583513 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727d265-47ac-4c77-adb7-b89d2a9d51aa None admin] RBAC: Adding query filter params (role.id=815bd48c3df0405babfd5b53430fd3ea, scope.project.id=b14a16a022894289a30e8697bef1fcf0, user.id=28997ce3cd2e41feaf22b9742ba5e510) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:42.583815 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727d265-47ac-4c77-adb7-b89d2a9d51aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:42.585358 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727d265-47ac-4c77-adb7-b89d2a9d51aa None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:42.591317 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 644/1287] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 19:50:42 2022] GET /identity/v3/role_assignments?user.id=28997ce3cd2e41feaf22b9742ba5e510&scope.project.id=b14a16a022894289a30e8697bef1fcf0&role.id=815bd48c3df0405babfd5b53430fd3ea => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:44.829994 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49c50a-759c-42d2-a8c5-94be91e3895e None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.830544 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49c50a-759c-42d2-a8c5-94be91e3895e None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.830544 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49c50a-759c-42d2-a8c5-94be91e3895e None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.831906 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 644/1288] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:44.837186 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969d64d-4f69-4534-9547-c16d837ba44d None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:44.838487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969d64d-4f69-4534-9547-c16d837ba44d None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:44.838487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969d64d-4f69-4534-9547-c16d837ba44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:44.855125 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-7969d64d-4f69-4534-9547-c16d837ba44d None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:44.856973 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-7969d64d-4f69-4534-9547-c16d837ba44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:44.885648 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:44.923352 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 645/1289] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:50:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:50:44.946301 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49c50a-759c-42d2-a8c5-94be91e3895e None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.046977 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49c50a-759c-42d2-a8c5-94be91e3895e None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.058411 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:45.081698 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb60d4-4673-4479-85bc-f8efed0fb902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.082511 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb60d4-4673-4479-85bc-f8efed0fb902 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.082805 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb60d4-4673-4479-85bc-f8efed0fb902 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.083056 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb60d4-4673-4479-85bc-f8efed0fb902 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.083701 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb60d4-4673-4479-85bc-f8efed0fb902 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:45.085085 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb60d4-4673-4479-85bc-f8efed0fb902 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-b4bb60d4-4673-4479-85bc-f8efed0fb902 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:45.097310 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:45.100164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:45.101105 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 645/1290] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 19:50:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:45.104627 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969d64d-4f69-4534-9547-c16d837ba44d None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.206658 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969d64d-4f69-4534-9547-c16d837ba44d None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.241337 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bf724-9378-409a-bb65-4006e164b611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.242001 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bf724-9378-409a-bb65-4006e164b611 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.242278 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bf724-9378-409a-bb65-4006e164b611 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.242611 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bf724-9378-409a-bb65-4006e164b611 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.243239 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bf724-9378-409a-bb65-4006e164b611 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:45.243546 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bf724-9378-409a-bb65-4006e164b611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:45.244950 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bf724-9378-409a-bb65-4006e164b611 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:45.249783 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 646/1291] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:45.255519 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb60d4-4673-4479-85bc-f8efed0fb902 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:45.357946 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb60d4-4673-4479-85bc-f8efed0fb902 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:45.390684 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ed05f-0f05-400d-bf3a-fca29aa672ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:45.391349 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ed05f-0f05-400d-bf3a-fca29aa672ff None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:45.391628 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ed05f-0f05-400d-bf3a-fca29aa672ff None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:45.391905 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ed05f-0f05-400d-bf3a-fca29aa672ff None admin] PATH_INFO: `/v3/services` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:45.392649 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ed05f-0f05-400d-bf3a-fca29aa672ff None admin] RBAC: Adding query filter params (type=placement) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:45.392954 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ed05f-0f05-400d-bf3a-fca29aa672ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:45.394314 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ed05f-0f05-400d-bf3a-fca29aa672ff None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:45.398707 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 646/1292] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 19:50:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:47.603132 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b3ee5-8f56-406b-9cbc-73676c402037 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.603632 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b3ee5-8f56-406b-9cbc-73676c402037 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.603632 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b3ee5-8f56-406b-9cbc-73676c402037 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.605844 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 647/1293] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:50:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:47.612493 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2e73-a019-444a-b102-d1abbd3c379d None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.612975 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2e73-a019-444a-b102-d1abbd3c379d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.613068 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b2e73-a019-444a-b102-d1abbd3c379d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.629943 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-2d1b2e73-a019-444a-b102-d1abbd3c379d None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:47.635254 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-2d1b2e73-a019-444a-b102-d1abbd3c379d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:47.661756 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:47.698043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 647/1294] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:50:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:50:47.721567 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b3ee5-8f56-406b-9cbc-73676c402037 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:47.756155 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:47.822775 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b3ee5-8f56-406b-9cbc-73676c402037 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:47.855603 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f1a33-3f95-449b-b278-cad3e4cf84ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:47.856518 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f1a33-3f95-449b-b278-cad3e4cf84ae None admin] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:47.856847 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f1a33-3f95-449b-b278-cad3e4cf84ae None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:47.857124 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f1a33-3f95-449b-b278-cad3e4cf84ae None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:47.857839 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f1a33-3f95-449b-b278-cad3e4cf84ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:47.859211 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f1a33-3f95-449b-b278-cad3e4cf84ae None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:47.992767 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 648/1295] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 19:50:47 2022] POST /identity/v3/services => generated 246 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:50:50.239397 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02741aec-9800-4ab9-a296-7dec50803baa None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.239958 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02741aec-9800-4ab9-a296-7dec50803baa None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.239958 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02741aec-9800-4ab9-a296-7dec50803baa None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.241346 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 648/1296] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:50.247132 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9277f7-d1d0-4f96-a33d-13581ab319d8 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.247550 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9277f7-d1d0-4f96-a33d-13581ab319d8 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.247670 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9277f7-d1d0-4f96-a33d-13581ab319d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.265287 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-6b9277f7-d1d0-4f96-a33d-13581ab319d8 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:50.266467 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-6b9277f7-d1d0-4f96-a33d-13581ab319d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:50.339734 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 649/1297] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:50:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:50:50.361935 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02741aec-9800-4ab9-a296-7dec50803baa None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.463660 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02741aec-9800-4ab9-a296-7dec50803baa None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.497927 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10301fff-f651-43ac-8f7a-eee7cbc64356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.498642 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10301fff-f651-43ac-8f7a-eee7cbc64356 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.498922 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10301fff-f651-43ac-8f7a-eee7cbc64356 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.499193 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10301fff-f651-43ac-8f7a-eee7cbc64356 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.499843 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10301fff-f651-43ac-8f7a-eee7cbc64356 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:50.501224 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10301fff-f651-43ac-8f7a-eee7cbc64356 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-10301fff-f651-43ac-8f7a-eee7cbc64356 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:50.513256 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:50.514815 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:50.515739 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 649/1298] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 19:50:50.521534 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9277f7-d1d0-4f96-a33d-13581ab319d8 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.539804 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:50.618828 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9277f7-d1d0-4f96-a33d-13581ab319d8 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.652048 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce410e7-28b7-4d44-8f28-8ab9d344aea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.652763 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce410e7-28b7-4d44-8f28-8ab9d344aea9 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.653093 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce410e7-28b7-4d44-8f28-8ab9d344aea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.653418 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce410e7-28b7-4d44-8f28-8ab9d344aea9 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.654060 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce410e7-28b7-4d44-8f28-8ab9d344aea9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:50.654359 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce410e7-28b7-4d44-8f28-8ab9d344aea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:50.655718 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce410e7-28b7-4d44-8f28-8ab9d344aea9 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:50.660776 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 650/1299] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:50.666777 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10301fff-f651-43ac-8f7a-eee7cbc64356 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.699523 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:50.765324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10301fff-f651-43ac-8f7a-eee7cbc64356 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.798702 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84038f1-0950-4ce2-a701-474dc3a736de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.799347 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84038f1-0950-4ce2-a701-474dc3a736de None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.799631 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84038f1-0950-4ce2-a701-474dc3a736de None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.799901 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84038f1-0950-4ce2-a701-474dc3a736de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.800719 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84038f1-0950-4ce2-a701-474dc3a736de None admin] RBAC: Adding query filter params (interface=public, service_id=fdf7db71787f4d359a4eb901ee0da813, region_id=RegionOne) {{(pid=80452) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:50.801026 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84038f1-0950-4ce2-a701-474dc3a736de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:50.802366 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84038f1-0950-4ce2-a701-474dc3a736de None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:50.808076 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 650/1300] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/endpoints?service_id=fdf7db71787f4d359a4eb901ee0da813&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:50.813854 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce410e7-28b7-4d44-8f28-8ab9d344aea9 None admin] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:50.894130 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:50.912948 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce410e7-28b7-4d44-8f28-8ab9d344aea9 None admin] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:50.947443 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5493003-26d8-441a-bf1e-b07288101974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:50.948092 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5493003-26d8-441a-bf1e-b07288101974 None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:50.948398 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5493003-26d8-441a-bf1e-b07288101974 None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:50.948675 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5493003-26d8-441a-bf1e-b07288101974 None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:50.949416 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5493003-26d8-441a-bf1e-b07288101974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:50.950848 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5493003-26d8-441a-bf1e-b07288101974 None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:50.957762 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 651/1301] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 19:50:50 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 19:50:53.201668 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16701cdf-86bf-4363-b5b2-206b0ce3b946 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.202230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16701cdf-86bf-4363-b5b2-206b0ce3b946 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.202230 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16701cdf-86bf-4363-b5b2-206b0ce3b946 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.203580 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 651/1302] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:50:53.208941 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb8371-8b1f-406f-a766-5104323671b5 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.209331 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb8371-8b1f-406f-a766-5104323671b5 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.209501 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb8371-8b1f-406f-a766-5104323671b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.226447 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-f8cb8371-8b1f-406f-a766-5104323671b5 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:50:53.227612 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-f8cb8371-8b1f-406f-a766-5104323671b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:50:53.294673 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 652/1303] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:50:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 19:50:53.318113 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16701cdf-86bf-4363-b5b2-206b0ce3b946 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.414325 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:53.421930 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16701cdf-86bf-4363-b5b2-206b0ce3b946 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.456421 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261990f-f6c2-4913-9ebe-e9c1ad8b6fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.457169 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261990f-f6c2-4913-9ebe-e9c1ad8b6fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.457453 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261990f-f6c2-4913-9ebe-e9c1ad8b6fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.457728 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261990f-f6c2-4913-9ebe-e9c1ad8b6fc6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.458385 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261990f-f6c2-4913-9ebe-e9c1ad8b6fc6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.459787 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261990f-f6c2-4913-9ebe-e9c1ad8b6fc6 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application [None req-5261990f-f6c2-4913-9ebe-e9c1ad8b6fc6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 19:50:53.472375 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 19:50:53.478577 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.get_or_create( May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application with Lock( May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self._enter() May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application return self.creator() May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application created_value = creator( May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: ERROR keystone.server.flask.application May 19 19:50:53.479704 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 652/1304] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 19:50:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 19:50:53.481265 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb8371-8b1f-406f-a766-5104323671b5 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.555833 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:50:53.578999 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb8371-8b1f-406f-a766-5104323671b5 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.612648 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5362a-e7a8-44b8-be0c-feb329c5e70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.613345 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5362a-e7a8-44b8-be0c-feb329c5e70a None admin] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.613631 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5362a-e7a8-44b8-be0c-feb329c5e70a None admin] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.613904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5362a-e7a8-44b8-be0c-feb329c5e70a None admin] PATH_INFO: `/v3/services` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.614608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5362a-e7a8-44b8-be0c-feb329c5e70a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80453) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 19:50:53.614912 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5362a-e7a8-44b8-be0c-feb329c5e70a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.616282 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5362a-e7a8-44b8-be0c-feb329c5e70a None admin] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.622859 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 653/1305] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 19:50:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 19:50:53.628939 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261990f-f6c2-4913-9ebe-e9c1ad8b6fc6 None admin] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:50:53.728416 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261990f-f6c2-4913-9ebe-e9c1ad8b6fc6 None admin] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:50:53.761477 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d751b-ed83-4140-b7f7-364d981f0586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f3a6a2da4254702bc7bd15361b6c9ae', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:50:53.762122 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d751b-ed83-4140-b7f7-364d981f0586 None admin] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:50:53.762400 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d751b-ed83-4140-b7f7-364d981f0586 None admin] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:50:53.762670 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d751b-ed83-4140-b7f7-364d981f0586 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:50:53.763414 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d751b-ed83-4140-b7f7-364d981f0586 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:50:53.764863 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d751b-ed83-4140-b7f7-364d981f0586 None admin] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:50:53.881717 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 653/1306] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 19:50:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 19:51:03.643427 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=30 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:51:03.877027 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:38.204041 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb405f7-3204-4190-aaa9-0c37b0805bfb None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.204633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb405f7-3204-4190-aaa9-0c37b0805bfb None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.204633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb405f7-3204-4190-aaa9-0c37b0805bfb None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.205930 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 654/1307] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:38.212010 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd88f5-587c-4811-b8d7-2d89dcac1d60 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.212485 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd88f5-587c-4811-b8d7-2d89dcac1d60 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.212701 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd88f5-587c-4811-b8d7-2d89dcac1d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.300120 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-31cd88f5-587c-4811-b8d7-2d89dcac1d60 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:38.301953 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-31cd88f5-587c-4811-b8d7-2d89dcac1d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:38.437389 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 654/1308] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:57:38.494710 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb58f13-3d7f-4fb5-8117-1038479f88ee None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.495242 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb58f13-3d7f-4fb5-8117-1038479f88ee None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.495350 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb58f13-3d7f-4fb5-8117-1038479f88ee None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.497219 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 655/1309] 10.222.0.88 () {54 vars in 964 bytes} [Thu May 19 19:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:38.506560 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc28e1a-1e26-41db-bc74-08eed5b891d2 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:38.507158 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc28e1a-1e26-41db-bc74-08eed5b891d2 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:38.507592 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc28e1a-1e26-41db-bc74-08eed5b891d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:38.653359 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-8dc28e1a-1e26-41db-bc74-08eed5b891d2 None None] MFA Rules not processed for user `6d5a01edfe0444f8a6fef9d7ee9645f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:38.655042 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-8dc28e1a-1e26-41db-bc74-08eed5b891d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:38.685884 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:38.747332 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 655/1310] 10.222.0.88 () {58 vars in 1061 bytes} [Thu May 19 19:57:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:57:38.758751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb58f13-3d7f-4fb5-8117-1038479f88ee None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:38.760856 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-efb58f13-3d7f-4fb5-8117-1038479f88ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:38.918162 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb58f13-3d7f-4fb5-8117-1038479f88ee None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:39.003089 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ff169-cce1-4442-95ae-fa856a5db28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:39.003907 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ff169-cce1-4442-95ae-fa856a5db28e service neutron] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:39.004832 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ff169-cce1-4442-95ae-fa856a5db28e service neutron] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:39.005175 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ff169-cce1-4442-95ae-fa856a5db28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:39.007078 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-3a8ff169-cce1-4442-95ae-fa856a5db28e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:39.068731 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:39.135318 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ff169-cce1-4442-95ae-fa856a5db28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:39.137206 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ff169-cce1-4442-95ae-fa856a5db28e service neutron] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:39.157600 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 656/1311] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:57:41.755075 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68ea6e-a4e1-4342-8947-c1b6292958b0 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.756231 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68ea6e-a4e1-4342-8947-c1b6292958b0 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.756231 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68ea6e-a4e1-4342-8947-c1b6292958b0 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.759257 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 656/1312] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:57:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:41.763364 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc38a-b662-46a4-9d46-9c8e7ee5523e None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:41.763884 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc38a-b662-46a4-9d46-9c8e7ee5523e None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:41.763965 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc38a-b662-46a4-9d46-9c8e7ee5523e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:41.847363 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-f39dc38a-b662-46a4-9d46-9c8e7ee5523e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:41.848536 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-f39dc38a-b662-46a4-9d46-9c8e7ee5523e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:41.953631 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 657/1313] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:57:41.991557 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a68ea6e-a4e1-4342-8947-c1b6292958b0 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:41.993140 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-6a68ea6e-a4e1-4342-8947-c1b6292958b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:42.104958 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a68ea6e-a4e1-4342-8947-c1b6292958b0 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:42.184420 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:42.187384 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ccb6f-827a-4a28-bf83-795cfee21840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:42.188206 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ccb6f-827a-4a28-bf83-795cfee21840 service neutron] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:42.188752 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ccb6f-827a-4a28-bf83-795cfee21840 service neutron] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:42.189041 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ccb6f-827a-4a28-bf83-795cfee21840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:42.190725 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-023ccb6f-827a-4a28-bf83-795cfee21840 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:42.314062 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ccb6f-827a-4a28-bf83-795cfee21840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:42.315562 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ccb6f-827a-4a28-bf83-795cfee21840 service neutron] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:42.348572 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 657/1314] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 19:57:45.078560 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429d92-c284-43d8-b371-2cda9e01ac00 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:45.079137 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429d92-c284-43d8-b371-2cda9e01ac00 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:45.079137 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53429d92-c284-43d8-b371-2cda9e01ac00 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:45.080487 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 658/1315] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:45.085790 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5e241-7a3f-4f21-86e1-d32340d29f20 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:45.086192 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5e241-7a3f-4f21-86e1-d32340d29f20 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:45.086345 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5e241-7a3f-4f21-86e1-d32340d29f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:45.148496 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-dcc5e241-7a3f-4f21-86e1-d32340d29f20 None None] MFA Rules not processed for user `c0ff018c6850486aa01127bb8c1bd8b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:45.149706 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-dcc5e241-7a3f-4f21-86e1-d32340d29f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:45.201734 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:45.245507 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 658/1316] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:57:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:57:45.283751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53429d92-c284-43d8-b371-2cda9e01ac00 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:45.285418 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-53429d92-c284-43d8-b371-2cda9e01ac00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:45.295673 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:45.388418 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53429d92-c284-43d8-b371-2cda9e01ac00 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:45.461004 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81dbe3-fd21-4f3e-9938-18ca6f46406e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:45.461651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81dbe3-fd21-4f3e-9938-18ca6f46406e service neutron] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:45.461940 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81dbe3-fd21-4f3e-9938-18ca6f46406e service neutron] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:45.462214 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81dbe3-fd21-4f3e-9938-18ca6f46406e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:45.463915 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-9e81dbe3-fd21-4f3e-9938-18ca6f46406e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:45.527133 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:45.574014 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81dbe3-fd21-4f3e-9938-18ca6f46406e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:45.575479 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81dbe3-fd21-4f3e-9938-18ca6f46406e service neutron] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:45.595830 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 659/1317] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:57:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:57:48.586213 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d22554-657d-4336-a7f7-b01df769e7f5 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.586772 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d22554-657d-4336-a7f7-b01df769e7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.586772 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d22554-657d-4336-a7f7-b01df769e7f5 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.588164 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 659/1318] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:48.593494 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfec23-4620-4991-aafe-512625271242 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.593897 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfec23-4620-4991-aafe-512625271242 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.593997 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfec23-4620-4991-aafe-512625271242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.657681 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-a3bfec23-4620-4991-aafe-512625271242 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:48.658891 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-a3bfec23-4620-4991-aafe-512625271242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:48.754825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 660/1319] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:57:48 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:57:48.792985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d22554-657d-4336-a7f7-b01df769e7f5 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:48.794533 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-b5d22554-657d-4336-a7f7-b01df769e7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:48.895427 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d22554-657d-4336-a7f7-b01df769e7f5 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:48.967665 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d30e3bc-af44-4372-8b2c-e40bd9ca237b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:48.968313 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30e3bc-af44-4372-8b2c-e40bd9ca237b service neutron] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:48.968629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30e3bc-af44-4372-8b2c-e40bd9ca237b service neutron] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:48.968906 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d30e3bc-af44-4372-8b2c-e40bd9ca237b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:48.970574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-8d30e3bc-af44-4372-8b2c-e40bd9ca237b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:48.979389 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:49.077904 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30e3bc-af44-4372-8b2c-e40bd9ca237b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:49.079333 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d30e3bc-af44-4372-8b2c-e40bd9ca237b service neutron] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:49.097437 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 660/1320] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:57:48 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:57:52.505375 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d83533-cc47-49f4-b4c2-c7ffdd6af1cf None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:52.505935 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d83533-cc47-49f4-b4c2-c7ffdd6af1cf None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:52.505935 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d83533-cc47-49f4-b4c2-c7ffdd6af1cf None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:52.507397 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 661/1321] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:52.513548 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc8bcc-c835-4b69-97f4-acdaed9c4889 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:52.518181 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc8bcc-c835-4b69-97f4-acdaed9c4889 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:52.518181 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc8bcc-c835-4b69-97f4-acdaed9c4889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:52.589683 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-64cc8bcc-c835-4b69-97f4-acdaed9c4889 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:52.590947 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-64cc8bcc-c835-4b69-97f4-acdaed9c4889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:52.652208 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:52.695480 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 661/1322] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:57:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:57:52.731238 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d83533-cc47-49f4-b4c2-c7ffdd6af1cf None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:52.732820 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-e2d83533-cc47-49f4-b4c2-c7ffdd6af1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:52.752904 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:52.842102 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d83533-cc47-49f4-b4c2-c7ffdd6af1cf None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:52.922198 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000cbb9-14ed-4475-99b2-acf3e86c7c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:52.922766 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000cbb9-14ed-4475-99b2-acf3e86c7c71 service neutron] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:52.923049 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000cbb9-14ed-4475-99b2-acf3e86c7c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:52.923329 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000cbb9-14ed-4475-99b2-acf3e86c7c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:52.925027 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-2000cbb9-14ed-4475-99b2-acf3e86c7c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:53.000054 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:53.044186 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000cbb9-14ed-4475-99b2-acf3e86c7c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:53.045669 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000cbb9-14ed-4475-99b2-acf3e86c7c71 service neutron] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:53.064522 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 662/1323] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:57:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:57:55.370895 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773bde0-9eb0-4b38-8410-b38e6a2e434e None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:55.371434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773bde0-9eb0-4b38-8410-b38e6a2e434e None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:55.371434 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773bde0-9eb0-4b38-8410-b38e6a2e434e None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:55.372872 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 662/1324] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:55.378241 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dd8f0-4cb6-4576-b2b2-322725a2b476 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:55.378656 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dd8f0-4cb6-4576-b2b2-322725a2b476 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:55.378835 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dd8f0-4cb6-4576-b2b2-322725a2b476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:55.448947 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-bd7dd8f0-4cb6-4576-b2b2-322725a2b476 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:55.450096 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-bd7dd8f0-4cb6-4576-b2b2-322725a2b476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:55.549757 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 663/1325] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:57:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:57:55.585522 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773bde0-9eb0-4b38-8410-b38e6a2e434e None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:55.587084 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-1773bde0-9eb0-4b38-8410-b38e6a2e434e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:55.693381 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773bde0-9eb0-4b38-8410-b38e6a2e434e None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:55.766200 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27082381-158d-4201-b5ab-67b5bcf1f341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:55.766762 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082381-158d-4201-b5ab-67b5bcf1f341 service neutron] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:55.767047 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082381-158d-4201-b5ab-67b5bcf1f341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:55.767324 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082381-158d-4201-b5ab-67b5bcf1f341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:55.769010 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-27082381-158d-4201-b5ab-67b5bcf1f341 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:55.785356 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:55.877791 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27082381-158d-4201-b5ab-67b5bcf1f341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:55.879345 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27082381-158d-4201-b5ab-67b5bcf1f341 service neutron] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:55.897629 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 663/1326] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:57:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:57:58.180233 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636908f-88f0-43af-b569-72d7e38a3166 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.180786 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636908f-88f0-43af-b569-72d7e38a3166 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.180903 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636908f-88f0-43af-b569-72d7e38a3166 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.182222 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 664/1327] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:58.187661 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d3095-35f9-44f2-a040-02da4388946f None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.188101 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d3095-35f9-44f2-a040-02da4388946f None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.188233 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d3095-35f9-44f2-a040-02da4388946f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.258606 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-200d3095-35f9-44f2-a040-02da4388946f None None] MFA Rules not processed for user `c0ff018c6850486aa01127bb8c1bd8b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:58.259822 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-200d3095-35f9-44f2-a040-02da4388946f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:58.324441 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:58.359596 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 664/1328] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:57:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:57:58.409662 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81dc2e-9b7e-4df9-af32-042db9a36564 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.409956 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81dc2e-9b7e-4df9-af32-042db9a36564 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.410236 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81dc2e-9b7e-4df9-af32-042db9a36564 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.412202 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 665/1329] 10.222.0.88 () {54 vars in 964 bytes} [Thu May 19 19:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:57:58.418258 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687176e-66c7-4e53-9fda-65571554339d None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.418539 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687176e-66c7-4e53-9fda-65571554339d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.418819 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2687176e-66c7-4e53-9fda-65571554339d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.494860 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-2687176e-66c7-4e53-9fda-65571554339d None None] MFA Rules not processed for user `6d5a01edfe0444f8a6fef9d7ee9645f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:57:58.496063 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-2687176e-66c7-4e53-9fda-65571554339d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:58.579780 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 665/1330] 10.222.0.88 () {58 vars in 1061 bytes} [Thu May 19 19:57:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:57:58.589651 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81dc2e-9b7e-4df9-af32-042db9a36564 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:57:58.591226 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-da81dc2e-9b7e-4df9-af32-042db9a36564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:58.623745 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:58.701939 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81dc2e-9b7e-4df9-af32-042db9a36564 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:57:58.776659 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bdde6-e507-4b35-8616-994f97da6ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:57:58.777304 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bdde6-e507-4b35-8616-994f97da6ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:57:58.777591 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bdde6-e507-4b35-8616-994f97da6ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:57:58.777867 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bdde6-e507-4b35-8616-994f97da6ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:57:58.779524 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-d06bdde6-e507-4b35-8616-994f97da6ac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:57:58.856583 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:57:58.888956 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bdde6-e507-4b35-8616-994f97da6ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:57:58.890330 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bdde6-e507-4b35-8616-994f97da6ac7 service neutron] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:57:58.908166 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 666/1331] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:57:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:58:01.975045 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cc8f3-78fe-49a6-b1b5-7e360afb2a5f None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:01.975568 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cc8f3-78fe-49a6-b1b5-7e360afb2a5f None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:01.975678 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cc8f3-78fe-49a6-b1b5-7e360afb2a5f None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:01.977020 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 666/1332] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:58:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:01.982632 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e8874-88f7-487d-9517-eb4fd7886655 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:01.983076 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e8874-88f7-487d-9517-eb4fd7886655 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:01.983167 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e8874-88f7-487d-9517-eb4fd7886655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:02.052716 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-8f0e8874-88f7-487d-9517-eb4fd7886655 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:02.053915 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-8f0e8874-88f7-487d-9517-eb4fd7886655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:02.170343 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 667/1333] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:02.206965 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cc8f3-78fe-49a6-b1b5-7e360afb2a5f None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:02.209467 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-816cc8f3-78fe-49a6-b1b5-7e360afb2a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:02.289149 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:02.326497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cc8f3-78fe-49a6-b1b5-7e360afb2a5f None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:02.411043 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83219918-7843-4f14-afdf-b60241b829d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:02.411693 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83219918-7843-4f14-afdf-b60241b829d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:02.411973 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83219918-7843-4f14-afdf-b60241b829d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:02.412370 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83219918-7843-4f14-afdf-b60241b829d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:02.413993 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-83219918-7843-4f14-afdf-b60241b829d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:02.545173 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83219918-7843-4f14-afdf-b60241b829d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:02.546624 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83219918-7843-4f14-afdf-b60241b829d9 service neutron] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:02.557877 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:02.566894 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 667/1334] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:58:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:58:04.835143 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99512f-7447-48df-b6b1-e4a6eb35ddf4 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.835676 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99512f-7447-48df-b6b1-e4a6eb35ddf4 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.835676 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d99512f-7447-48df-b6b1-e4a6eb35ddf4 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.837182 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 668/1335] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:04.842399 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e70ed-e4c2-4d7e-9dd9-4a86141d0172 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:04.842783 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e70ed-e4c2-4d7e-9dd9-4a86141d0172 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:04.842969 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e70ed-e4c2-4d7e-9dd9-4a86141d0172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:04.906322 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-637e70ed-e4c2-4d7e-9dd9-4a86141d0172 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:04.907520 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-637e70ed-e4c2-4d7e-9dd9-4a86141d0172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:05.001866 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 668/1336] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:05.041816 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99512f-7447-48df-b6b1-e4a6eb35ddf4 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:05.043351 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-9d99512f-7447-48df-b6b1-e4a6eb35ddf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:05.083288 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:05.148410 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d99512f-7447-48df-b6b1-e4a6eb35ddf4 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:05.225914 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405f2d1-bc98-4ed7-9fd0-d53865ecae2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:05.226553 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405f2d1-bc98-4ed7-9fd0-d53865ecae2c service neutron] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:05.226828 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405f2d1-bc98-4ed7-9fd0-d53865ecae2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:05.227098 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405f2d1-bc98-4ed7-9fd0-d53865ecae2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:05.228771 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-b405f2d1-bc98-4ed7-9fd0-d53865ecae2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:05.310351 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:05.337396 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405f2d1-bc98-4ed7-9fd0-d53865ecae2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:05.338891 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405f2d1-bc98-4ed7-9fd0-d53865ecae2c service neutron] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:05.357646 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 669/1337] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:58:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:58:08.255523 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a8501-9d8f-4d7f-b9be-b1b2b62e6ae0 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.256072 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a8501-9d8f-4d7f-b9be-b1b2b62e6ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.256072 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a8501-9d8f-4d7f-b9be-b1b2b62e6ae0 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.257503 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 669/1338] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:08.262608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38871e-d6e0-4fcf-aa60-563ca1425d88 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.262999 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38871e-d6e0-4fcf-aa60-563ca1425d88 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.263163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38871e-d6e0-4fcf-aa60-563ca1425d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.328088 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-ec38871e-d6e0-4fcf-aa60-563ca1425d88 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:08.329310 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-ec38871e-d6e0-4fcf-aa60-563ca1425d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:08.425147 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 670/1339] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:08.461399 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a8501-9d8f-4d7f-b9be-b1b2b62e6ae0 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:08.464853 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-195a8501-9d8f-4d7f-b9be-b1b2b62e6ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:08.544202 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:08.577116 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a8501-9d8f-4d7f-b9be-b1b2b62e6ae0 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:08.581327 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c556d8-7097-4ed7-a164-cd5470f6e0aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:08.581976 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c556d8-7097-4ed7-a164-cd5470f6e0aa service neutron] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:08.582803 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c556d8-7097-4ed7-a164-cd5470f6e0aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:08.583082 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c556d8-7097-4ed7-a164-cd5470f6e0aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:08.587423 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-35c556d8-7097-4ed7-a164-cd5470f6e0aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:08.686748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c556d8-7097-4ed7-a164-cd5470f6e0aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:08.688103 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c556d8-7097-4ed7-a164-cd5470f6e0aa service neutron] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:08.708780 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 670/1340] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:58:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:58:13.201995 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01c527-6850-4211-a66f-9ae58800fc43 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:13.202595 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01c527-6850-4211-a66f-9ae58800fc43 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:13.202595 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01c527-6850-4211-a66f-9ae58800fc43 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:13.203914 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 671/1341] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:13.209857 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5411e-564b-4e01-84a3-d95c3262c90e None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:13.210318 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5411e-564b-4e01-84a3-d95c3262c90e None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:13.210426 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5411e-564b-4e01-84a3-d95c3262c90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:13.257533 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-fcb5411e-564b-4e01-84a3-d95c3262c90e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:13.258748 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-fcb5411e-564b-4e01-84a3-d95c3262c90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:13.317681 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=100 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:13.339205 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 671/1342] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:58:13.376282 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01c527-6850-4211-a66f-9ae58800fc43 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:13.388050 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01c527-6850-4211-a66f-9ae58800fc43 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:13.391552 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffa89f-a7fe-4c9a-b6cf-000ced213182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:13.392186 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffa89f-a7fe-4c9a-b6cf-000ced213182 service neutron] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:13.392499 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffa89f-a7fe-4c9a-b6cf-000ced213182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:13.392778 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffa89f-a7fe-4c9a-b6cf-000ced213182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:13.402326 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffa89f-a7fe-4c9a-b6cf-000ced213182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:13.403768 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffa89f-a7fe-4c9a-b6cf-000ced213182 service neutron] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:13.414017 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 672/1343] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:58:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:58:16.594163 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad56fb-84c5-4ddb-9972-51bb5253098a None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:16.594713 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad56fb-84c5-4ddb-9972-51bb5253098a None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:16.594713 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad56fb-84c5-4ddb-9972-51bb5253098a None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:16.596041 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 672/1344] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:16.602045 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff4cda-673a-4fa0-b6a2-be81b93fa6ca None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:16.602491 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff4cda-673a-4fa0-b6a2-be81b93fa6ca None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:16.602604 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff4cda-673a-4fa0-b6a2-be81b93fa6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:16.630424 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-36ff4cda-673a-4fa0-b6a2-be81b93fa6ca None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:16.631602 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-36ff4cda-673a-4fa0-b6a2-be81b93fa6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:16.639478 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 673/1345] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:58:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:16.675002 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad56fb-84c5-4ddb-9972-51bb5253098a None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:16.686387 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad56fb-84c5-4ddb-9972-51bb5253098a None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:16.689850 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35030c-188c-4ba2-be0f-1e13f54ded95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:16.690476 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35030c-188c-4ba2-be0f-1e13f54ded95 service neutron] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:16.690753 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35030c-188c-4ba2-be0f-1e13f54ded95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:16.691023 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35030c-188c-4ba2-be0f-1e13f54ded95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:16.699915 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35030c-188c-4ba2-be0f-1e13f54ded95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:16.701372 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35030c-188c-4ba2-be0f-1e13f54ded95 service neutron] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:16.710595 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 673/1346] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:58:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:58:21.174455 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485df24-411f-4125-96eb-76dd42f648aa None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:21.175059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485df24-411f-4125-96eb-76dd42f648aa None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:21.175059 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485df24-411f-4125-96eb-76dd42f648aa None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:21.176736 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 674/1347] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:21.184470 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f4ca5-32a7-47e7-9f1c-b9d7562b6d08 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:21.184470 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f4ca5-32a7-47e7-9f1c-b9d7562b6d08 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:21.184470 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f4ca5-32a7-47e7-9f1c-b9d7562b6d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:21.202829 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-d19f4ca5-32a7-47e7-9f1c-b9d7562b6d08 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:21.204092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-d19f4ca5-32a7-47e7-9f1c-b9d7562b6d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:21.215024 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 674/1348] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 19:58:21.254897 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485df24-411f-4125-96eb-76dd42f648aa None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:21.267043 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485df24-411f-4125-96eb-76dd42f648aa None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:21.271283 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb36c9-fc7d-4291-ab2d-4039fff41327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d5a01edfe0444f8a6fef9d7ee9645f3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:21.271937 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb36c9-fc7d-4291-ab2d-4039fff41327 service neutron] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:21.272223 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb36c9-fc7d-4291-ab2d-4039fff41327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:21.272538 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb36c9-fc7d-4291-ab2d-4039fff41327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:21.282660 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb36c9-fc7d-4291-ab2d-4039fff41327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:21.284129 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb36c9-fc7d-4291-ab2d-4039fff41327 service neutron] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:21.295185 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 675/1349] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:58:27.010878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60141b-2eb1-40d3-8e93-e66b449c0bd5 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:27.011412 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60141b-2eb1-40d3-8e93-e66b449c0bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:27.011412 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b60141b-2eb1-40d3-8e93-e66b449c0bd5 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:27.012841 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 675/1350] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 19:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:27.018325 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae496d69-c7df-45c2-b540-2c455d8f36df None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:27.019460 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae496d69-c7df-45c2-b540-2c455d8f36df None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:27.019961 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae496d69-c7df-45c2-b540-2c455d8f36df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:27.024401 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:27.103708 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-ae496d69-c7df-45c2-b540-2c455d8f36df None None] MFA Rules not processed for user `28997ce3cd2e41feaf22b9742ba5e510`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:27.105472 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-ae496d69-c7df-45c2-b540-2c455d8f36df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:27.197473 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 676/1351] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 19:58:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:31.197121 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=15 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:31.491444 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf782d-e285-4ad3-8fa1-52c384299573 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:31.491741 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf782d-e285-4ad3-8fa1-52c384299573 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:31.492201 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf782d-e285-4ad3-8fa1-52c384299573 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:31.493675 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 676/1352] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 19:58:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:31.500035 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d8be6-6d65-4a1b-b9bd-de06ed6976db None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:31.500582 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d8be6-6d65-4a1b-b9bd-de06ed6976db None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:31.500811 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d8be6-6d65-4a1b-b9bd-de06ed6976db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:31.524887 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-c38d8be6-6d65-4a1b-b9bd-de06ed6976db None None] MFA Rules not processed for user `28997ce3cd2e41feaf22b9742ba5e510`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:31.526434 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-c38d8be6-6d65-4a1b-b9bd-de06ed6976db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:31.536706 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 677/1353] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 19:58:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:33.056123 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119cb0c-816e-4ab2-a959-62faa0a4df53 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:33.056719 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119cb0c-816e-4ab2-a959-62faa0a4df53 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:33.056719 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119cb0c-816e-4ab2-a959-62faa0a4df53 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:33.058106 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 677/1354] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 19:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:33.063187 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7bdfd-c1fa-49e6-af8e-8421b6417927 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:33.063580 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7bdfd-c1fa-49e6-af8e-8421b6417927 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:33.063751 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7bdfd-c1fa-49e6-af8e-8421b6417927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:33.083163 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-4fb7bdfd-c1fa-49e6-af8e-8421b6417927 None None] MFA Rules not processed for user `28997ce3cd2e41feaf22b9742ba5e510`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:33.084717 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-4fb7bdfd-c1fa-49e6-af8e-8421b6417927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:33.093592 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 678/1355] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 19:58:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:37.093771 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976506b6-89ea-42c4-a3f6-90b76eeafcdf None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:37.094298 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976506b6-89ea-42c4-a3f6-90b76eeafcdf None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:37.094514 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976506b6-89ea-42c4-a3f6-90b76eeafcdf None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:37.096132 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 678/1356] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:37.101548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7b669-9dca-48ea-8cfb-4b8e5a887ce9 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:37.102347 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7b669-9dca-48ea-8cfb-4b8e5a887ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:37.102633 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7b669-9dca-48ea-8cfb-4b8e5a887ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:37.121594 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-edc7b669-9dca-48ea-8cfb-4b8e5a887ce9 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:37.122917 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-edc7b669-9dca-48ea-8cfb-4b8e5a887ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:37.132449 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 679/1357] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:37.192799 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104098e-1542-4974-8b75-6a30088bbcc1 None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:37.194081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104098e-1542-4974-8b75-6a30088bbcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:37.194672 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104098e-1542-4974-8b75-6a30088bbcc1 None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:37.196775 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 679/1358] 10.222.0.88 () {54 vars in 961 bytes} [Thu May 19 19:58:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 19:58:37.211091 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e69eb-57c5-4477-9974-51dee044ba2b None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:37.218148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e69eb-57c5-4477-9974-51dee044ba2b None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:37.218148 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e69eb-57c5-4477-9974-51dee044ba2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:37.270974 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-7e2e69eb-57c5-4477-9974-51dee044ba2b None None] MFA Rules not processed for user `0c0e71329c4a4470935593636c025257`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:37.272531 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-7e2e69eb-57c5-4477-9974-51dee044ba2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:37.332834 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=100 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:37.369955 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 680/1359] 10.222.0.88 () {58 vars in 1058 bytes} [Thu May 19 19:58:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:37.378964 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104098e-1542-4974-8b75-6a30088bbcc1 None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:37.396631 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104098e-1542-4974-8b75-6a30088bbcc1 None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:37.403464 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992970b-ca8a-4d16-b4f4-a824b80be1fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c0e71329c4a4470935593636c025257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:37.404398 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992970b-ca8a-4d16-b4f4-a824b80be1fb service nova] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:37.404838 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992970b-ca8a-4d16-b4f4-a824b80be1fb service nova] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:37.405278 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992970b-ca8a-4d16-b4f4-a824b80be1fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:37.416563 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992970b-ca8a-4d16-b4f4-a824b80be1fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:37.424319 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992970b-ca8a-4d16-b4f4-a824b80be1fb service nova] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:37.431712 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 680/1360] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:58:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:58:39.907249 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e2cd0-cf16-4669-a842-fc3409e5fdde None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:39.907825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e2cd0-cf16-4669-a842-fc3409e5fdde None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:39.907825 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e2cd0-cf16-4669-a842-fc3409e5fdde None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:39.909277 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 681/1361] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:39.915504 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efcae7-0cb9-45a6-ac9f-64e9cbe1e58f None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:39.915986 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efcae7-0cb9-45a6-ac9f-64e9cbe1e58f None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:39.916086 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efcae7-0cb9-45a6-ac9f-64e9cbe1e58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:39.934878 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-b8efcae7-0cb9-45a6-ac9f-64e9cbe1e58f None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:39.936092 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-b8efcae7-0cb9-45a6-ac9f-64e9cbe1e58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:39.944985 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 681/1362] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:39.989257 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e2cd0-cf16-4669-a842-fc3409e5fdde None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:40.007005 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e2cd0-cf16-4669-a842-fc3409e5fdde None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:40.012796 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcad9c-103d-4fab-9ea2-831eeeaf1cad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c0e71329c4a4470935593636c025257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:40.013395 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcad9c-103d-4fab-9ea2-831eeeaf1cad service nova] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:40.013680 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcad9c-103d-4fab-9ea2-831eeeaf1cad service nova] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:40.013959 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcad9c-103d-4fab-9ea2-831eeeaf1cad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:40.023555 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcad9c-103d-4fab-9ea2-831eeeaf1cad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:40.025286 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcad9c-103d-4fab-9ea2-831eeeaf1cad service nova] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:40.036151 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 682/1363] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:58:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:58:42.733081 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ce596-8107-4bf0-b556-609ebfc60b4d None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:42.733671 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ce596-8107-4bf0-b556-609ebfc60b4d None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:42.733671 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ce596-8107-4bf0-b556-609ebfc60b4d None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:42.734995 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 682/1364] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:42.741195 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f334-6d41-4682-9667-7eb637b04b70 None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:42.741994 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f334-6d41-4682-9667-7eb637b04b70 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:42.742430 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3f334-6d41-4682-9667-7eb637b04b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:42.760945 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-eec3f334-6d41-4682-9667-7eb637b04b70 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:42.762458 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-eec3f334-6d41-4682-9667-7eb637b04b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:42.770488 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 683/1365] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:42.817767 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae331c2-94a3-4903-8373-0f35ce55b08c None None] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:42.818082 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae331c2-94a3-4903-8373-0f35ce55b08c None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:42.820213 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae331c2-94a3-4903-8373-0f35ce55b08c None None] PATH_INFO: `/` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:42.820213 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 683/1366] 10.222.0.88 () {54 vars in 961 bytes} [Thu May 19 19:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:42.825970 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5deff1-badc-4111-924e-78446551d22c None None] REQUEST_METHOD: `POST` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:42.826729 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5deff1-badc-4111-924e-78446551d22c None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:42.827014 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5deff1-badc-4111-924e-78446551d22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:42.846860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.auth.core [None req-5d5deff1-badc-4111-924e-78446551d22c None None] MFA Rules not processed for user `0c0e71329c4a4470935593636c025257`. Rule list: `[]` (Enabled: `True`). {{(pid=80453) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:42.848601 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.fernet_utils [None req-5d5deff1-badc-4111-924e-78446551d22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80453) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:42.859151 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 684/1367] 10.222.0.88 () {58 vars in 1058 bytes} [Thu May 19 19:58:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:42.868574 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae331c2-94a3-4903-8373-0f35ce55b08c None None] Authenticating user token {{(pid=80452) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:42.884682 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae331c2-94a3-4903-8373-0f35ce55b08c None None] Validating token access rules against request {{(pid=80452) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:42.890855 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac97448-0ff9-41b3-99cc-05d99472b0eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c0e71329c4a4470935593636c025257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80452) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:42.891530 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97448-0ff9-41b3-99cc-05d99472b0eb service nova] REQUEST_METHOD: `GET` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:42.892038 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97448-0ff9-41b3-99cc-05d99472b0eb service nova] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:42.892321 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97448-0ff9-41b3-99cc-05d99472b0eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:42.902239 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac97448-0ff9-41b3-99cc-05d99472b0eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:42.903868 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac97448-0ff9-41b3-99cc-05d99472b0eb service nova] RBAC: Authorization granted {{(pid=80452) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:42.915490 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 684/1368] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:58:43.663296 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b3876-615d-48ff-ba90-ff6a9cac4ddf None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:43.663643 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b3876-615d-48ff-ba90-ff6a9cac4ddf None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:43.663860 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b3876-615d-48ff-ba90-ff6a9cac4ddf None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:43.665548 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 685/1369] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:43.671338 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8bd3-c9a3-43c3-8241-ecc2cf4129b4 None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:43.671891 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8bd3-c9a3-43c3-8241-ecc2cf4129b4 None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:43.672351 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8bd3-c9a3-43c3-8241-ecc2cf4129b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:43.693524 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-6a7e8bd3-c9a3-43c3-8241-ecc2cf4129b4 None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:43.695104 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-6a7e8bd3-c9a3-43c3-8241-ecc2cf4129b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:43.707681 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 685/1370] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:45.362304 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600be103-b2a3-40b3-8a8c-4f2fa23a2722 None None] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:45.362807 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600be103-b2a3-40b3-8a8c-4f2fa23a2722 None None] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:45.362921 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600be103-b2a3-40b3-8a8c-4f2fa23a2722 None None] PATH_INFO: `/` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:45.364193 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 686/1371] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 19:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 19:58:45.369530 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d71047-24db-41dc-95a4-13572b08424e None None] REQUEST_METHOD: `POST` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:45.369953 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d71047-24db-41dc-95a4-13572b08424e None None] SCRIPT_NAME: `/identity` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:45.370098 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d71047-24db-41dc-95a4-13572b08424e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80452) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:45.388242 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.auth.core [None req-e3d71047-24db-41dc-95a4-13572b08424e None None] MFA Rules not processed for user `3f3a6a2da4254702bc7bd15361b6c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80452) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 19:58:45.389497 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG keystone.common.fernet_utils [None req-e3d71047-24db-41dc-95a4-13572b08424e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80452) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 19:58:45.398066 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: [pid: 80452|app: 0|req: 686/1372] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 19:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 19:58:45.442684 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600be103-b2a3-40b3-8a8c-4f2fa23a2722 None None] Authenticating user token {{(pid=80453) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 19:58:45.457718 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600be103-b2a3-40b3-8a8c-4f2fa23a2722 None None] Validating token access rules against request {{(pid=80453) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 19:58:45.463245 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59d41c-de21-432b-8204-8e9e187ad495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c0e71329c4a4470935593636c025257', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b14a16a022894289a30e8697bef1fcf0', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80453) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 19:58:45.463906 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59d41c-de21-432b-8204-8e9e187ad495 service nova] REQUEST_METHOD: `GET` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 19:58:45.464196 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59d41c-de21-432b-8204-8e9e187ad495 service nova] SCRIPT_NAME: `/identity` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 19:58:45.464521 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59d41c-de21-432b-8204-8e9e187ad495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80453) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 19:58:45.473992 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59d41c-de21-432b-8204-8e9e187ad495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 19:58:45.475608 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59d41c-de21-432b-8204-8e9e187ad495 service nova] RBAC: Authorization granted {{(pid=80453) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 19:58:45.486021 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: [pid: 80453|app: 0|req: 687/1373] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 19:58:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 19:58:55.382608 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: DEBUG dbcounter [-] [80452] Writing DB stats keystone:SELECT=21 {{(pid=80452) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:58:55.481428 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: DEBUG dbcounter [-] [80453] Writing DB stats keystone:SELECT=27 {{(pid=80453) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 19:59:07.276895 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: Thu May 19 19:59:07 2022 - graceful shutdown triggered... May 19 19:59:07.277357 dv-cinder-841302-4-iscsi devstack@keystone.service[80452]: Gracefully killing worker 1 (pid: 80452)... May 19 19:59:07.277721 dv-cinder-841302-4-iscsi devstack@keystone.service[80453]: Gracefully killing worker 2 (pid: 80453)... May 19 19:59:07.280276 dv-cinder-841302-4-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 19:59:08.278049 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: worker 1 buried after 1 seconds May 19 19:59:08.278049 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: worker 2 buried after 1 seconds May 19 19:59:08.278049 dv-cinder-841302-4-iscsi devstack@keystone.service[80441]: goodbye to uWSGI. May 19 19:59:08.279640 dv-cinder-841302-4-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 19:59:08.280197 dv-cinder-841302-4-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 20:05:48.983825 dv-cinder-841302-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 20:05:48.992048 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 20:05:48.994138 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 20:05:48 2022] *** May 19 20:05:48.994138 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 20:05:48.994138 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 20:05:48.994138 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: nodename: dv-cinder-841302-4-iscsi May 19 20:05:48.994138 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: machine: x86_64 May 19 20:05:48.994138 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: clock source: unix May 19 20:05:48.994138 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: pcre jit disabled May 19 20:05:48.994717 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: detected number of CPU cores: 4 May 19 20:05:48.994717 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: current working directory: / May 19 20:05:48.994717 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: detected binary path: /usr/bin/uwsgi-core May 19 20:05:48.994717 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: your processes number limit is 47680 May 19 20:05:48.994717 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: your memory page size is 4096 bytes May 19 20:05:48.994717 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: detected max file descriptor number: 2048 May 19 20:05:48.994717 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: lock engine: pthread robust mutexes May 19 20:05:48.994717 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: thunder lock: enabled May 19 20:05:48.994717 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 20:05:48.994717 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 20:05:49.042499 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: Python main interpreter initialized at 0x55c3a5b079c0 May 19 20:05:49.042499 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: python threads support enabled May 19 20:05:49.042499 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: your server socket listen backlog is limited to 100 connections May 19 20:05:49.042499 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: your mercy for graceful operations on workers is 90 seconds May 19 20:05:49.042966 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: mapped 403077 bytes (393 KB) for 2 cores May 19 20:05:49.042966 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: *** Operational MODE: preforking *** May 19 20:05:49.043274 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: *** uWSGI is running in multiple interpreter mode *** May 19 20:05:49.043274 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: spawned uWSGI master process (pid: 144609) May 19 20:05:49.043455 dv-cinder-841302-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 20:05:49.048360 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: spawned uWSGI worker 1 (pid: 144623, cores: 1) May 19 20:05:49.048360 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: spawned uWSGI worker 2 (pid: 144624, cores: 1) May 19 20:05:49.048360 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 20:05:54.889787 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:05:54.889787 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: from cryptography.utils import int_from_bytes May 19 20:05:54.890718 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:05:54.890718 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: from cryptography.utils import int_from_bytes May 19 20:05:54.899878 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:05:54.899878 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: from cryptography.utils import int_from_bytes May 19 20:05:54.900524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:05:54.900524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: from cryptography.utils import int_from_bytes May 19 20:05:54.941098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:05:54.941338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:05:54.941466 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] command line args: [] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:05:54.941745 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:05:54.941925 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ================================================================================ {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:05:54.942194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] admin_token = **** {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.942443 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] config_dir = [] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.942631 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.942812 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] config_source = [] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.942991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] debug = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.943191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.944596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.944596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] insecure_debug = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.944596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.944596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.944596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] list_limit = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.945253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] log_config_append = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.945253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.945253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] log_dir = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.945253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] log_file = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.945666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.945666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.945862 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.946069 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.946248 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.946412 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.946656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.946902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.947136 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.947369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.947599 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.947829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.948077 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.948257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:05:54.948382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] notification_format = cadf {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.948471 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:05:54.948611 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.948712 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] command line args: [] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:05:54.948914 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:05:54.949092 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ================================================================================ {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:05:54.949200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.949293 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] publish_errors = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.949400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] admin_token = **** {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.949496 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.949607 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] config_dir = [] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.949717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.949810 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.949920 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.950013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] config_source = [] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.950104 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] debug = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.950207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.950300 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.950430 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.950531 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.950637 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] standard_threads = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.950730 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] insecure_debug = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.950863 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] strict_password_check = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.950970 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.951109 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.951209 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.951296 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] use_eventlog = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.951396 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] list_limit = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.951497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] use_journal = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.951589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] log_config_append = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.951693 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] use_json = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.951792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.951884 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] use_stderr = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.951975 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] log_dir = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.952074 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] use_syslog = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.952172 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] log_file = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.952276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] watch_log_file = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.952401 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.952497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.952589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.952688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.952774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.952913 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.953007 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.953131 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.953231 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.953338 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.953445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.953588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.953683 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.953846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.953941 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.954083 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.954203 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.954315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.954422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.954578 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.954688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.954849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.954949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.955042 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.955197 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.955294 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] notification_format = cadf {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.955392 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.955493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.955588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.955833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.956030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.956140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.956234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] publish_errors = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.956328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.956462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.956582 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.956697 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.956838 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.956939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.957080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.957186 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.957315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.957415 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.957550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] standard_threads = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.957650 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.957782 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] strict_password_check = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.957883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.958025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.958124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.958227 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] use_eventlog = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.958314 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.958411 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] use_journal = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.958527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.958635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] use_json = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.958721 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] use_stderr = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.958811 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.958904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] use_syslog = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.958999 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.959096 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] watch_log_file = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:05:54.959187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.959324 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.959439 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.959551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.959692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.959797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.959924 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.960023 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.960143 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.960277 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.960403 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.960557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.960671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.960809 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.960915 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.961046 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.961158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.961295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.961389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.961543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.961641 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.961778 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.961878 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.962011 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.962126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.962249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.962341 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.962481 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.962589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.962714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.962814 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.962952 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.963051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.963187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.963287 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.963421 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.963518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.963658 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.963765 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.963890 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.963997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.964126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.964226 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.964409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.964513 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.964634 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.964741 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.964865 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.964965 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.965103 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.965196 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.965339 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.965446 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.965586 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.965692 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.965817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.965922 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.966050 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.966156 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.966284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.966378 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.966532 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.966678 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.966786 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.966887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.966975 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.967080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.967216 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.967311 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.967465 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.967579 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.967683 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.967778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.967920 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.968034 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.968157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.968257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.968421 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.968529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.968660 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.968767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.969454 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.969636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.969751 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.969855 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.969968 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.970058 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.970181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.970285 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.970420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.970526 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.970670 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.970772 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.970902 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.971059 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.enabled = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.971161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.971305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.971409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.971546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.971648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.971791 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.971899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.972029 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.972123 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.972256 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.972401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.972516 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.972625 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.972748 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.973114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.973114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.973274 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.973274 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.973448 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.973547 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.973649 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.973741 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.973835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.973928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.974037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.974136 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.974237 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.974342 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.974446 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.974547 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.974644 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.974796 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.974898 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.975032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.975128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.975263 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.975357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.975498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.975598 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.975733 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.975844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.975965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.976060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.976207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.976301 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.976469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.976586 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.976720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.976871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.976970 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.977073 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.977185 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.977274 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.977399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.977507 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.977651 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.enabled = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.977762 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.977891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.977999 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] auth.external = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.978120 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] auth.mapped = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.978223 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.978361 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.978522 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.978654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.978763 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.978870 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.978965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] auth.password = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.979071 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.979157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] auth.token = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.979263 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.979350 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.979470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.979591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] catalog.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.979732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.979832 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.979974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.980078 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.980211 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.980318 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.980477 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.980587 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.980712 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.980810 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.980948 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.981050 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] credential.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.981185 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.981279 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] credential.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.981420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.981535 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.981636 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.981737 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.981869 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.981965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.982098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.982207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.982333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.982435 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.982568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.982663 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.982807 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.982920 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.983043 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.983138 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.983291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.983397 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.983539 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.983655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.983784 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.983885 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.984043 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.984143 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.984283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.985226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.985365 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] auth.external = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.985501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.985610 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] auth.mapped = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.985700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] federation.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.985700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.985881 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.985977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] federation.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.986081 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.986170 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.986263 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] auth.password = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.986351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.986451 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] auth.token = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.986589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.986703 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.986828 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.986929 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] catalog.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.987081 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.987189 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.987318 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.987419 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.987574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.987668 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.987810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.987912 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.988055 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.988149 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.988291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.988419 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] credential.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.988568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] credential.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.988663 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.988817 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.988930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.989039 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.989141 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.989274 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.989384 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.989523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.989628 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.989752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.989857 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.989991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.990101 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.990240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.990346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.990490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.990585 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.990755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.990844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.990944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.991129 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.991256 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.991354 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.991452 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.991551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.991645 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.991739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.991842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.991979 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.992090 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.992229 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] federation.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.992351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.992496 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.992613 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.992745 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] federation.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.992858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.992995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.993107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.993228 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.993323 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.993478 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.993583 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.993719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.993834 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.993951 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.994051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.994192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.994295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.994431 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.994524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.994671 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.994771 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.994898 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.995004 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.995144 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.995236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.995388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.995494 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.995622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.995727 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.995857 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.995952 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.996109 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.996206 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.996410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.996502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.996588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.996673 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.996785 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.password = **** {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.996899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.997021 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.997128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.997257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.997351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.997500 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.997594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.997733 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.997838 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.997965 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.998059 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.998210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.998303 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.998456 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.998550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.998697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.998811 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.998900 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.998995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.999150 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.999251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.999397 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.999508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.999634 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.999743 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.999871 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:54.999975 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.000104 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.000221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.000388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.000485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.000626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.000729 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.000864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.000971 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.001103 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.001206 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.001340 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.001441 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.001584 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.001677 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.001813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.001912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.002051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.002153 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.002281 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.002383 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.002516 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.002610 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.002755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.002859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.002988 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.003083 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.003216 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.003320 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.003455 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.003563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.003685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.password = **** {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.003779 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.003922 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.004028 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.004159 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.004255 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.004423 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.004531 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.004656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.004752 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.004890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.004993 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.005123 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.005228 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.005357 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.005451 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.005590 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.005691 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.005820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.005931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.006064 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.006157 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.006307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.006414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.006542 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.006638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.006786 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.006884 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] policy.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.007012 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.007119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.007244 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.007351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.007476 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.007590 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.007713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] receipt.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.007812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.007995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.008145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.008318 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.008496 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.008643 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.008788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.008945 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.009104 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.009256 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.009407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.009565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.009712 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] resource.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.009859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.010012 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.010164 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.010319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] resource.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.010470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.010615 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.010774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.010926 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.011072 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.011225 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.011370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.011512 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] revoke.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.011659 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.011799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.011951 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.012106 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.012251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.012425 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] role.cache_time = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.012576 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.012723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] role.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.012866 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.013025 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] role.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.013171 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.013327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] role.list_limit = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.013481 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.013636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.013781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.013948 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.014094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.014250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.014393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.014560 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.014706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.014859 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.015012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.015157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.015301 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] policy.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.015444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.015587 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.015689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.015824 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.015923 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.016011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.016106 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.016193 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] receipt.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.016298 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.016482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.016588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.016677 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.016767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.016879 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.016879 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.017066 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.017160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.017247 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.017339 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] resource.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.017426 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.017520 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.017622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.017723 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] resource.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.017812 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.017896 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.017990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.018086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.018173 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.018271 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.018360 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.018445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] revoke.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.018538 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.018626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.018723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.018810 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.018903 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.018990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] role.cache_time = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.019094 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.019198 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] role.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.019286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.019379 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] role.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.019466 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.019569 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] role.list_limit = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.019738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.019828 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.019914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.019999 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.020084 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.020169 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.020263 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.020367 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.020473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] token.cache_time = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.020570 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.020659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] token.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.020768 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.020888 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.020988 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.021090 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] token.provider = fernet {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.021188 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.021276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.021375 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.021471 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.021559 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.021645 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.021746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.021849 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.021937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.022035 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.022129 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.022217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.022217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] trust.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.022371 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.022470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.022573 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.022662 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.022759 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.022847 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.022932 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.023024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.023117 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.023216 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.023303 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.023389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.023474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.023641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.023750 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.023892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.023998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.024130 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.connection = **** {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.024392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.024508 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.024626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.024728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.024858 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.024960 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.025109 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.025204 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.025361 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.025453 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.025600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.025695 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.025834 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.025940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.026072 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.026176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.026303 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] token.cache_time = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.026410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.026537 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.026631 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] token.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.026769 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.026871 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.027002 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.027101 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] token.provider = fernet {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.027238 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.027345 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.027472 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.027575 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.027715 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.027826 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.027949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.028045 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.028191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.028284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.028456 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.028548 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.028634 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=144624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:05:55.028719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] trust.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.028918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.029158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.029393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.029638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.029887 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.030118 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.030361 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.030658 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.030927 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.connection = **** {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.031173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.034066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.034066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.034066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.034066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.034066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.034066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.034066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.034066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.034066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.036798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.036798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.036798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.036798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.036798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.036798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.036798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.036798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:05:55.036798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=144623) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:05:55.059563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.059793 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.060010 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.071076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.080801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.080801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.080801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.091085 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.106909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.107144 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.111717 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.111952 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.161914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.162129 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.219275 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.219514 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.219712 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.219812 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.290884 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.291114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.291234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.291345 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.291439 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.291537 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.291632 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.291718 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.291816 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.291919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.292016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.292116 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.317916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.318154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.318263 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.318348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.318423 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.318577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.318680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.318780 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.318878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.318980 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.319077 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.319206 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.319316 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.319435 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.331291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.331710 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.344954 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.345310 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.345470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.345696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.345914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.346131 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.346347 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=144624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.346532 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.346629 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.346816 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.347041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.347261 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.347476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.347692 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=144623) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:05:55.365050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.365308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.365308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:05:55.365621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:05:55.365778 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.365933 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.366072 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:05:55.366232 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:05:55.366369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.366528 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.366663 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.366808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.366944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.367143 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.367293 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.367446 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.378815 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.379110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.379373 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.379660 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.380606 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.380908 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.381177 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.381461 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.388021 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.389783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.391423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.391709 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.391992 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.392431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.392720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.392986 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.393255 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.393383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.393552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.393715 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.393873 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.394059 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.394152 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.394265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.394363 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.394551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.394813 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.395084 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.395360 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.395633 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.395907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.396183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.422589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.424247 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.424558 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.424660 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.426330 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.426624 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.430064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.430361 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.430646 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.432084 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.432416 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.432703 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.439107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.439403 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.440986 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.441292 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.444391 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.444770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.445061 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.445346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.445625 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.445903 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.446172 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.446274 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.446442 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.446562 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.446755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.447041 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.447321 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.447598 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.447869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.448138 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.466004 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.466295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.467560 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.467859 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.471257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.472726 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.475194 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.475506 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.476464 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.476779 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.480735 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.481840 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.484376 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.486270 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.488763 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.489077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.489360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.489649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.489929 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.490248 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.491093 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.491408 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.491696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.491983 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.492270 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.492655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.513214 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.513450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.513556 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.513682 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.513839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.513954 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.514046 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.514134 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.522369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.522638 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.524124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.524285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.524410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.524526 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.527516 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.527692 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.527797 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.527890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.528081 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.528199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.528384 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.528506 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.528668 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.528767 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.542439 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.542770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.542930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.543069 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.543211 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.543358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.543481 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.543574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.543670 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.544024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.544125 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.544231 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.544321 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.544445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.563959 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.564271 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.567356 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.567546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.570788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.571004 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.572457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.572574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.574135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.574267 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.574427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.574539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.574707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.574829 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.583159 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.583423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.584844 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.585034 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.588144 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.588326 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.588455 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.588652 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.588758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.588940 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.589055 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.589219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.599347 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.599575 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.599677 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.599793 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.599915 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.600010 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.607996 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=144623) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.608195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=144624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:05:55.608313 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.608441 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.608573 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.608682 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.608848 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.608963 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.609117 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.609222 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.609396 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.609493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.609717 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.609833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.609985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.610079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.610255 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.610349 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.610542 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.610638 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.610815 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.610929 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.611074 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.611169 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.611338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.611433 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.611601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=144623) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.611703 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=144624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:05:55.651190 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c3a5b079c0 pid: 144623 (default app) May 19 20:05:55.651465 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55c3a5b079c0 pid: 144624 (default app) May 19 20:05:55.655831 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d17bd-165c-41d6-8b0c-018720702f75 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:05:55.656130 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d17bd-165c-41d6-8b0c-018720702f75 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:05:55.656518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d17bd-165c-41d6-8b0c-018720702f75 None None] PATH_INFO: `/v3` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:05:55.661435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 1/1] 10.222.0.88 () {50 vars in 791 bytes} [Thu May 19 20:05:55 2022] GET /identity/v3/ => generated 255 bytes in 10 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:04.625842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720b722-8c18-4af4-8e05-4b23702e61c7 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:04.626413 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720b722-8c18-4af4-8e05-4b23702e61c7 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:04.626413 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0720b722-8c18-4af4-8e05-4b23702e61c7 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:04.630811 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 1/2] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:06:04.636190 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7483c-29c1-41d7-ac33-555fff285796 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:04.636673 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7483c-29c1-41d7-ac33-555fff285796 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:04.636788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7483c-29c1-41d7-ac33-555fff285796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:04.706175 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: INFO dbcounter [None req-e0c7483c-29c1-41d7-ac33-555fff285796 None None] Registered counter for database keystone May 19 20:06:04.727839 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG oslo_db.sqlalchemy.engines [None req-e0c7483c-29c1-41d7-ac33-555fff285796 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=144623) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:06:04.734305 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writer thread running {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:06:04.916458 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG passlib.handlers.bcrypt [None req-e0c7483c-29c1-41d7-ac33-555fff285796 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=144623) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:06:04.916812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG passlib.handlers.bcrypt [None req-e0c7483c-29c1-41d7-ac33-555fff285796 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=144623) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:06:04.983683 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-e0c7483c-29c1-41d7-ac33-555fff285796 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:04.985367 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-e0c7483c-29c1-41d7-ac33-555fff285796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:05.113687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 2/3] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 20:06:05.137030 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720b722-8c18-4af4-8e05-4b23702e61c7 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:05.206896 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: INFO dbcounter [None req-0720b722-8c18-4af4-8e05-4b23702e61c7 None None] Registered counter for database keystone May 19 20:06:05.227193 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG oslo_db.sqlalchemy.engines [None req-0720b722-8c18-4af4-8e05-4b23702e61c7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=144624) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:06:05.233644 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writer thread running {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:06:05.357047 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0720b722-8c18-4af4-8e05-4b23702e61c7 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:05.362042 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637aebc5-23bb-4d55-9786-37d7c6b67b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f2a61498743436aab0f28a2590bd946', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:05.362703 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637aebc5-23bb-4d55-9786-37d7c6b67b38 admin admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:05.362989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637aebc5-23bb-4d55-9786-37d7c6b67b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:05.363264 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637aebc5-23bb-4d55-9786-37d7c6b67b38 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:05.371618 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637aebc5-23bb-4d55-9786-37d7c6b67b38 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:05.424629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: WARNING keystone.common.rbac_enforcer.enforcer [None req-637aebc5-23bb-4d55-9786-37d7c6b67b38 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:06:05.424981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637aebc5-23bb-4d55-9786-37d7c6b67b38 admin admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:05.435000 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 2/4] 10.222.0.88 () {58 vars in 1324 bytes} [Thu May 19 20:06:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:06:05.441091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7483c-29c1-41d7-ac33-555fff285796 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:05.455830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7483c-29c1-41d7-ac33-555fff285796 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:05.460267 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0ed22-b348-41fd-a8ff-2529a12eef79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f2a61498743436aab0f28a2590bd946', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:05.461357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0ed22-b348-41fd-a8ff-2529a12eef79 admin admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:05.461645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0ed22-b348-41fd-a8ff-2529a12eef79 admin admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:05.461923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0ed22-b348-41fd-a8ff-2529a12eef79 admin admin] PATH_INFO: `/v3/projects/4f2a61498743436aab0f28a2590bd946` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:05.463445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0ed22-b348-41fd-a8ff-2529a12eef79 admin admin] RBAC: Authorizing `identity:get_project(project_id=4f2a61498743436aab0f28a2590bd946)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:05.626530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: WARNING keystone.common.rbac_enforcer.enforcer [None req-71c0ed22-b348-41fd-a8ff-2529a12eef79 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:06:05.626860 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0ed22-b348-41fd-a8ff-2529a12eef79 admin admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:05.631808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 3/5] 10.222.0.88 () {56 vars in 1207 bytes} [Thu May 19 20:06:05 2022] GET /identity/v3/projects/4f2a61498743436aab0f28a2590bd946 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:07.857887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3542f11-f158-499d-9e2e-36b6cd1810f8 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:07.858479 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3542f11-f158-499d-9e2e-36b6cd1810f8 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:07.858479 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3542f11-f158-499d-9e2e-36b6cd1810f8 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:07.863118 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 3/6] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:07.868965 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ca55f-d39b-4329-9694-7e9535308ffe None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:07.869415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ca55f-d39b-4329-9694-7e9535308ffe None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:07.869530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ca55f-d39b-4329-9694-7e9535308ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:07.890299 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-7f8ca55f-d39b-4329-9694-7e9535308ffe None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:07.892456 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-7f8ca55f-d39b-4329-9694-7e9535308ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:07.900569 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 4/7] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 20:06:07.923040 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3542f11-f158-499d-9e2e-36b6cd1810f8 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:07.935350 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3542f11-f158-499d-9e2e-36b6cd1810f8 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:07.940284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6fdb6-14d8-4bc9-84b9-7bee54277fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f2a61498743436aab0f28a2590bd946', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:07.940983 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6fdb6-14d8-4bc9-84b9-7bee54277fe9 admin admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:07.941270 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6fdb6-14d8-4bc9-84b9-7bee54277fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:07.941549 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6fdb6-14d8-4bc9-84b9-7bee54277fe9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:07.949483 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6fdb6-14d8-4bc9-84b9-7bee54277fe9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:07.951257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6fdb6-14d8-4bc9-84b9-7bee54277fe9 admin admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:07.960819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 4/8] 10.222.0.88 () {58 vars in 1324 bytes} [Thu May 19 20:06:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:06:07.966522 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ca55f-d39b-4329-9694-7e9535308ffe None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:07.975761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ca55f-d39b-4329-9694-7e9535308ffe None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:07.980577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62d6ba-ce4c-4b4e-a389-28ab805cd598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f2a61498743436aab0f28a2590bd946', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:07.981454 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62d6ba-ce4c-4b4e-a389-28ab805cd598 admin admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:07.981767 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62d6ba-ce4c-4b4e-a389-28ab805cd598 admin admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:07.982049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62d6ba-ce4c-4b4e-a389-28ab805cd598 admin admin] PATH_INFO: `/v3/users/48915a5b6f6e4fd8a53e478480a2c94c` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:07.983451 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62d6ba-ce4c-4b4e-a389-28ab805cd598 admin admin] RBAC: Authorizing `identity:get_user(user_id=48915a5b6f6e4fd8a53e478480a2c94c)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:07.985419 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62d6ba-ce4c-4b4e-a389-28ab805cd598 admin admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:07.992314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 5/9] 10.222.0.88 () {56 vars in 1198 bytes} [Thu May 19 20:06:07 2022] GET /identity/v3/users/48915a5b6f6e4fd8a53e478480a2c94c => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:10.296353 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18081e33-56ce-4de3-b255-7038949461d1 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:10.296939 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18081e33-56ce-4de3-b255-7038949461d1 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:10.296939 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18081e33-56ce-4de3-b255-7038949461d1 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:10.298486 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 5/10] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:06:10.303847 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c10cc-4275-43bd-99ba-3a10f9171951 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:10.304277 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c10cc-4275-43bd-99ba-3a10f9171951 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:10.304583 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c10cc-4275-43bd-99ba-3a10f9171951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:10.321544 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-bd2c10cc-4275-43bd-99ba-3a10f9171951 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:10.322738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-bd2c10cc-4275-43bd-99ba-3a10f9171951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:10.361266 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8b85f-7e47-4ffe-afca-4cafe4e8e9e9 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:10.361530 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8b85f-7e47-4ffe-afca-4cafe4e8e9e9 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:10.361784 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8b85f-7e47-4ffe-afca-4cafe4e8e9e9 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:10.363151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 6/11] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:06:10.368449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a59a2-81f9-46b4-b6fd-3a40e60f3f05 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:10.368738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a59a2-81f9-46b4-b6fd-3a40e60f3f05 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:10.369019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a59a2-81f9-46b4-b6fd-3a40e60f3f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:10.408307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 6/12] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:10.409789 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG passlib.handlers.bcrypt [None req-3e3a59a2-81f9-46b4-b6fd-3a40e60f3f05 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=144624) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:06:10.410220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG passlib.handlers.bcrypt [None req-3e3a59a2-81f9-46b4-b6fd-3a40e60f3f05 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=144624) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:06:10.431249 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c10cc-4275-43bd-99ba-3a10f9171951 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:10.455245 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:10.456957 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-3e3a59a2-81f9-46b4-b6fd-3a40e60f3f05 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:10.458570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-3e3a59a2-81f9-46b4-b6fd-3a40e60f3f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:10.537592 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c10cc-4275-43bd-99ba-3a10f9171951 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:10.559878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 7/13] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:10.573824 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03633ef1-e3c7-44d4-9551-715a784fbf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:10.574626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03633ef1-e3c7-44d4-9551-715a784fbf48 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:10.574918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03633ef1-e3c7-44d4-9551-715a784fbf48 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:10.575202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03633ef1-e3c7-44d4-9551-715a784fbf48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:10.582435 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a59a2-81f9-46b4-b6fd-3a40e60f3f05 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:10.586806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03633ef1-e3c7-44d4-9551-715a784fbf48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-03633ef1-e3c7-44d4-9551-715a784fbf48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:10.598236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:10.599641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:10.600871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 7/14] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:06:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:06:10.605601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03633ef1-e3c7-44d4-9551-715a784fbf48 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:10.694665 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a59a2-81f9-46b4-b6fd-3a40e60f3f05 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:10.695132 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:10.697079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:10.714486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03633ef1-e3c7-44d4-9551-715a784fbf48 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:10.732513 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8240e7-aaeb-4a5d-9c2e-ad585f4ff899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:10.733915 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8240e7-aaeb-4a5d-9c2e-ad585f4ff899 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:10.734388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8240e7-aaeb-4a5d-9c2e-ad585f4ff899 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:10.735227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8240e7-aaeb-4a5d-9c2e-ad585f4ff899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:10.750616 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1fdaa-4fcb-4384-9478-75a61c5c6b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:10.751291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1fdaa-4fcb-4384-9478-75a61c5c6b27 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:10.751579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1fdaa-4fcb-4384-9478-75a61c5c6b27 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:10.751857 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1fdaa-4fcb-4384-9478-75a61c5c6b27 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:10.752650 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1fdaa-4fcb-4384-9478-75a61c5c6b27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:10.753066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1fdaa-4fcb-4384-9478-75a61c5c6b27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:10.754489 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea1fdaa-4fcb-4384-9478-75a61c5c6b27 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:10.766810 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 8/15] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:10.772412 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1fdaa-4fcb-4384-9478-75a61c5c6b27 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:10.810238 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8240e7-aaeb-4a5d-9c2e-ad585f4ff899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:10.811781 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8240e7-aaeb-4a5d-9c2e-ad585f4ff899 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:10.871766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1fdaa-4fcb-4384-9478-75a61c5c6b27 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:10.907400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cbdde-794f-46f6-8d14-83f5386e15ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:10.908285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cbdde-794f-46f6-8d14-83f5386e15ba None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:10.908592 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cbdde-794f-46f6-8d14-83f5386e15ba None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:10.908872 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175cbdde-794f-46f6-8d14-83f5386e15ba None admin] PATH_INFO: `/v3/users/48915a5b6f6e4fd8a53e478480a2c94c` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:10.910107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175cbdde-794f-46f6-8d14-83f5386e15ba None admin] RBAC: Authorizing `identity:get_user(user_id=48915a5b6f6e4fd8a53e478480a2c94c)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:10.911589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175cbdde-794f-46f6-8d14-83f5386e15ba None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:10.913105 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 8/16] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:06:10.913253 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 9/17] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:10 2022] GET /identity/v3/users/48915a5b6f6e4fd8a53e478480a2c94c => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:10.918614 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cbdde-794f-46f6-8d14-83f5386e15ba None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:10.918768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8240e7-aaeb-4a5d-9c2e-ad585f4ff899 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:10.921944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:10.934470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:11.027486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175cbdde-794f-46f6-8d14-83f5386e15ba None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:11.030177 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8240e7-aaeb-4a5d-9c2e-ad585f4ff899 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:11.064382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49b33d-2c56-41cb-b637-0d2fe07518b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:11.065191 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49b33d-2c56-41cb-b637-0d2fe07518b7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:11.065476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49b33d-2c56-41cb-b637-0d2fe07518b7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:11.065758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49b33d-2c56-41cb-b637-0d2fe07518b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:11.066446 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b956d81-64bb-404a-b2e8-f388f431cd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:11.066947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49b33d-2c56-41cb-b637-0d2fe07518b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:11.067245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b956d81-64bb-404a-b2e8-f388f431cd68 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:11.067534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b956d81-64bb-404a-b2e8-f388f431cd68 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:11.067818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b956d81-64bb-404a-b2e8-f388f431cd68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:11.068444 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a49b33d-2c56-41cb-b637-0d2fe07518b7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:11.070035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 10/18] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:11.075665 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49b33d-2c56-41cb-b637-0d2fe07518b7 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:11.086940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b956d81-64bb-404a-b2e8-f388f431cd68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:11.088471 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b956d81-64bb-404a-b2e8-f388f431cd68 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-0b956d81-64bb-404a-b2e8-f388f431cd68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:06:11.110566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:11.111619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:11.113135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:06:11.113135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:06:11.113135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:06:11.113135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:11.113135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 9/19] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:11.117568 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b956d81-64bb-404a-b2e8-f388f431cd68 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:11.161628 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:11.176428 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:11.182890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49b33d-2c56-41cb-b637-0d2fe07518b7 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:11.224733 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b956d81-64bb-404a-b2e8-f388f431cd68 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:11.225784 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080194a5-5cec-4076-9e43-e748dca4e482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:11.226452 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080194a5-5cec-4076-9e43-e748dca4e482 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:11.226738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080194a5-5cec-4076-9e43-e748dca4e482 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:11.227016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080194a5-5cec-4076-9e43-e748dca4e482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:11.227850 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080194a5-5cec-4076-9e43-e748dca4e482 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.domain.id=default, user.id=48915a5b6f6e4fd8a53e478480a2c94c) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:11.228163 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080194a5-5cec-4076-9e43-e748dca4e482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:11.229769 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080194a5-5cec-4076-9e43-e748dca4e482 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:11.237307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 11/20] 10.222.0.88 () {56 vars in 1322 bytes} [Thu May 19 20:06:11 2022] GET /identity/v3/role_assignments?user.id=48915a5b6f6e4fd8a53e478480a2c94c&scope.domain.id=default&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 233 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:11.266095 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92023a-6bd0-45e9-bd5d-1ae1f5fe6f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:11.266795 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92023a-6bd0-45e9-bd5d-1ae1f5fe6f7b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:11.267086 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92023a-6bd0-45e9-bd5d-1ae1f5fe6f7b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:11.267370 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92023a-6bd0-45e9-bd5d-1ae1f5fe6f7b None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:11.268158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92023a-6bd0-45e9-bd5d-1ae1f5fe6f7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:11.268514 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92023a-6bd0-45e9-bd5d-1ae1f5fe6f7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:11.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92023a-6bd0-45e9-bd5d-1ae1f5fe6f7b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:11.290857 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 10/21] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:13.912208 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4e67b-e20d-42c8-b5ab-459af25b7732 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:13.912783 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4e67b-e20d-42c8-b5ab-459af25b7732 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:13.912895 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4e67b-e20d-42c8-b5ab-459af25b7732 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:13.914362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 12/22] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:13.920383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ef264-4ba4-42f2-8d26-e38b43855330 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:13.920853 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ef264-4ba4-42f2-8d26-e38b43855330 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:13.920953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8ef264-4ba4-42f2-8d26-e38b43855330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.036372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9740f-b69e-4b25-8027-9fe19973171d None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.036744 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9740f-b69e-4b25-8027-9fe19973171d None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.036957 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9740f-b69e-4b25-8027-9fe19973171d None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.038628 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 13/23] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:14.051022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3610-73bb-469c-a7c3-911a7f007de4 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.051346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3610-73bb-469c-a7c3-911a7f007de4 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.051593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cd3610-73bb-469c-a7c3-911a7f007de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.082571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-33cd3610-73bb-469c-a7c3-911a7f007de4 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:14.083947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-33cd3610-73bb-469c-a7c3-911a7f007de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:14.114147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-da8ef264-4ba4-42f2-8d26-e38b43855330 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:14.115348 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-da8ef264-4ba4-42f2-8d26-e38b43855330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:14.206927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 11/24] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:14.244936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ef264-4ba4-42f2-8d26-e38b43855330 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:14.266795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 14/25] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:14.309439 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd3610-73bb-469c-a7c3-911a7f007de4 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:14.336779 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:14.404483 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8ef264-4ba4-42f2-8d26-e38b43855330 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:14.460993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:14.464365 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7979c-e043-494b-a81b-9588c5a85f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:14.470261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7979c-e043-494b-a81b-9588c5a85f04 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.470722 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7979c-e043-494b-a81b-9588c5a85f04 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.471171 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7979c-e043-494b-a81b-9588c5a85f04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.489570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7979c-e043-494b-a81b-9588c5a85f04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-fea7979c-e043-494b-a81b-9588c5a85f04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:14.502864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:14.503944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:14.504964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:14.505656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cd3610-73bb-469c-a7c3-911a7f007de4 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:14.507391 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 12/26] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:14.513611 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7979c-e043-494b-a81b-9588c5a85f04 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:14.585147 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b868ea-0ef7-4f33-b944-e8ffedd60cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:14.585987 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b868ea-0ef7-4f33-b944-e8ffedd60cb6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.586281 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b868ea-0ef7-4f33-b944-e8ffedd60cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.586564 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b868ea-0ef7-4f33-b944-e8ffedd60cb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.605601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b868ea-0ef7-4f33-b944-e8ffedd60cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:14.607208 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b868ea-0ef7-4f33-b944-e8ffedd60cb6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-56b868ea-0ef7-4f33-b944-e8ffedd60cb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:06:14.638605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:14.639832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:14.641189 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:06:14.641189 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:06:14.641189 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:06:14.641189 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:14.641189 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 15/27] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:14.649341 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b868ea-0ef7-4f33-b944-e8ffedd60cb6 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:14.701633 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7979c-e043-494b-a81b-9588c5a85f04 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:14.720445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:14.752934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61c507-74b6-484d-98dd-832c8f37eef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:14.753907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61c507-74b6-484d-98dd-832c8f37eef2 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.754746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61c507-74b6-484d-98dd-832c8f37eef2 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.755255 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61c507-74b6-484d-98dd-832c8f37eef2 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.760361 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61c507-74b6-484d-98dd-832c8f37eef2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:14.760602 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61c507-74b6-484d-98dd-832c8f37eef2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:14.762933 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce61c507-74b6-484d-98dd-832c8f37eef2 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:14.783389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 13/28] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:14.785835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e2951-a32a-4bd6-a226-97d04c8baaa6 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.786259 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e2951-a32a-4bd6-a226-97d04c8baaa6 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.786673 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e2951-a32a-4bd6-a226-97d04c8baaa6 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.789417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 14/29] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:14.790924 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf5b68-6e27-409f-a5a2-94a376da3cb6 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.791222 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf5b68-6e27-409f-a5a2-94a376da3cb6 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.791511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf5b68-6e27-409f-a5a2-94a376da3cb6 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.794177 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 15/30] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:14.795633 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf5b68-6e27-409f-a5a2-94a376da3cb6 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:14.813613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b868ea-0ef7-4f33-b944-e8ffedd60cb6 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:14.842787 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:14.852628 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b5678-fc99-49f1-942a-e2d62616a680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:14.853312 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b5678-fc99-49f1-942a-e2d62616a680 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.853606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b5678-fc99-49f1-942a-e2d62616a680 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.858016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b5678-fc99-49f1-942a-e2d62616a680 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.858016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b5678-fc99-49f1-942a-e2d62616a680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:14.858016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b5678-fc99-49f1-942a-e2d62616a680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:14.858016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b5678-fc99-49f1-942a-e2d62616a680 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:14.877425 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 16/31] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:14.883471 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cac812-4d20-4dd7-b744-9e711c079f70 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.883838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cac812-4d20-4dd7-b744-9e711c079f70 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.884278 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cac812-4d20-4dd7-b744-9e711c079f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.905450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-f0cac812-4d20-4dd7-b744-9e711c079f70 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:14.907442 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-f0cac812-4d20-4dd7-b744-9e711c079f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:14.929011 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf5b68-6e27-409f-a5a2-94a376da3cb6 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:14.978465 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6028f6-380b-4813-88f4-049fda95ef2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:14.979511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6028f6-380b-4813-88f4-049fda95ef2f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.980069 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6028f6-380b-4813-88f4-049fda95ef2f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.980486 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6028f6-380b-4813-88f4-049fda95ef2f None admin] PATH_INFO: `/v3/users/48915a5b6f6e4fd8a53e478480a2c94c` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:14.982525 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6028f6-380b-4813-88f4-049fda95ef2f None admin] RBAC: Authorizing `identity:get_user(user_id=48915a5b6f6e4fd8a53e478480a2c94c)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:14.984260 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6028f6-380b-4813-88f4-049fda95ef2f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:14.986737 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 16/32] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:14 2022] GET /identity/v3/users/48915a5b6f6e4fd8a53e478480a2c94c => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:14.988112 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f169e-45cc-4174-aa2c-486103df8856 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:14.988515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f169e-45cc-4174-aa2c-486103df8856 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:14.988894 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f169e-45cc-4174-aa2c-486103df8856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.001416 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 17/33] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:15.002066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cac812-4d20-4dd7-b744-9e711c079f70 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:15.010036 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-a37f169e-45cc-4174-aa2c-486103df8856 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:15.011562 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-a37f169e-45cc-4174-aa2c-486103df8856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:15.046075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:15.096125 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 17/34] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:15.096793 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f169e-45cc-4174-aa2c-486103df8856 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:15.147206 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cac812-4d20-4dd7-b744-9e711c079f70 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:15.165679 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:15.185452 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34591c85-f4e7-460f-bf9e-97b99ac8382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:15.186285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34591c85-f4e7-460f-bf9e-97b99ac8382c None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:15.186682 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34591c85-f4e7-460f-bf9e-97b99ac8382c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:15.187132 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34591c85-f4e7-460f-bf9e-97b99ac8382c None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.188075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34591c85-f4e7-460f-bf9e-97b99ac8382c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:15.189780 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34591c85-f4e7-460f-bf9e-97b99ac8382c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:15.216762 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f169e-45cc-4174-aa2c-486103df8856 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:15.256676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940510ba-4147-49fe-89c7-288bac9ad936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:15.257410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940510ba-4147-49fe-89c7-288bac9ad936 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:15.257695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940510ba-4147-49fe-89c7-288bac9ad936 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:15.257973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940510ba-4147-49fe-89c7-288bac9ad936 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.259279 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940510ba-4147-49fe-89c7-288bac9ad936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:15.260817 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940510ba-4147-49fe-89c7-288bac9ad936 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:15.262788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 18/35] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:15.267841 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940510ba-4147-49fe-89c7-288bac9ad936 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:15.311869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:15.384886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940510ba-4147-49fe-89c7-288bac9ad936 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:15.430436 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1266a7e-65ba-42d1-9d01-a835d8d20601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:15.431525 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1266a7e-65ba-42d1-9d01-a835d8d20601 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:15.432028 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1266a7e-65ba-42d1-9d01-a835d8d20601 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:15.432562 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1266a7e-65ba-42d1-9d01-a835d8d20601 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.433580 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1266a7e-65ba-42d1-9d01-a835d8d20601 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:15.435208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1266a7e-65ba-42d1-9d01-a835d8d20601 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:15.447029 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 18/36] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:06:15 2022] POST /identity/v3/projects => generated 301 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:15.448031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34591c85-f4e7-460f-bf9e-97b99ac8382c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:15.572893 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34591c85-f4e7-460f-bf9e-97b99ac8382c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:15.615231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3b4b0-e2dd-4a53-b944-4533df45ea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:15.616242 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3b4b0-e2dd-4a53-b944-4533df45ea42 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:15.616717 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3b4b0-e2dd-4a53-b944-4533df45ea42 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:15.617180 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3b4b0-e2dd-4a53-b944-4533df45ea42 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.618230 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3b4b0-e2dd-4a53-b944-4533df45ea42 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:15.620044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3b4b0-e2dd-4a53-b944-4533df45ea42 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:15.667150 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 19/37] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:06:15 2022] POST /identity/v3/roles => generated 225 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:15.668151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1266a7e-65ba-42d1-9d01-a835d8d20601 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:15.701045 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 19/38] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:06:15 2022] POST /identity/v3/roles => generated 219 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:15.703080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdbd96-f96e-4cab-bb62-710468a84164 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:15.703080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdbd96-f96e-4cab-bb62-710468a84164 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:15.703872 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cdbd96-f96e-4cab-bb62-710468a84164 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.707182 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 20/39] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:15.714370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585e4f2-fff3-43a4-a039-573009fafa0e None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:15.715520 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585e4f2-fff3-43a4-a039-573009fafa0e None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:15.716326 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585e4f2-fff3-43a4-a039-573009fafa0e None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.719727 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 21/40] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:15 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:15.721797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f30b6-88f9-4898-9f42-23f029e9ed85 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:15.722356 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f30b6-88f9-4898-9f42-23f029e9ed85 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:15.723021 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f30b6-88f9-4898-9f42-23f029e9ed85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.756567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-654f30b6-88f9-4898-9f42-23f029e9ed85 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:15.758944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-654f30b6-88f9-4898-9f42-23f029e9ed85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:15.800758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:15.826424 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:15.836315 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1266a7e-65ba-42d1-9d01-a835d8d20601 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:15.868851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 22/41] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:15.870891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee41046-3f76-46a4-af3c-58839446978f None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:15.871528 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee41046-3f76-46a4-af3c-58839446978f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:15.872033 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee41046-3f76-46a4-af3c-58839446978f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.884792 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8315b29-5834-41b8-a217-009f342fc93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:15.886735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8315b29-5834-41b8-a217-009f342fc93c None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:15.887002 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8315b29-5834-41b8-a217-009f342fc93c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:15.887310 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8315b29-5834-41b8-a217-009f342fc93c None admin] PATH_INFO: `/v3/domains/default/users/48915a5b6f6e4fd8a53e478480a2c94c/roles/ab7a63cec3a744848972fcbb89ecd0ca` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:15.892371 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8315b29-5834-41b8-a217-009f342fc93c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=48915a5b6f6e4fd8a53e478480a2c94c, role_id=ab7a63cec3a744848972fcbb89ecd0ca)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:15.892371 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8315b29-5834-41b8-a217-009f342fc93c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:15.907981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-5ee41046-3f76-46a4-af3c-58839446978f None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:15.915721 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-5ee41046-3f76-46a4-af3c-58839446978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:15.982868 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 23/42] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:15.983821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee41046-3f76-46a4-af3c-58839446978f None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:16.022685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 20/43] 10.222.0.88 () {58 vars in 1361 bytes} [Thu May 19 20:06:15 2022] PUT /identity/v3/domains/default/users/48915a5b6f6e4fd8a53e478480a2c94c/roles/ab7a63cec3a744848972fcbb89ecd0ca => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:16.024138 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8315b29-5834-41b8-a217-009f342fc93c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:16.087165 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:16.101734 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee41046-3f76-46a4-af3c-58839446978f None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:16.144861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fa376-09ea-4e19-bbd0-ac9db3de2274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:16.145698 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fa376-09ea-4e19-bbd0-ac9db3de2274 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:16.145988 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fa376-09ea-4e19-bbd0-ac9db3de2274 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:16.146267 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fa376-09ea-4e19-bbd0-ac9db3de2274 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:16.147596 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fa376-09ea-4e19-bbd0-ac9db3de2274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:16.149150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fa376-09ea-4e19-bbd0-ac9db3de2274 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:16.151086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 24/44] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:16.151223 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8315b29-5834-41b8-a217-009f342fc93c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:16.157693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fa376-09ea-4e19-bbd0-ac9db3de2274 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:16.196134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269ce3e5-d0cc-4d2c-acff-24038a33d99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:16.196898 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ce3e5-d0cc-4d2c-acff-24038a33d99b None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:16.197181 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ce3e5-d0cc-4d2c-acff-24038a33d99b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:16.197471 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269ce3e5-d0cc-4d2c-acff-24038a33d99b None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:16.198276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ce3e5-d0cc-4d2c-acff-24038a33d99b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:16.199784 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269ce3e5-d0cc-4d2c-acff-24038a33d99b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:16.280147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 21/45] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:06:16 2022] POST /identity/v3/roles => generated 223 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:16.281125 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fa376-09ea-4e19-bbd0-ac9db3de2274 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:16.320433 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d2f28-1d65-4120-ad80-df96144a5f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:16.321141 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d2f28-1d65-4120-ad80-df96144a5f5d None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:16.321434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d2f28-1d65-4120-ad80-df96144a5f5d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:16.321718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d2f28-1d65-4120-ad80-df96144a5f5d None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:16.322461 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d2f28-1d65-4120-ad80-df96144a5f5d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:16.323973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d2f28-1d65-4120-ad80-df96144a5f5d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:16.402301 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 25/46] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:06:16 2022] POST /identity/v3/projects => generated 312 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:06:18.310103 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae23c0-0413-4c33-bda8-c5956da2ea28 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:18.310674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae23c0-0413-4c33-bda8-c5956da2ea28 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:18.310674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae23c0-0413-4c33-bda8-c5956da2ea28 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:18.312046 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 22/47] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:18.317558 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3e536-6e0a-4be3-be95-0cb27bf32938 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:18.318035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3e536-6e0a-4be3-be95-0cb27bf32938 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:18.318248 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3e536-6e0a-4be3-be95-0cb27bf32938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:18.335420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-cac3e536-6e0a-4be3-be95-0cb27bf32938 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:18.336569 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-cac3e536-6e0a-4be3-be95-0cb27bf32938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:18.338596 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:18.404812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 26/48] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:18.428661 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae23c0-0413-4c33-bda8-c5956da2ea28 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:18.444831 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:18.538722 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae23c0-0413-4c33-bda8-c5956da2ea28 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:18.583087 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fe8a9-920d-43c2-aad6-6b78828c9657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:18.583893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fe8a9-920d-43c2-aad6-6b78828c9657 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:18.584178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fe8a9-920d-43c2-aad6-6b78828c9657 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:18.584500 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fe8a9-920d-43c2-aad6-6b78828c9657 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:18.597983 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fe8a9-920d-43c2-aad6-6b78828c9657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-006fe8a9-920d-43c2-aad6-6b78828c9657 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:18.609115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:18.610364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:18.611317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 23/49] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:06:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:18.618401 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3e536-6e0a-4be3-be95-0cb27bf32938 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:18.673833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4907c9-0679-4589-8d1b-a9e8af1b9c11 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:18.674126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4907c9-0679-4589-8d1b-a9e8af1b9c11 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:18.674413 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4907c9-0679-4589-8d1b-a9e8af1b9c11 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:18.675794 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 24/50] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:06:18.680983 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc7d19-1141-48e0-87ba-6601cc449e3a None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:18.681287 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc7d19-1141-48e0-87ba-6601cc449e3a None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:18.681573 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc7d19-1141-48e0-87ba-6601cc449e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:18.699676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-6dcc7d19-1141-48e0-87ba-6601cc449e3a None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:18.701245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-6dcc7d19-1141-48e0-87ba-6601cc449e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:18.720261 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3e536-6e0a-4be3-be95-0cb27bf32938 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:18.754663 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7329b6-9ea0-46a3-a7f0-01b43e455174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:18.755336 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7329b6-9ea0-46a3-a7f0-01b43e455174 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:18.755623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7329b6-9ea0-46a3-a7f0-01b43e455174 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:18.755904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7329b6-9ea0-46a3-a7f0-01b43e455174 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:18.756689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7329b6-9ea0-46a3-a7f0-01b43e455174 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:18.757003 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7329b6-9ea0-46a3-a7f0-01b43e455174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:18.758415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7329b6-9ea0-46a3-a7f0-01b43e455174 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:18.763924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:18.768057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 27/51] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:18.771727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 25/52] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:18.781977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7329b6-9ea0-46a3-a7f0-01b43e455174 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:18.795694 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc7d19-1141-48e0-87ba-6601cc449e3a None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:18.799340 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:18.886603 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7329b6-9ea0-46a3-a7f0-01b43e455174 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:18.918701 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc7d19-1141-48e0-87ba-6601cc449e3a None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:18.932776 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17464de5-dbd8-4886-b528-7fc9d8b7d605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:18.934148 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17464de5-dbd8-4886-b528-7fc9d8b7d605 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:18.934660 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17464de5-dbd8-4886-b528-7fc9d8b7d605 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:18.935153 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17464de5-dbd8-4886-b528-7fc9d8b7d605 None admin] PATH_INFO: `/v3/users/48915a5b6f6e4fd8a53e478480a2c94c` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:18.936657 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17464de5-dbd8-4886-b528-7fc9d8b7d605 None admin] RBAC: Authorizing `identity:get_user(user_id=48915a5b6f6e4fd8a53e478480a2c94c)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:18.938883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17464de5-dbd8-4886-b528-7fc9d8b7d605 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:18.941119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 28/53] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:18 2022] GET /identity/v3/users/48915a5b6f6e4fd8a53e478480a2c94c => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:18.948171 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17464de5-dbd8-4886-b528-7fc9d8b7d605 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:18.969236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00da831-3d1f-427a-a7ec-5d45ce0d49e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:18.970818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00da831-3d1f-427a-a7ec-5d45ce0d49e4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:18.971238 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00da831-3d1f-427a-a7ec-5d45ce0d49e4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:18.971518 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00da831-3d1f-427a-a7ec-5d45ce0d49e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:18.973003 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00da831-3d1f-427a-a7ec-5d45ce0d49e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:18.974484 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00da831-3d1f-427a-a7ec-5d45ce0d49e4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:18.976380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 26/54] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:18.982894 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00da831-3d1f-427a-a7ec-5d45ce0d49e4 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:19.029265 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:19.066565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:19.070300 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17464de5-dbd8-4886-b528-7fc9d8b7d605 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:19.098819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00da831-3d1f-427a-a7ec-5d45ce0d49e4 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:19.109443 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff630d0c-6a00-470c-9ec1-735f5f800ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:19.110249 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff630d0c-6a00-470c-9ec1-735f5f800ea6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:19.110538 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff630d0c-6a00-470c-9ec1-735f5f800ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:19.110822 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff630d0c-6a00-470c-9ec1-735f5f800ea6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:19.112388 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff630d0c-6a00-470c-9ec1-735f5f800ea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:19.113599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff630d0c-6a00-470c-9ec1-735f5f800ea6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:19.115240 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 29/55] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:19.121329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff630d0c-6a00-470c-9ec1-735f5f800ea6 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:19.144924 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988be6d2-c19d-41da-b5ac-edeb351837f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:19.145623 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988be6d2-c19d-41da-b5ac-edeb351837f7 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:19.145978 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988be6d2-c19d-41da-b5ac-edeb351837f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:19.146331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988be6d2-c19d-41da-b5ac-edeb351837f7 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:19.147122 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988be6d2-c19d-41da-b5ac-edeb351837f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:19.148675 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988be6d2-c19d-41da-b5ac-edeb351837f7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:19.230663 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 27/56] 10.222.0.88 () {60 vars in 1140 bytes} [Thu May 19 20:06:18 2022] POST /identity/v3/projects => generated 298 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:06:19.231530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff630d0c-6a00-470c-9ec1-735f5f800ea6 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:19.274993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df22030-74f0-4b9e-abee-53bad1d493a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:19.275679 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df22030-74f0-4b9e-abee-53bad1d493a9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:19.275969 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df22030-74f0-4b9e-abee-53bad1d493a9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:19.276382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df22030-74f0-4b9e-abee-53bad1d493a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:19.277234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df22030-74f0-4b9e-abee-53bad1d493a9 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.domain.id=default, user.id=48915a5b6f6e4fd8a53e478480a2c94c) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:19.277551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df22030-74f0-4b9e-abee-53bad1d493a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:19.279092 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df22030-74f0-4b9e-abee-53bad1d493a9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:19.283917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:19.287648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 30/57] 10.222.0.88 () {56 vars in 1322 bytes} [Thu May 19 20:06:19 2022] GET /identity/v3/role_assignments?user.id=48915a5b6f6e4fd8a53e478480a2c94c&scope.domain.id=default&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 530 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:21.495293 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754087-6742-4be0-bef3-333cecee546d None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:21.495848 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754087-6742-4be0-bef3-333cecee546d None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:21.495848 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab754087-6742-4be0-bef3-333cecee546d None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:21.497290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 28/58] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:21.503410 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdac5c8-90f2-47e5-bb37-9242970e4150 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:21.504006 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdac5c8-90f2-47e5-bb37-9242970e4150 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:21.504006 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdac5c8-90f2-47e5-bb37-9242970e4150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:21.521868 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-ccdac5c8-90f2-47e5-bb37-9242970e4150 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:21.523089 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-ccdac5c8-90f2-47e5-bb37-9242970e4150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:21.595070 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 31/59] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:21.620824 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab754087-6742-4be0-bef3-333cecee546d None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:21.727992 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab754087-6742-4be0-bef3-333cecee546d None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:21.752058 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:21.762937 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ebc058-75d7-44ab-a23a-9b29d0c22681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:21.763742 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebc058-75d7-44ab-a23a-9b29d0c22681 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:21.764036 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebc058-75d7-44ab-a23a-9b29d0c22681 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:21.764320 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebc058-75d7-44ab-a23a-9b29d0c22681 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:21.765596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ebc058-75d7-44ab-a23a-9b29d0c22681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:21.767114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ebc058-75d7-44ab-a23a-9b29d0c22681 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:21.768805 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 29/60] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:21.774360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdac5c8-90f2-47e5-bb37-9242970e4150 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:21.873009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdac5c8-90f2-47e5-bb37-9242970e4150 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:21.901607 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:21.907907 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7efda4c-d17d-49c7-be4e-47efdac24da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:21.908974 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efda4c-d17d-49c7-be4e-47efdac24da1 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:21.911529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efda4c-d17d-49c7-be4e-47efdac24da1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:21.911529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7efda4c-d17d-49c7-be4e-47efdac24da1 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:21.911529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efda4c-d17d-49c7-be4e-47efdac24da1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:21.911803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7efda4c-d17d-49c7-be4e-47efdac24da1 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:22.034548 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 32/61] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:06:21 2022] POST /identity/v3/users => generated 275 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:25.508711 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e54b35-d5b8-4d16-8dd1-bf4670d0bfda None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:25.509781 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e54b35-d5b8-4d16-8dd1-bf4670d0bfda None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:25.510307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e54b35-d5b8-4d16-8dd1-bf4670d0bfda None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:25.512006 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 30/62] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:25.517888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ca86c-4f66-44f2-ba59-5dec2721e4b7 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:25.518864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ca86c-4f66-44f2-ba59-5dec2721e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:25.519372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ca86c-4f66-44f2-ba59-5dec2721e4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:25.541422 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-538ca86c-4f66-44f2-ba59-5dec2721e4b7 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:25.543014 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-538ca86c-4f66-44f2-ba59-5dec2721e4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:25.563024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3f4ca-aa02-4d8b-8340-b01d496aa393 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:25.563787 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3f4ca-aa02-4d8b-8340-b01d496aa393 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:25.564290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e3f4ca-aa02-4d8b-8340-b01d496aa393 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:25.565946 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 31/63] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:25.571329 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f669c5-9d66-4fdc-9bb2-27f35da042b4 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:25.571899 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f669c5-9d66-4fdc-9bb2-27f35da042b4 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:25.572469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f669c5-9d66-4fdc-9bb2-27f35da042b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:25.599328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-58f669c5-9d66-4fdc-9bb2-27f35da042b4 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:25.602417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-58f669c5-9d66-4fdc-9bb2-27f35da042b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:25.663025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 33/64] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:25.702481 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ca86c-4f66-44f2-ba59-5dec2721e4b7 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:25.744479 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 32/65] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:25.773957 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f669c5-9d66-4fdc-9bb2-27f35da042b4 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:25.839633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ca86c-4f66-44f2-ba59-5dec2721e4b7 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:25.868456 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:25.890249 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4cb18-8e50-4232-ab75-f985d31b960e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:25.891338 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4cb18-8e50-4232-ab75-f985d31b960e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:25.892292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4cb18-8e50-4232-ab75-f985d31b960e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:25.892780 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4cb18-8e50-4232-ab75-f985d31b960e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:25.908937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4cb18-8e50-4232-ab75-f985d31b960e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-5da4cb18-8e50-4232-ab75-f985d31b960e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:25.920726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:25.924009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:25.925121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 34/66] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:06:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:06:25.926761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6da66-3b70-43e7-b25e-fadc0138b282 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:25.926897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6da66-3b70-43e7-b25e-fadc0138b282 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:25.927158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6da66-3b70-43e7-b25e-fadc0138b282 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:25.928808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 35/67] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:25.931018 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6da66-3b70-43e7-b25e-fadc0138b282 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:25.931897 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f669c5-9d66-4fdc-9bb2-27f35da042b4 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:25.964872 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:25.989894 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1d077-5a7d-4694-a528-5a406e89a620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:25.991019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1d077-5a7d-4694-a528-5a406e89a620 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:25.991471 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1d077-5a7d-4694-a528-5a406e89a620 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:25.991924 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1d077-5a7d-4694-a528-5a406e89a620 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.003636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed1d077-5a7d-4694-a528-5a406e89a620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-8ed1d077-5a7d-4694-a528-5a406e89a620 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:26.011845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:26.013251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:26.014252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:26.017203 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 33/68] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:06:26.019231 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19da762-7d47-4c43-a590-34c0ddf31536 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.019729 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19da762-7d47-4c43-a590-34c0ddf31536 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.020244 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19da762-7d47-4c43-a590-34c0ddf31536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.039221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-c19da762-7d47-4c43-a590-34c0ddf31536 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:26.040467 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-c19da762-7d47-4c43-a590-34c0ddf31536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:26.056378 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6da66-3b70-43e7-b25e-fadc0138b282 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.096240 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307722c-5dd2-444c-8832-198e0b7f0858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.096716 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307722c-5dd2-444c-8832-198e0b7f0858 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.097006 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307722c-5dd2-444c-8832-198e0b7f0858 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.097288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307722c-5dd2-444c-8832-198e0b7f0858 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.098049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307722c-5dd2-444c-8832-198e0b7f0858 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:26.098365 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307722c-5dd2-444c-8832-198e0b7f0858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:26.099788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307722c-5dd2-444c-8832-198e0b7f0858 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:26.114128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 36/69] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:26.115270 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307722c-5dd2-444c-8832-198e0b7f0858 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:26.123069 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 34/70] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:26.124870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebd90c-aec1-46e1-96b9-005a39659d80 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.125373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebd90c-aec1-46e1-96b9-005a39659d80 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.125844 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebd90c-aec1-46e1-96b9-005a39659d80 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.127483 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 35/71] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:26.128478 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebd90c-aec1-46e1-96b9-005a39659d80 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:26.158249 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:26.219524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307722c-5dd2-444c-8832-198e0b7f0858 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.240250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebd90c-aec1-46e1-96b9-005a39659d80 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.253628 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58276f-73c4-4be8-8be2-2721c93dbca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.254275 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:26.254398 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58276f-73c4-4be8-8be2-2721c93dbca4 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.254946 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58276f-73c4-4be8-8be2-2721c93dbca4 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.255253 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58276f-73c4-4be8-8be2-2721c93dbca4 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.256018 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58276f-73c4-4be8-8be2-2721c93dbca4 None admin] RBAC: Adding query filter params (name=member) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:26.256352 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58276f-73c4-4be8-8be2-2721c93dbca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:26.257778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58276f-73c4-4be8-8be2-2721c93dbca4 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:26.266714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 37/72] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:26.271785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f199cc6-8bc2-4551-8498-ea098d154fe1 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.271785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f199cc6-8bc2-4551-8498-ea098d154fe1 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.271785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f199cc6-8bc2-4551-8498-ea098d154fe1 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.271785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 38/73] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:26.271785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375c2c-60ba-41f0-94c4-a0485d67264f None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.271785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375c2c-60ba-41f0-94c4-a0485d67264f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.272426 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a375c2c-60ba-41f0-94c4-a0485d67264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.279048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181c077-6cc3-4f4d-aa5d-4c21c9c9a0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.279953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181c077-6cc3-4f4d-aa5d-4c21c9c9a0f3 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.280240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181c077-6cc3-4f4d-aa5d-4c21c9c9a0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.280551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181c077-6cc3-4f4d-aa5d-4c21c9c9a0f3 None admin] PATH_INFO: `/v3/users/48915a5b6f6e4fd8a53e478480a2c94c` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.282355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181c077-6cc3-4f4d-aa5d-4c21c9c9a0f3 None admin] RBAC: Authorizing `identity:get_user(user_id=48915a5b6f6e4fd8a53e478480a2c94c)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:26.283832 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181c077-6cc3-4f4d-aa5d-4c21c9c9a0f3 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:26.285542 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 36/74] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:26 2022] GET /identity/v3/users/48915a5b6f6e4fd8a53e478480a2c94c => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:26.286329 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181c077-6cc3-4f4d-aa5d-4c21c9c9a0f3 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:26.292287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-5a375c2c-60ba-41f0-94c4-a0485d67264f None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:26.293468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-5a375c2c-60ba-41f0-94c4-a0485d67264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:26.361492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 39/75] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:26.365363 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a375c2c-60ba-41f0-94c4-a0485d67264f None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:26.393761 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181c077-6cc3-4f4d-aa5d-4c21c9c9a0f3 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.401125 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:26.433272 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e839c1-601b-433b-a351-c28ab86890dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.434095 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e839c1-601b-433b-a351-c28ab86890dc None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.434384 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e839c1-601b-433b-a351-c28ab86890dc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.434669 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e839c1-601b-433b-a351-c28ab86890dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.445740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e839c1-601b-433b-a351-c28ab86890dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-b3e839c1-601b-433b-a351-c28ab86890dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:26.454407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:26.455511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:26.456457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 37/76] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:06:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:26.457250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446533-84e6-4065-9daa-d90c2b1490aa None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.457250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446533-84e6-4065-9daa-d90c2b1490aa None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.457551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21446533-84e6-4065-9daa-d90c2b1490aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.474002 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-21446533-84e6-4065-9daa-d90c2b1490aa None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:26.475204 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-21446533-84e6-4065-9daa-d90c2b1490aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:26.480266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a375c2c-60ba-41f0-94c4-a0485d67264f None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.510332 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:26.515568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c8d75-68da-4446-a240-5c8e1249f42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.516975 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c8d75-68da-4446-a240-5c8e1249f42b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.517265 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c8d75-68da-4446-a240-5c8e1249f42b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.517544 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c8d75-68da-4446-a240-5c8e1249f42b None admin] PATH_INFO: `/v3/users/e92111952a254ff58d562324db8f63fb` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.531322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c8d75-68da-4446-a240-5c8e1249f42b None admin] RBAC: Authorizing `identity:get_user(user_id=e92111952a254ff58d562324db8f63fb)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:26.532940 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c8d75-68da-4446-a240-5c8e1249f42b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:26.534679 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 40/77] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:26 2022] GET /identity/v3/users/e92111952a254ff58d562324db8f63fb => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:26.535470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c8d75-68da-4446-a240-5c8e1249f42b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:26.544054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 38/78] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:26.545198 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21446533-84e6-4065-9daa-d90c2b1490aa None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:26.628148 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:26.640025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c8d75-68da-4446-a240-5c8e1249f42b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.647478 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21446533-84e6-4065-9daa-d90c2b1490aa None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.674994 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980322a3-c79f-4db0-a4e1-1bf0ab0e0265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.675652 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980322a3-c79f-4db0-a4e1-1bf0ab0e0265 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.675942 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980322a3-c79f-4db0-a4e1-1bf0ab0e0265 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.676219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980322a3-c79f-4db0-a4e1-1bf0ab0e0265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.681325 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deef25e-4137-4fb5-b6a9-af9b7b249865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.682147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deef25e-4137-4fb5-b6a9-af9b7b249865 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.682433 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deef25e-4137-4fb5-b6a9-af9b7b249865 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.682708 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deef25e-4137-4fb5-b6a9-af9b7b249865 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.683910 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deef25e-4137-4fb5-b6a9-af9b7b249865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:26.685416 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deef25e-4137-4fb5-b6a9-af9b7b249865 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:26.687047 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 39/79] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:26.688380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deef25e-4137-4fb5-b6a9-af9b7b249865 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:26.730093 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:26.745897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980322a3-c79f-4db0-a4e1-1bf0ab0e0265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:26.747433 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980322a3-c79f-4db0-a4e1-1bf0ab0e0265 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:26.791327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deef25e-4137-4fb5-b6a9-af9b7b249865 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.826699 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724370e-6c42-4259-9a38-5859c3904d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.827290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724370e-6c42-4259-9a38-5859c3904d0c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.827578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724370e-6c42-4259-9a38-5859c3904d0c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.827860 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724370e-6c42-4259-9a38-5859c3904d0c None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.828658 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724370e-6c42-4259-9a38-5859c3904d0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:26.828969 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724370e-6c42-4259-9a38-5859c3904d0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:26.830373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724370e-6c42-4259-9a38-5859c3904d0c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:26.835763 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:26.838611 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 40/80] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:06:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:26.841284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724370e-6c42-4259-9a38-5859c3904d0c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:26.849012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 41/81] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:26.850228 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980322a3-c79f-4db0-a4e1-1bf0ab0e0265 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:26.940295 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:26.943456 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724370e-6c42-4259-9a38-5859c3904d0c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.953064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980322a3-c79f-4db0-a4e1-1bf0ab0e0265 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:26.977129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb9aa5-850a-476c-a7d6-622cfb22f551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.977787 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb9aa5-850a-476c-a7d6-622cfb22f551 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.978076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb9aa5-850a-476c-a7d6-622cfb22f551 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.978364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb9aa5-850a-476c-a7d6-622cfb22f551 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.987031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9def63b-5ceb-4738-a187-ca5accc1f075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:26.987830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9def63b-5ceb-4738-a187-ca5accc1f075 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:26.988116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9def63b-5ceb-4738-a187-ca5accc1f075 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:26.988423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9def63b-5ceb-4738-a187-ca5accc1f075 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:26.998920 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9def63b-5ceb-4738-a187-ca5accc1f075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-b9def63b-5ceb-4738-a187-ca5accc1f075 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:27.007077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:27.008176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 42/82] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:27.014397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9def63b-5ceb-4738-a187-ca5accc1f075 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:27.048860 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb9aa5-850a-476c-a7d6-622cfb22f551 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:27.050362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb9aa5-850a-476c-a7d6-622cfb22f551 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:27.060773 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:27.116154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9def63b-5ceb-4738-a187-ca5accc1f075 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:27.147970 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:27.150238 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36c760-4c6f-4def-9c69-3a1a49cbcd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:27.150926 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36c760-4c6f-4def-9c69-3a1a49cbcd64 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:27.151214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36c760-4c6f-4def-9c69-3a1a49cbcd64 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:27.151495 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36c760-4c6f-4def-9c69-3a1a49cbcd64 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:27.151802 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 41/83] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:27.152229 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36c760-4c6f-4def-9c69-3a1a49cbcd64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:27.152672 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb9aa5-850a-476c-a7d6-622cfb22f551 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:27.153835 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36c760-4c6f-4def-9c69-3a1a49cbcd64 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:27.214506 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 43/84] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:06:27 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:27.215442 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36c760-4c6f-4def-9c69-3a1a49cbcd64 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:27.271581 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb9aa5-850a-476c-a7d6-622cfb22f551 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:27.317001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa25359-87ba-4489-824b-03b28854c865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:27.317994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa25359-87ba-4489-824b-03b28854c865 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:27.318284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa25359-87ba-4489-824b-03b28854c865 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:27.318568 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa25359-87ba-4489-824b-03b28854c865 None admin] PATH_INFO: `/v3/users/e92111952a254ff58d562324db8f63fb` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:27.319954 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa25359-87ba-4489-824b-03b28854c865 None admin] RBAC: Authorizing `identity:get_user(user_id=e92111952a254ff58d562324db8f63fb)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:27.321510 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa25359-87ba-4489-824b-03b28854c865 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:27.323255 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 42/85] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:27 2022] GET /identity/v3/users/e92111952a254ff58d562324db8f63fb => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:27.324069 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa25359-87ba-4489-824b-03b28854c865 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:27.341345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36c760-4c6f-4def-9c69-3a1a49cbcd64 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:27.345637 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:27.380861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce2b28-b9bf-488d-8db4-7b206af1d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:27.381771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce2b28-b9bf-488d-8db4-7b206af1d53b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:27.382064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce2b28-b9bf-488d-8db4-7b206af1d53b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:27.382354 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ce2b28-b9bf-488d-8db4-7b206af1d53b None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:27.383632 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce2b28-b9bf-488d-8db4-7b206af1d53b None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:27.385173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ce2b28-b9bf-488d-8db4-7b206af1d53b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:27.387234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 44/86] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:27 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:27.387981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce2b28-b9bf-488d-8db4-7b206af1d53b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:27.424836 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:27.480214 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa25359-87ba-4489-824b-03b28854c865 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:27.531658 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16c0e7-1086-446a-bcb3-9b2733e1f27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:27.532611 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c0e7-1086-446a-bcb3-9b2733e1f27e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:27.532798 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c0e7-1086-446a-bcb3-9b2733e1f27e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:27.533371 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c0e7-1086-446a-bcb3-9b2733e1f27e None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:27.535048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16c0e7-1086-446a-bcb3-9b2733e1f27e None admin] RBAC: Adding query filter params (name=member) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:27.535048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16c0e7-1086-446a-bcb3-9b2733e1f27e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:27.536141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16c0e7-1086-446a-bcb3-9b2733e1f27e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:27.544397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ce2b28-b9bf-488d-8db4-7b206af1d53b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:27.549698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 43/87] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:27.550690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16c0e7-1086-446a-bcb3-9b2733e1f27e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:27.596764 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236dcd3-7dbc-4082-b7ad-06c287916a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:27.597706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236dcd3-7dbc-4082-b7ad-06c287916a14 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:27.597999 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236dcd3-7dbc-4082-b7ad-06c287916a14 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:27.598282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236dcd3-7dbc-4082-b7ad-06c287916a14 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:27.599645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236dcd3-7dbc-4082-b7ad-06c287916a14 None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:27.601509 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236dcd3-7dbc-4082-b7ad-06c287916a14 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:27.604647 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 45/88] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:27 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:27.604647 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236dcd3-7dbc-4082-b7ad-06c287916a14 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:27.669390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:27.692083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16c0e7-1086-446a-bcb3-9b2733e1f27e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:27.727208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:27.735433 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21354181-1e13-42c1-adb3-93d0dfa83a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:27.742833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21354181-1e13-42c1-adb3-93d0dfa83a12 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:27.742833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21354181-1e13-42c1-adb3-93d0dfa83a12 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:27.742833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21354181-1e13-42c1-adb3-93d0dfa83a12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:27.758665 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236dcd3-7dbc-4082-b7ad-06c287916a14 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:27.812682 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d860a5-0832-4633-93eb-1d7d128d86c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:27.812956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d860a5-0832-4633-93eb-1d7d128d86c6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:27.813066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d860a5-0832-4633-93eb-1d7d128d86c6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:27.813349 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d860a5-0832-4633-93eb-1d7d128d86c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:27.814606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d860a5-0832-4633-93eb-1d7d128d86c6 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a, user.id=48915a5b6f6e4fd8a53e478480a2c94c) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:27.814926 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d860a5-0832-4633-93eb-1d7d128d86c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:27.817499 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d860a5-0832-4633-93eb-1d7d128d86c6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:27.825452 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 46/89] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:27 2022] GET /identity/v3/role_assignments?user.id=48915a5b6f6e4fd8a53e478480a2c94c&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:27.826769 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d860a5-0832-4633-93eb-1d7d128d86c6 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:27.830541 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21354181-1e13-42c1-adb3-93d0dfa83a12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:27.832232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21354181-1e13-42c1-adb3-93d0dfa83a12 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:27.969215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 44/90] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:27.970240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21354181-1e13-42c1-adb3-93d0dfa83a12 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:27.979684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d860a5-0832-4633-93eb-1d7d128d86c6 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:27.982803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:28.012766 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:28.047713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81c014-65a6-4210-bd0a-4941a743a589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:28.048672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81c014-65a6-4210-bd0a-4941a743a589 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:28.048959 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81c014-65a6-4210-bd0a-4941a743a589 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:28.049244 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81c014-65a6-4210-bd0a-4941a743a589 None admin] PATH_INFO: `/v3/users/e92111952a254ff58d562324db8f63fb` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:28.052369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81c014-65a6-4210-bd0a-4941a743a589 None admin] RBAC: Authorizing `identity:get_user(user_id=e92111952a254ff58d562324db8f63fb)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:28.055522 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81c014-65a6-4210-bd0a-4941a743a589 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:28.057417 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 47/91] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:27 2022] GET /identity/v3/users/e92111952a254ff58d562324db8f63fb => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:28.063054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81c014-65a6-4210-bd0a-4941a743a589 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:28.219685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81c014-65a6-4210-bd0a-4941a743a589 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:28.243039 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21354181-1e13-42c1-adb3-93d0dfa83a12 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:28.280804 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607ad0b-2774-446a-b4f7-94894d38d7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:28.282102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607ad0b-2774-446a-b4f7-94894d38d7e4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:28.282385 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607ad0b-2774-446a-b4f7-94894d38d7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:28.282659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607ad0b-2774-446a-b4f7-94894d38d7e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:28.283501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607ad0b-2774-446a-b4f7-94894d38d7e4 None admin] RBAC: Adding query filter params (role.id=ba04654a403444e4ae54a1a216888fa5, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a, user.id=e92111952a254ff58d562324db8f63fb) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:28.283822 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607ad0b-2774-446a-b4f7-94894d38d7e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:28.285347 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607ad0b-2774-446a-b4f7-94894d38d7e4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:28.294284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 45/92] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:27 2022] GET /identity/v3/role_assignments?user.id=e92111952a254ff58d562324db8f63fb&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:28.296469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6438b-8f8a-4b3b-94ab-0a19c9c46759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:28.297404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6438b-8f8a-4b3b-94ab-0a19c9c46759 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:28.297687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6438b-8f8a-4b3b-94ab-0a19c9c46759 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:28.297961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6438b-8f8a-4b3b-94ab-0a19c9c46759 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:28.299104 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607ad0b-2774-446a-b4f7-94894d38d7e4 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:28.300244 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6438b-8f8a-4b3b-94ab-0a19c9c46759 None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:28.302261 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6438b-8f8a-4b3b-94ab-0a19c9c46759 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:28.304414 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 48/93] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:28 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:28.315325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6438b-8f8a-4b3b-94ab-0a19c9c46759 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:28.416118 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:28.483588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:28.499453 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607ad0b-2774-446a-b4f7-94894d38d7e4 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:28.549366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6438b-8f8a-4b3b-94ab-0a19c9c46759 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:28.573802 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6102e4-0aae-49ed-9ed3-5ffd70a5e880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:28.574515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6102e4-0aae-49ed-9ed3-5ffd70a5e880 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:28.574759 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6102e4-0aae-49ed-9ed3-5ffd70a5e880 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:28.575036 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6102e4-0aae-49ed-9ed3-5ffd70a5e880 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:28.608077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b3515-a442-44e9-9f10-79548257d0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:28.608793 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b3515-a442-44e9-9f10-79548257d0f4 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:28.609079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b3515-a442-44e9-9f10-79548257d0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:28.609982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b3515-a442-44e9-9f10-79548257d0f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:28.610836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b3515-a442-44e9-9f10-79548257d0f4 None admin] RBAC: Adding query filter params (role.id=c9fbfe89c48d458c94f2a8a11652fcf2, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a, user.id=e92111952a254ff58d562324db8f63fb) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:28.611153 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b3515-a442-44e9-9f10-79548257d0f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:28.612677 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b3515-a442-44e9-9f10-79548257d0f4 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:28.618871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 49/94] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:28 2022] GET /identity/v3/role_assignments?user.id=e92111952a254ff58d562324db8f63fb&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=c9fbfe89c48d458c94f2a8a11652fcf2 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:28.714910 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6102e4-0aae-49ed-9ed3-5ffd70a5e880 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:28.716470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6102e4-0aae-49ed-9ed3-5ffd70a5e880 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:28.841130 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:28.849443 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 46/95] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:28.855395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b3515-a442-44e9-9f10-79548257d0f4 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:28.971964 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b3515-a442-44e9-9f10-79548257d0f4 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:28.975019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:29.010612 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6115bf3-626c-428c-9ae3-6e512c03e3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:29.011498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6115bf3-626c-428c-9ae3-6e512c03e3e9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:29.011790 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6115bf3-626c-428c-9ae3-6e512c03e3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:29.012076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6115bf3-626c-428c-9ae3-6e512c03e3e9 None admin] PATH_INFO: `/v3/projects/05f4d9d30ea44bf28334370502d103db` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:29.013598 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6115bf3-626c-428c-9ae3-6e512c03e3e9 None admin] RBAC: Authorizing `identity:get_project(project_id=05f4d9d30ea44bf28334370502d103db)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:29.016019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6115bf3-626c-428c-9ae3-6e512c03e3e9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:29.018974 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 50/96] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:28 2022] GET /identity/v3/projects/05f4d9d30ea44bf28334370502d103db => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:29.025061 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6102e4-0aae-49ed-9ed3-5ffd70a5e880 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:29.176236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6102e4-0aae-49ed-9ed3-5ffd70a5e880 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:29.230488 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe3056-6cb2-4f78-90fd-cde7fc143ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:29.231154 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe3056-6cb2-4f78-90fd-cde7fc143ca8 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:29.231442 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe3056-6cb2-4f78-90fd-cde7fc143ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:29.231726 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe3056-6cb2-4f78-90fd-cde7fc143ca8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:29.232595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe3056-6cb2-4f78-90fd-cde7fc143ca8 None admin] RBAC: Adding query filter params (role.id=ba04654a403444e4ae54a1a216888fa5, scope.project.id=05f4d9d30ea44bf28334370502d103db, user.id=e92111952a254ff58d562324db8f63fb) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:29.232915 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe3056-6cb2-4f78-90fd-cde7fc143ca8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:29.234409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe3056-6cb2-4f78-90fd-cde7fc143ca8 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:29.247950 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 47/97] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:29 2022] GET /identity/v3/role_assignments?user.id=e92111952a254ff58d562324db8f63fb&scope.project.id=05f4d9d30ea44bf28334370502d103db&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:30.434285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ee6ef-8902-4072-90df-da3e12aea29f None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:30.434826 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ee6ef-8902-4072-90df-da3e12aea29f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:30.434826 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ee6ef-8902-4072-90df-da3e12aea29f None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:30.436142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 51/98] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:30.442596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac058d7b-d97d-4766-ba02-e879b60c06ff None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:30.443120 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac058d7b-d97d-4766-ba02-e879b60c06ff None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:30.443216 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac058d7b-d97d-4766-ba02-e879b60c06ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:30.465000 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-ac058d7b-d97d-4766-ba02-e879b60c06ff None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:30.466235 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-ac058d7b-d97d-4766-ba02-e879b60c06ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:30.549458 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:30.568355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 48/99] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:30.604897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ee6ef-8902-4072-90df-da3e12aea29f None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:30.742755 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ee6ef-8902-4072-90df-da3e12aea29f None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:30.785230 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad5863-b7ae-42bc-a7df-0b4032da8b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:30.786120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5863-b7ae-42bc-a7df-0b4032da8b98 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:30.786418 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5863-b7ae-42bc-a7df-0b4032da8b98 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:30.786704 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5863-b7ae-42bc-a7df-0b4032da8b98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:30.787939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad5863-b7ae-42bc-a7df-0b4032da8b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:30.789450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad5863-b7ae-42bc-a7df-0b4032da8b98 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:30.796429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 52/100] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:30.796886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3390-b5e7-4e6e-badb-cdb9f87fca6b None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:30.797196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3390-b5e7-4e6e-badb-cdb9f87fca6b None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:30.797493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e3390-b5e7-4e6e-badb-cdb9f87fca6b None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:30.798385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad5863-b7ae-42bc-a7df-0b4032da8b98 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:30.799129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 49/101] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:30.815492 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9c533-505c-4a2e-bab6-edfadb685533 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:30.819027 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9c533-505c-4a2e-bab6-edfadb685533 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:30.819027 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9c533-505c-4a2e-bab6-edfadb685533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:30.845497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-38d9c533-505c-4a2e-bab6-edfadb685533 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:30.846729 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-38d9c533-505c-4a2e-bab6-edfadb685533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:30.848007 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:30.922437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad5863-b7ae-42bc-a7df-0b4032da8b98 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:30.963243 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 50/102] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:30.973983 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cee6bb-f33f-48c7-9b8c-d27be1f545c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:30.974883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cee6bb-f33f-48c7-9b8c-d27be1f545c9 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:30.975357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cee6bb-f33f-48c7-9b8c-d27be1f545c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:30.975812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cee6bb-f33f-48c7-9b8c-d27be1f545c9 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:30.976746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cee6bb-f33f-48c7-9b8c-d27be1f545c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:30.978400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cee6bb-f33f-48c7-9b8c-d27be1f545c9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:30.987317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9c533-505c-4a2e-bab6-edfadb685533 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:31.105140 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 53/103] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:06:30 2022] POST /identity/v3/projects => generated 302 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:31.142876 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9c533-505c-4a2e-bab6-edfadb685533 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:31.171054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:31.196245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df0fb0-866f-4ec0-a98e-440812e2feb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:31.197156 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df0fb0-866f-4ec0-a98e-440812e2feb8 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:31.197453 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df0fb0-866f-4ec0-a98e-440812e2feb8 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:31.197744 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df0fb0-866f-4ec0-a98e-440812e2feb8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:31.218917 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df0fb0-866f-4ec0-a98e-440812e2feb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-28df0fb0-866f-4ec0-a98e-440812e2feb8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:31.227473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:31.228632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:31.229577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 51/104] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:06:31.237232 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cee6bb-f33f-48c7-9b8c-d27be1f545c9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:31.384210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:31.393716 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cee6bb-f33f-48c7-9b8c-d27be1f545c9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:31.418035 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b653a7-8440-4751-8b18-ce6efeb60b9d None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:31.418336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b653a7-8440-4751-8b18-ce6efeb60b9d None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:31.418628 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b653a7-8440-4751-8b18-ce6efeb60b9d None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:31.420308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 52/105] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:31.427518 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5193125-df5a-4f3b-84ca-5f62c3f3c1df None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:31.432868 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5193125-df5a-4f3b-84ca-5f62c3f3c1df None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:31.432868 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5193125-df5a-4f3b-84ca-5f62c3f3c1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:31.454890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9432fe5-ba2a-4254-9081-7b4a7e961545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:31.456201 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9432fe5-ba2a-4254-9081-7b4a7e961545 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:31.456704 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9432fe5-ba2a-4254-9081-7b4a7e961545 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:31.457430 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9432fe5-ba2a-4254-9081-7b4a7e961545 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:31.458568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9432fe5-ba2a-4254-9081-7b4a7e961545 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:31.459064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9432fe5-ba2a-4254-9081-7b4a7e961545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:31.460970 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9432fe5-ba2a-4254-9081-7b4a7e961545 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:31.470906 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 54/106] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:31.479435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9432fe5-ba2a-4254-9081-7b4a7e961545 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:31.487387 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-e5193125-df5a-4f3b-84ca-5f62c3f3c1df None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:31.490126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-e5193125-df5a-4f3b-84ca-5f62c3f3c1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:31.617094 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 53/107] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:31.643401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5193125-df5a-4f3b-84ca-5f62c3f3c1df None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:31.676360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9432fe5-ba2a-4254-9081-7b4a7e961545 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:31.716777 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8096063-e148-4b23-acf7-1ff226aef15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:31.719743 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096063-e148-4b23-acf7-1ff226aef15c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:31.720498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096063-e148-4b23-acf7-1ff226aef15c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:31.721719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8096063-e148-4b23-acf7-1ff226aef15c None admin] PATH_INFO: `/v3/users/48eaa6ef76d5455891a14a08849eead6` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:31.750990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096063-e148-4b23-acf7-1ff226aef15c None admin] RBAC: Authorizing `identity:get_user(user_id=48eaa6ef76d5455891a14a08849eead6)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:31.753016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8096063-e148-4b23-acf7-1ff226aef15c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:31.755078 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 55/108] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:31 2022] GET /identity/v3/users/48eaa6ef76d5455891a14a08849eead6 => generated 289 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:31.761206 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8096063-e148-4b23-acf7-1ff226aef15c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:31.778044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:31.833244 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5193125-df5a-4f3b-84ca-5f62c3f3c1df None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:31.846778 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:31.879329 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed382e7-9809-4414-a224-099bd10e6c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:31.880160 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed382e7-9809-4414-a224-099bd10e6c90 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:31.880738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed382e7-9809-4414-a224-099bd10e6c90 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:31.881024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed382e7-9809-4414-a224-099bd10e6c90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:31.905891 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed382e7-9809-4414-a224-099bd10e6c90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-4ed382e7-9809-4414-a224-099bd10e6c90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:31.923128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:31.924257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:31.925529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:31.931905 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 54/109] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:06:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:06:31.934381 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704778fb-b951-4c06-8aae-9cee17becdd5 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:31.934672 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704778fb-b951-4c06-8aae-9cee17becdd5 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:31.934957 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704778fb-b951-4c06-8aae-9cee17becdd5 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:31.936244 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 55/110] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:31.945948 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704778fb-b951-4c06-8aae-9cee17becdd5 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:31.971422 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8096063-e148-4b23-acf7-1ff226aef15c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:32.026527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e1cb4-c17d-4034-aadb-7604b126b378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:32.032357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e1cb4-c17d-4034-aadb-7604b126b378 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.032357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e1cb4-c17d-4034-aadb-7604b126b378 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.032357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e1cb4-c17d-4034-aadb-7604b126b378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.184232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704778fb-b951-4c06-8aae-9cee17becdd5 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:32.197807 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e1cb4-c17d-4034-aadb-7604b126b378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:32.199735 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e1cb4-c17d-4034-aadb-7604b126b378 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:32.220739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:32.257975 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761c1fb-1dd0-42f8-8997-2658d78fc5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:32.258671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761c1fb-1dd0-42f8-8997-2658d78fc5ce None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.259000 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761c1fb-1dd0-42f8-8997-2658d78fc5ce None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.259238 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761c1fb-1dd0-42f8-8997-2658d78fc5ce None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.259999 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761c1fb-1dd0-42f8-8997-2658d78fc5ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:32.260311 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761c1fb-1dd0-42f8-8997-2658d78fc5ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:32.264204 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761c1fb-1dd0-42f8-8997-2658d78fc5ce None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:32.274098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 56/111] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:32.275402 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8372d6f-4193-4e65-a194-118f9f56987a None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.275697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8372d6f-4193-4e65-a194-118f9f56987a None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.275973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8372d6f-4193-4e65-a194-118f9f56987a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.293179 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-f8372d6f-4193-4e65-a194-118f9f56987a None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:32.294457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-f8372d6f-4193-4e65-a194-118f9f56987a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:32.321594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:32.369178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 57/112] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:32.370165 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8372d6f-4193-4e65-a194-118f9f56987a None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:32.401266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 56/113] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:32.407102 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fea43-5376-4aed-9737-7c9fe3ae488a None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.407102 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fea43-5376-4aed-9737-7c9fe3ae488a None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.407102 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fea43-5376-4aed-9737-7c9fe3ae488a None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.409083 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 57/114] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:32.409887 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fea43-5376-4aed-9737-7c9fe3ae488a None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:32.482230 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8372d6f-4193-4e65-a194-118f9f56987a None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:32.518977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0232a1-9f80-434c-9f14-7acfa29548de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:32.520015 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0232a1-9f80-434c-9f14-7acfa29548de None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.520320 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0232a1-9f80-434c-9f14-7acfa29548de None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.520971 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0232a1-9f80-434c-9f14-7acfa29548de None admin] PATH_INFO: `/v3/users/48915a5b6f6e4fd8a53e478480a2c94c` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.521892 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0232a1-9f80-434c-9f14-7acfa29548de None admin] RBAC: Authorizing `identity:get_user(user_id=48915a5b6f6e4fd8a53e478480a2c94c)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:32.523375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0232a1-9f80-434c-9f14-7acfa29548de None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:32.525070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 58/115] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:32 2022] GET /identity/v3/users/48915a5b6f6e4fd8a53e478480a2c94c => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:32.530546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0232a1-9f80-434c-9f14-7acfa29548de None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:32.543426 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fea43-5376-4aed-9737-7c9fe3ae488a None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:32.559913 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:32.562365 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:32.587523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8051d769-904e-4c8e-ac49-b5df4579d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:32.588777 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051d769-904e-4c8e-ac49-b5df4579d155 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.589300 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051d769-904e-4c8e-ac49-b5df4579d155 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.589812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8051d769-904e-4c8e-ac49-b5df4579d155 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.602568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8051d769-904e-4c8e-ac49-b5df4579d155 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-8051d769-904e-4c8e-ac49-b5df4579d155 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:32.613285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:32.614369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:32.615307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:32.618118 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 58/116] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:32.619633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54d7e-a206-4098-8bc7-fd44cdf4517c None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.620227 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54d7e-a206-4098-8bc7-fd44cdf4517c None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.620792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec54d7e-a206-4098-8bc7-fd44cdf4517c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.637973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0232a1-9f80-434c-9f14-7acfa29548de None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:32.638795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-dec54d7e-a206-4098-8bc7-fd44cdf4517c None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:32.641024 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-dec54d7e-a206-4098-8bc7-fd44cdf4517c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:32.679145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a8a5f-2ef6-4322-828f-9fdb09e94c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:32.680004 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a8a5f-2ef6-4322-828f-9fdb09e94c90 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.680292 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a8a5f-2ef6-4322-828f-9fdb09e94c90 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.680651 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a8a5f-2ef6-4322-828f-9fdb09e94c90 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.681884 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a8a5f-2ef6-4322-828f-9fdb09e94c90 None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:32.683363 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a8a5f-2ef6-4322-828f-9fdb09e94c90 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:32.689874 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 59/117] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:32 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:32.689874 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a8a5f-2ef6-4322-828f-9fdb09e94c90 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:32.723977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 59/118] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:32.726744 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec54d7e-a206-4098-8bc7-fd44cdf4517c None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:32.801383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:32.808233 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a8a5f-2ef6-4322-828f-9fdb09e94c90 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:32.848736 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec54d7e-a206-4098-8bc7-fd44cdf4517c None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:32.850517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:32.853646 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4c3e7-7ff3-4364-a3fe-e465f8438444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:32.854527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4c3e7-7ff3-4364-a3fe-e465f8438444 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.854816 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4c3e7-7ff3-4364-a3fe-e465f8438444 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.855097 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4c3e7-7ff3-4364-a3fe-e465f8438444 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.856314 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4c3e7-7ff3-4364-a3fe-e465f8438444 None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:32.858291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4c3e7-7ff3-4364-a3fe-e465f8438444 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:32.860747 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 60/119] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:32 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:32.866246 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca4059-6d12-4513-b97b-ab66bbf4aae4 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.866650 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca4059-6d12-4513-b97b-ab66bbf4aae4 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.866951 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca4059-6d12-4513-b97b-ab66bbf4aae4 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.868383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 61/120] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:32.873369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca4059-6d12-4513-b97b-ab66bbf4aae4 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:32.887960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f55da0-1758-4c36-8dca-d9b8e7026134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:32.888994 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f55da0-1758-4c36-8dca-d9b8e7026134 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:32.889293 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f55da0-1758-4c36-8dca-d9b8e7026134 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:32.889581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f55da0-1758-4c36-8dca-d9b8e7026134 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:32.891126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f55da0-1758-4c36-8dca-d9b8e7026134 None admin] RBAC: Adding query filter params (name=member) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:32.891919 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f55da0-1758-4c36-8dca-d9b8e7026134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:32.893439 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f55da0-1758-4c36-8dca-d9b8e7026134 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:32.907699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 60/121] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:32.907699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f55da0-1758-4c36-8dca-d9b8e7026134 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:32.994446 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca4059-6d12-4513-b97b-ab66bbf4aae4 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:33.013627 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f55da0-1758-4c36-8dca-d9b8e7026134 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:33.032460 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19d847-5431-4387-b15f-99c5dbcb13cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:33.033275 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19d847-5431-4387-b15f-99c5dbcb13cb None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:33.033565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19d847-5431-4387-b15f-99c5dbcb13cb None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:33.033850 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19d847-5431-4387-b15f-99c5dbcb13cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:33.034686 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19d847-5431-4387-b15f-99c5dbcb13cb None admin] RBAC: Adding query filter params (role.id=a0e21378e6134985a2ab8a70003fdad9, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a, user.id=48eaa6ef76d5455891a14a08849eead6) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:33.035005 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19d847-5431-4387-b15f-99c5dbcb13cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:33.036596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19d847-5431-4387-b15f-99c5dbcb13cb None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:33.052460 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 62/122] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:32 2022] GET /identity/v3/role_assignments?user.id=48eaa6ef76d5455891a14a08849eead6&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=a0e21378e6134985a2ab8a70003fdad9 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:33.052460 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19d847-5431-4387-b15f-99c5dbcb13cb None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:33.060301 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c8532-9e2a-4d6d-a56c-160dca3bbb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:33.062068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c8532-9e2a-4d6d-a56c-160dca3bbb79 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:33.063252 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c8532-9e2a-4d6d-a56c-160dca3bbb79 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:33.063544 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c8532-9e2a-4d6d-a56c-160dca3bbb79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:33.079345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c8532-9e2a-4d6d-a56c-160dca3bbb79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:33.083604 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-c15c8532-9e2a-4d6d-a56c-160dca3bbb79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:33.091588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:33.098385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 61/123] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:06:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:33.099323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5513d-9043-44fe-a8ad-d36f137128b9 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:33.100178 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5513d-9043-44fe-a8ad-d36f137128b9 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:33.100178 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5513d-9043-44fe-a8ad-d36f137128b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:33.124893 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-93b5513d-9043-44fe-a8ad-d36f137128b9 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:33.127028 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-93b5513d-9043-44fe-a8ad-d36f137128b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:33.153508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:33.177357 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19d847-5431-4387-b15f-99c5dbcb13cb None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:33.214557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 62/124] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:33.215635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5513d-9043-44fe-a8ad-d36f137128b9 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:33.233353 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abae4f-26b1-4d66-835a-9f8baae63a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:33.235377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abae4f-26b1-4d66-835a-9f8baae63a64 None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:33.235914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abae4f-26b1-4d66-835a-9f8baae63a64 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:33.236440 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abae4f-26b1-4d66-835a-9f8baae63a64 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/users/48915a5b6f6e4fd8a53e478480a2c94c/roles/ab7a63cec3a744848972fcbb89ecd0ca` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:33.239199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abae4f-26b1-4d66-835a-9f8baae63a64 None admin] RBAC: Authorizing `identity:create_grant(project_id=003a311d04e34ac4ab05c59a55d5a47a, user_id=48915a5b6f6e4fd8a53e478480a2c94c, role_id=ab7a63cec3a744848972fcbb89ecd0ca)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:33.240709 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abae4f-26b1-4d66-835a-9f8baae63a64 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:33.327845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 63/125] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:06:33 2022] PUT /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/users/48915a5b6f6e4fd8a53e478480a2c94c/roles/ab7a63cec3a744848972fcbb89ecd0ca => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:33.330099 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abae4f-26b1-4d66-835a-9f8baae63a64 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:33.380848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5513d-9043-44fe-a8ad-d36f137128b9 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:33.462501 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d14df-0a87-48c2-ad28-a406b2914286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:33.463409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d14df-0a87-48c2-ad28-a406b2914286 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:33.463701 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d14df-0a87-48c2-ad28-a406b2914286 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:33.463978 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d14df-0a87-48c2-ad28-a406b2914286 None admin] PATH_INFO: `/v3/users/e92111952a254ff58d562324db8f63fb` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:33.471110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4d14df-0a87-48c2-ad28-a406b2914286 None admin] RBAC: Authorizing `identity:get_user(user_id=e92111952a254ff58d562324db8f63fb)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:33.472719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4d14df-0a87-48c2-ad28-a406b2914286 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:33.478344 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 63/126] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:33 2022] GET /identity/v3/users/e92111952a254ff58d562324db8f63fb => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:33.479237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d14df-0a87-48c2-ad28-a406b2914286 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:33.479893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:33.508565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abae4f-26b1-4d66-835a-9f8baae63a64 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:33.528624 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:33.598995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2a7b4-e961-4426-9705-b4b114d5966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:33.598995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2a7b4-e961-4426-9705-b4b114d5966b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:33.598995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2a7b4-e961-4426-9705-b4b114d5966b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:33.598995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2a7b4-e961-4426-9705-b4b114d5966b None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:33.599749 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2a7b4-e961-4426-9705-b4b114d5966b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:33.599749 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2a7b4-e961-4426-9705-b4b114d5966b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:33.599749 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2a7b4-e961-4426-9705-b4b114d5966b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:33.600178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 64/127] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:06:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:33.601009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2a7b4-e961-4426-9705-b4b114d5966b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:33.665507 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d14df-0a87-48c2-ad28-a406b2914286 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:33.711476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41396c46-19a5-4e6f-beda-4b12ac8bed78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:33.712317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41396c46-19a5-4e6f-beda-4b12ac8bed78 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:33.712651 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41396c46-19a5-4e6f-beda-4b12ac8bed78 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:33.712936 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41396c46-19a5-4e6f-beda-4b12ac8bed78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:33.726178 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41396c46-19a5-4e6f-beda-4b12ac8bed78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:33.731762 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2a7b4-e961-4426-9705-b4b114d5966b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-41396c46-19a5-4e6f-beda-4b12ac8bed78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:33.735490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:33.736602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 64/128] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:33.739285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41396c46-19a5-4e6f-beda-4b12ac8bed78 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:33.769710 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d371b4-3f59-4bc4-8250-748270f8aa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:33.770586 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d371b4-3f59-4bc4-8250-748270f8aa9e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:33.770866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d371b4-3f59-4bc4-8250-748270f8aa9e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:33.771143 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d371b4-3f59-4bc4-8250-748270f8aa9e None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:33.772399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d371b4-3f59-4bc4-8250-748270f8aa9e None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:33.773894 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d371b4-3f59-4bc4-8250-748270f8aa9e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:33.775803 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 65/129] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:33 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:33.776520 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d371b4-3f59-4bc4-8250-748270f8aa9e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:33.801148 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:33.860830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:33.864922 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41396c46-19a5-4e6f-beda-4b12ac8bed78 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:33.910529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23457-f176-4bf4-b821-1c355057bd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:33.911475 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23457-f176-4bf4-b821-1c355057bd74 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:33.911768 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23457-f176-4bf4-b821-1c355057bd74 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:33.912058 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23457-f176-4bf4-b821-1c355057bd74 None admin] PATH_INFO: `/v3/users/e92111952a254ff58d562324db8f63fb` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:33.913529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae23457-f176-4bf4-b821-1c355057bd74 None admin] RBAC: Authorizing `identity:get_user(user_id=e92111952a254ff58d562324db8f63fb)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:33.915072 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae23457-f176-4bf4-b821-1c355057bd74 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:33.916906 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 65/130] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:33 2022] GET /identity/v3/users/e92111952a254ff58d562324db8f63fb => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:33.917750 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23457-f176-4bf4-b821-1c355057bd74 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:34.002019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d371b4-3f59-4bc4-8250-748270f8aa9e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:34.053186 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23457-f176-4bf4-b821-1c355057bd74 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:34.096545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d545c-c19e-471f-8664-5915c1835361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:34.098040 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d545c-c19e-471f-8664-5915c1835361 None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:34.098337 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d545c-c19e-471f-8664-5915c1835361 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:34.098626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d545c-c19e-471f-8664-5915c1835361 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/users/e92111952a254ff58d562324db8f63fb/roles/ba04654a403444e4ae54a1a216888fa5` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:34.101570 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d545c-c19e-471f-8664-5915c1835361 None admin] RBAC: Authorizing `identity:create_grant(project_id=003a311d04e34ac4ab05c59a55d5a47a, user_id=e92111952a254ff58d562324db8f63fb, role_id=ba04654a403444e4ae54a1a216888fa5)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:34.103121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d545c-c19e-471f-8664-5915c1835361 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:34.114815 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed7270-beb1-46e3-9bd7-ce56796c306d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:34.115428 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed7270-beb1-46e3-9bd7-ce56796c306d None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:34.115714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed7270-beb1-46e3-9bd7-ce56796c306d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:34.115995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed7270-beb1-46e3-9bd7-ce56796c306d None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:34.116806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed7270-beb1-46e3-9bd7-ce56796c306d None admin] RBAC: Adding query filter params (name=member) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:34.117120 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed7270-beb1-46e3-9bd7-ce56796c306d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:34.118603 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed7270-beb1-46e3-9bd7-ce56796c306d None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:34.130982 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 66/131] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:34.131796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed7270-beb1-46e3-9bd7-ce56796c306d None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:34.227597 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 66/132] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:06:33 2022] PUT /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/users/e92111952a254ff58d562324db8f63fb/roles/ba04654a403444e4ae54a1a216888fa5 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:34.228507 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d545c-c19e-471f-8664-5915c1835361 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:34.238933 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:34.252803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:34.280726 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed7270-beb1-46e3-9bd7-ce56796c306d None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:34.368854 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d545c-c19e-471f-8664-5915c1835361 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:34.391628 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2ee40-9d7b-4771-b38b-7576a9c985ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:34.392561 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2ee40-9d7b-4771-b38b-7576a9c985ea None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:34.392918 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2ee40-9d7b-4771-b38b-7576a9c985ea None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:34.393206 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2ee40-9d7b-4771-b38b-7576a9c985ea None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:34.396771 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2ee40-9d7b-4771-b38b-7576a9c985ea None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:34.398288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2ee40-9d7b-4771-b38b-7576a9c985ea None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:34.400139 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 67/133] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:34 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:34.405185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53e7e-690a-442a-a2da-2a3a62185c6c None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:34.405502 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53e7e-690a-442a-a2da-2a3a62185c6c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:34.405754 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c53e7e-690a-442a-a2da-2a3a62185c6c None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:34.407798 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 68/134] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:34.415634 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c53e7e-690a-442a-a2da-2a3a62185c6c None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:34.419147 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a9c88-94a2-49db-9b15-8f69c69fd430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:34.420930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a9c88-94a2-49db-9b15-8f69c69fd430 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:34.421451 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a9c88-94a2-49db-9b15-8f69c69fd430 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:34.422010 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a9c88-94a2-49db-9b15-8f69c69fd430 None admin] PATH_INFO: `/v3/users/e92111952a254ff58d562324db8f63fb` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:34.423618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a9c88-94a2-49db-9b15-8f69c69fd430 None admin] RBAC: Authorizing `identity:get_user(user_id=e92111952a254ff58d562324db8f63fb)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:34.426584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a9c88-94a2-49db-9b15-8f69c69fd430 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:34.428756 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 67/135] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:34 2022] GET /identity/v3/users/e92111952a254ff58d562324db8f63fb => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:34.431454 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc1fe3-0f1d-4718-bec4-b739230c5412 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:34.432008 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc1fe3-0f1d-4718-bec4-b739230c5412 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:34.432545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc1fe3-0f1d-4718-bec4-b739230c5412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:34.457239 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-78dc1fe3-0f1d-4718-bec4-b739230c5412 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:34.459847 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-78dc1fe3-0f1d-4718-bec4-b739230c5412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:34.551310 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 68/136] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:34.552750 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc1fe3-0f1d-4718-bec4-b739230c5412 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:34.568812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:34.629774 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c53e7e-690a-442a-a2da-2a3a62185c6c None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:34.669570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b28bd-555d-4a7f-91cd-301dbf175fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:34.670060 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b28bd-555d-4a7f-91cd-301dbf175fdd None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:34.670376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b28bd-555d-4a7f-91cd-301dbf175fdd None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:34.671269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b28bd-555d-4a7f-91cd-301dbf175fdd None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/users/e92111952a254ff58d562324db8f63fb/roles/c9fbfe89c48d458c94f2a8a11652fcf2` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:34.676182 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b28bd-555d-4a7f-91cd-301dbf175fdd None admin] RBAC: Authorizing `identity:create_grant(project_id=003a311d04e34ac4ab05c59a55d5a47a, user_id=e92111952a254ff58d562324db8f63fb, role_id=c9fbfe89c48d458c94f2a8a11652fcf2)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:34.677709 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b28bd-555d-4a7f-91cd-301dbf175fdd None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:34.684364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc1fe3-0f1d-4718-bec4-b739230c5412 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:34.722670 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2ef77-427a-4b24-9414-f28b82a2ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:34.723756 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2ef77-427a-4b24-9414-f28b82a2ae71 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:34.724045 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2ef77-427a-4b24-9414-f28b82a2ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:34.724405 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2ef77-427a-4b24-9414-f28b82a2ae71 None admin] PATH_INFO: `/v3/projects/05f4d9d30ea44bf28334370502d103db` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:34.725637 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2ef77-427a-4b24-9414-f28b82a2ae71 None admin] RBAC: Authorizing `identity:get_project(project_id=05f4d9d30ea44bf28334370502d103db)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:34.727208 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f2ef77-427a-4b24-9414-f28b82a2ae71 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:34.729085 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 69/137] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:34 2022] GET /identity/v3/projects/05f4d9d30ea44bf28334370502d103db => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:34.729890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2ef77-427a-4b24-9414-f28b82a2ae71 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:34.752647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 69/138] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:06:34 2022] PUT /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/users/e92111952a254ff58d562324db8f63fb/roles/c9fbfe89c48d458c94f2a8a11652fcf2 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:34.756366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b28bd-555d-4a7f-91cd-301dbf175fdd None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:34.778827 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:34.824440 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:34.852222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f2ef77-427a-4b24-9414-f28b82a2ae71 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:34.900639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080aee4e-de23-4682-a58b-5a5741264f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:34.900639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aee4e-de23-4682-a58b-5a5741264f90 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:34.900639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aee4e-de23-4682-a58b-5a5741264f90 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:34.900639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aee4e-de23-4682-a58b-5a5741264f90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:34.901078 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080aee4e-de23-4682-a58b-5a5741264f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:34.901078 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080aee4e-de23-4682-a58b-5a5741264f90 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:34.903008 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 70/139] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:34.908533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080aee4e-de23-4682-a58b-5a5741264f90 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:34.967167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b28bd-555d-4a7f-91cd-301dbf175fdd None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:35.048102 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080aee4e-de23-4682-a58b-5a5741264f90 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:35.085194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b29dab-6a21-4287-b041-a30724ac3eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:35.086136 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b29dab-6a21-4287-b041-a30724ac3eb7 None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:35.086434 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b29dab-6a21-4287-b041-a30724ac3eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:35.086725 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b29dab-6a21-4287-b041-a30724ac3eb7 None admin] PATH_INFO: `/v3/projects/05f4d9d30ea44bf28334370502d103db/users/e92111952a254ff58d562324db8f63fb/roles/ba04654a403444e4ae54a1a216888fa5` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:35.092765 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab94795b-d1cb-450e-9edd-82428bbaf58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:35.093592 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b29dab-6a21-4287-b041-a30724ac3eb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=05f4d9d30ea44bf28334370502d103db, user_id=e92111952a254ff58d562324db8f63fb, role_id=ba04654a403444e4ae54a1a216888fa5)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:35.093733 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94795b-d1cb-450e-9edd-82428bbaf58d None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:35.094270 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94795b-d1cb-450e-9edd-82428bbaf58d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:35.094664 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab94795b-d1cb-450e-9edd-82428bbaf58d None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:35.095165 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b29dab-6a21-4287-b041-a30724ac3eb7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:35.095849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94795b-d1cb-450e-9edd-82428bbaf58d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:35.099028 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab94795b-d1cb-450e-9edd-82428bbaf58d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:35.173343 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 70/140] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:06:34 2022] PUT /identity/v3/projects/05f4d9d30ea44bf28334370502d103db/users/e92111952a254ff58d562324db8f63fb/roles/ba04654a403444e4ae54a1a216888fa5 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:35.207488 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:35.212962 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 71/141] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:06:34 2022] POST /identity/v3/users => generated 283 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:06:35.982688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cdee1-c4ca-4b26-90b0-f2fe92c9aea0 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:35.983784 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cdee1-c4ca-4b26-90b0-f2fe92c9aea0 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:35.984424 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cdee1-c4ca-4b26-90b0-f2fe92c9aea0 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:35.986160 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 71/142] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:35.992480 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbef05-b977-48ed-b909-65de3281a2c9 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:35.993497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbef05-b977-48ed-b909-65de3281a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:35.994015 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bbef05-b977-48ed-b909-65de3281a2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:36.016033 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-77bbef05-b977-48ed-b909-65de3281a2c9 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:36.017637 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-77bbef05-b977-48ed-b909-65de3281a2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:36.109094 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5773b1-ca43-450a-853b-f682311d72a7 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:36.109493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5773b1-ca43-450a-853b-f682311d72a7 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:36.109690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5773b1-ca43-450a-853b-f682311d72a7 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:36.111077 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 72/143] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:36.116719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b1ba-0b9e-4c90-9f24-d7a89c2f1960 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:36.117009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b1ba-0b9e-4c90-9f24-d7a89c2f1960 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:36.117290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1b1ba-0b9e-4c90-9f24-d7a89c2f1960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:36.122145 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 72/144] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:36.136619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-c0f1b1ba-0b9e-4c90-9f24-d7a89c2f1960 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:36.137883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-c0f1b1ba-0b9e-4c90-9f24-d7a89c2f1960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:36.161846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbef05-b977-48ed-b909-65de3281a2c9 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:36.264798 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:36.272199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 73/145] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:36.320620 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bbef05-b977-48ed-b909-65de3281a2c9 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:36.322237 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b1ba-0b9e-4c90-9f24-d7a89c2f1960 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:36.386199 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:36.401148 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46a2e5-d4d1-401c-9e75-875114bec2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:36.401962 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46a2e5-d4d1-401c-9e75-875114bec2a3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:36.402243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46a2e5-d4d1-401c-9e75-875114bec2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:36.402518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46a2e5-d4d1-401c-9e75-875114bec2a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:36.424061 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46a2e5-d4d1-401c-9e75-875114bec2a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-1a46a2e5-d4d1-401c-9e75-875114bec2a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:36.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:36.433121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:36.434041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:36.440977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 73/146] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:06:36.447401 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46a2e5-d4d1-401c-9e75-875114bec2a3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:36.532806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1b1ba-0b9e-4c90-9f24-d7a89c2f1960 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:36.617674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ce43f-1064-4ac0-8942-38847e0a3641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:36.618902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ce43f-1064-4ac0-8942-38847e0a3641 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:36.619438 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ce43f-1064-4ac0-8942-38847e0a3641 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:36.619933 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ce43f-1064-4ac0-8942-38847e0a3641 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:36.652144 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46a2e5-d4d1-401c-9e75-875114bec2a3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:36.656981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ce43f-1064-4ac0-8942-38847e0a3641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-638ce43f-1064-4ac0-8942-38847e0a3641 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:36.665268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:36.666380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:36.667297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:36.670119 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 74/147] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:06:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:06:36.678768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ce43f-1064-4ac0-8942-38847e0a3641 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:36.708014 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c9b0-0df3-45d4-96c5-1cb3e694be5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:36.708907 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c9b0-0df3-45d4-96c5-1cb3e694be5d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:36.709189 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c9b0-0df3-45d4-96c5-1cb3e694be5d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:36.709473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c9b0-0df3-45d4-96c5-1cb3e694be5d None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:36.710241 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964c9b0-0df3-45d4-96c5-1cb3e694be5d None admin] RBAC: Adding query filter params (name=reader) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:36.710579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964c9b0-0df3-45d4-96c5-1cb3e694be5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:36.711962 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964c9b0-0df3-45d4-96c5-1cb3e694be5d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:36.733774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 74/148] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:36.739867 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c9b0-0df3-45d4-96c5-1cb3e694be5d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:36.815922 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:36.833560 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:36.913479 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c9b0-0df3-45d4-96c5-1cb3e694be5d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:36.914564 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ce43f-1064-4ac0-8942-38847e0a3641 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:37.002480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6d7b2-b376-4562-b6f1-38031bc4af85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:37.003265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d7b2-b376-4562-b6f1-38031bc4af85 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:37.003462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d7b2-b376-4562-b6f1-38031bc4af85 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:37.003740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6d7b2-b376-4562-b6f1-38031bc4af85 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:37.004542 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6d7b2-b376-4562-b6f1-38031bc4af85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:37.004853 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6d7b2-b376-4562-b6f1-38031bc4af85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:37.006431 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6d7b2-b376-4562-b6f1-38031bc4af85 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:37.009077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f1049-a88a-41eb-a5ba-991e9ddf2365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:37.010501 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f1049-a88a-41eb-a5ba-991e9ddf2365 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:37.010948 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f1049-a88a-41eb-a5ba-991e9ddf2365 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:37.011411 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f1049-a88a-41eb-a5ba-991e9ddf2365 None admin] PATH_INFO: `/v3/users/48eaa6ef76d5455891a14a08849eead6` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:37.013063 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f1049-a88a-41eb-a5ba-991e9ddf2365 None admin] RBAC: Authorizing `identity:get_user(user_id=48eaa6ef76d5455891a14a08849eead6)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:37.014769 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f1049-a88a-41eb-a5ba-991e9ddf2365 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:37.017593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 75/149] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:36 2022] GET /identity/v3/users/48eaa6ef76d5455891a14a08849eead6 => generated 289 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:37.022922 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f1049-a88a-41eb-a5ba-991e9ddf2365 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:37.024495 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 75/150] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:37.031296 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6d7b2-b376-4562-b6f1-38031bc4af85 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:37.219573 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f1049-a88a-41eb-a5ba-991e9ddf2365 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:37.257457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:37.280312 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa983b65-ed62-4317-a7fe-390f8664414c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:37.283838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa983b65-ed62-4317-a7fe-390f8664414c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:37.283838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa983b65-ed62-4317-a7fe-390f8664414c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:37.283838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa983b65-ed62-4317-a7fe-390f8664414c None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:37.283838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa983b65-ed62-4317-a7fe-390f8664414c None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:37.290423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa983b65-ed62-4317-a7fe-390f8664414c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:37.292319 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 76/151] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:37 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:37.298398 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa983b65-ed62-4317-a7fe-390f8664414c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:37.303186 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6d7b2-b376-4562-b6f1-38031bc4af85 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:37.361376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:37.407254 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f528a2-f7a9-43a8-9754-0101d0d5a2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:37.408172 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f528a2-f7a9-43a8-9754-0101d0d5a2b5 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:37.408493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f528a2-f7a9-43a8-9754-0101d0d5a2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:37.408779 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f528a2-f7a9-43a8-9754-0101d0d5a2b5 None admin] PATH_INFO: `/v3/users/48915a5b6f6e4fd8a53e478480a2c94c` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:37.413267 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f528a2-f7a9-43a8-9754-0101d0d5a2b5 None admin] RBAC: Authorizing `identity:get_user(user_id=48915a5b6f6e4fd8a53e478480a2c94c)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:37.414790 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f528a2-f7a9-43a8-9754-0101d0d5a2b5 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:37.416984 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 76/152] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:37 2022] GET /identity/v3/users/48915a5b6f6e4fd8a53e478480a2c94c => generated 247 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:37.429497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f528a2-f7a9-43a8-9754-0101d0d5a2b5 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:37.473955 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa983b65-ed62-4317-a7fe-390f8664414c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:37.521639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981e415-7e62-4db1-93a4-3d43e842211e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:37.522655 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981e415-7e62-4db1-93a4-3d43e842211e None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:37.523444 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981e415-7e62-4db1-93a4-3d43e842211e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:37.523444 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981e415-7e62-4db1-93a4-3d43e842211e None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/users/48eaa6ef76d5455891a14a08849eead6/roles/a0e21378e6134985a2ab8a70003fdad9` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:37.529761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981e415-7e62-4db1-93a4-3d43e842211e None admin] RBAC: Authorizing `identity:create_grant(project_id=003a311d04e34ac4ab05c59a55d5a47a, user_id=48eaa6ef76d5455891a14a08849eead6, role_id=a0e21378e6134985a2ab8a70003fdad9)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:37.532177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981e415-7e62-4db1-93a4-3d43e842211e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:37.553173 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f528a2-f7a9-43a8-9754-0101d0d5a2b5 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:37.595757 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f0fba-860e-4692-b237-f195d7b1d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:37.596830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f0fba-860e-4692-b237-f195d7b1d413 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:37.597349 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f0fba-860e-4692-b237-f195d7b1d413 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:37.597840 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f0fba-860e-4692-b237-f195d7b1d413 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:37.620205 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 77/153] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:06:37 2022] PUT /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/users/48eaa6ef76d5455891a14a08849eead6/roles/a0e21378e6134985a2ab8a70003fdad9 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:06:37.667426 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:37.683315 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f0fba-860e-4692-b237-f195d7b1d413 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:37.685165 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f0fba-860e-4692-b237-f195d7b1d413 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:37.803034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 77/154] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:06:37.809197 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981e415-7e62-4db1-93a4-3d43e842211e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:37.872796 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:37.939433 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981e415-7e62-4db1-93a4-3d43e842211e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:37.981953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ea646-4921-45c0-b41e-6b14ca22a3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:37.983286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ea646-4921-45c0-b41e-6b14ca22a3ab None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:37.983770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ea646-4921-45c0-b41e-6b14ca22a3ab None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:37.984298 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ea646-4921-45c0-b41e-6b14ca22a3ab None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:37.985890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ea646-4921-45c0-b41e-6b14ca22a3ab None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:37.987652 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ea646-4921-45c0-b41e-6b14ca22a3ab None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:37.989785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 78/155] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:37 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:37.996096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f0fba-860e-4692-b237-f195d7b1d413 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:38.009694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679514-0c41-492f-a0b4-a3264628f341 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:38.012821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679514-0c41-492f-a0b4-a3264628f341 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:38.013384 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd679514-0c41-492f-a0b4-a3264628f341 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:38.015293 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 79/156] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:38.021863 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe3372-9ff2-41f1-bb0b-f7f6d8c92ac9 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:38.022431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe3372-9ff2-41f1-bb0b-f7f6d8c92ac9 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:38.022900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe3372-9ff2-41f1-bb0b-f7f6d8c92ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:38.059449 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-7fbe3372-9ff2-41f1-bb0b-f7f6d8c92ac9 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:38.063578 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-7fbe3372-9ff2-41f1-bb0b-f7f6d8c92ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:38.188759 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:38.214228 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f0fba-860e-4692-b237-f195d7b1d413 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:38.221901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 80/157] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:38.247868 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe3372-9ff2-41f1-bb0b-f7f6d8c92ac9 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:38.270047 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9626682-3fd6-4de2-a584-b9bd7e052ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:38.270797 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9626682-3fd6-4de2-a584-b9bd7e052ae5 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:38.271088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9626682-3fd6-4de2-a584-b9bd7e052ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:38.271373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9626682-3fd6-4de2-a584-b9bd7e052ae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:38.272242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9626682-3fd6-4de2-a584-b9bd7e052ae5 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a, user.id=48915a5b6f6e4fd8a53e478480a2c94c) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:38.272599 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9626682-3fd6-4de2-a584-b9bd7e052ae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:38.274111 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9626682-3fd6-4de2-a584-b9bd7e052ae5 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:38.285521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 78/158] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:37 2022] GET /identity/v3/role_assignments?user.id=48915a5b6f6e4fd8a53e478480a2c94c&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:38.333828 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:38.407334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe3372-9ff2-41f1-bb0b-f7f6d8c92ac9 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:38.454817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43afe1-b1ba-4fce-a41b-658ef2d116ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:38.455627 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43afe1-b1ba-4fce-a41b-658ef2d116ef None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:38.455919 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43afe1-b1ba-4fce-a41b-658ef2d116ef None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:38.456202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43afe1-b1ba-4fce-a41b-658ef2d116ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:38.481469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43afe1-b1ba-4fce-a41b-658ef2d116ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-0c43afe1-b1ba-4fce-a41b-658ef2d116ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:38.490219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:38.491359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:38.492280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 81/159] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:38.498103 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9626682-3fd6-4de2-a584-b9bd7e052ae5 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:38.659245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9626682-3fd6-4de2-a584-b9bd7e052ae5 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:38.704165 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717ef51-b494-44ae-aea4-aeed88af5dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:38.704999 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717ef51-b494-44ae-aea4-aeed88af5dc2 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:38.705208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717ef51-b494-44ae-aea4-aeed88af5dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:38.705568 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717ef51-b494-44ae-aea4-aeed88af5dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:38.706356 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717ef51-b494-44ae-aea4-aeed88af5dc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:38.706671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717ef51-b494-44ae-aea4-aeed88af5dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:38.708130 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717ef51-b494-44ae-aea4-aeed88af5dc2 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:38.722207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 79/160] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:38.727768 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43afe1-b1ba-4fce-a41b-658ef2d116ef None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:38.891576 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43afe1-b1ba-4fce-a41b-658ef2d116ef None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:38.899284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:38.935344 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb0980-badd-4f7c-ba80-a9ac392b1cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:38.936242 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb0980-badd-4f7c-ba80-a9ac392b1cdf None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:38.936571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb0980-badd-4f7c-ba80-a9ac392b1cdf None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:38.936856 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb0980-badd-4f7c-ba80-a9ac392b1cdf None admin] PATH_INFO: `/v3/users/e92111952a254ff58d562324db8f63fb` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:38.938855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb0980-badd-4f7c-ba80-a9ac392b1cdf None admin] RBAC: Authorizing `identity:get_user(user_id=e92111952a254ff58d562324db8f63fb)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:38.940427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb0980-badd-4f7c-ba80-a9ac392b1cdf None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:38.942693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 82/161] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:38 2022] GET /identity/v3/users/e92111952a254ff58d562324db8f63fb => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:38.948697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717ef51-b494-44ae-aea4-aeed88af5dc2 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:38.972967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:39.098871 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717ef51-b494-44ae-aea4-aeed88af5dc2 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:39.138593 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c217695-ea34-4053-90df-e8ec61f89dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:39.139267 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c217695-ea34-4053-90df-e8ec61f89dd4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:39.139556 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c217695-ea34-4053-90df-e8ec61f89dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:39.139839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c217695-ea34-4053-90df-e8ec61f89dd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:39.214664 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c217695-ea34-4053-90df-e8ec61f89dd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:39.216165 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c217695-ea34-4053-90df-e8ec61f89dd4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:39.233222 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:39.323147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 80/162] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:39.328763 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb0980-badd-4f7c-ba80-a9ac392b1cdf None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:39.451838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb0980-badd-4f7c-ba80-a9ac392b1cdf None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:39.486901 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83245f-2f08-4a2b-901c-04c3d4f73f0b None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:39.487197 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83245f-2f08-4a2b-901c-04c3d4f73f0b None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:39.487450 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83245f-2f08-4a2b-901c-04c3d4f73f0b None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:39.488886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 81/163] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:06:39.495465 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc38853-26aa-42ac-a57c-637273c7536f None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:39.496035 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc38853-26aa-42ac-a57c-637273c7536f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:39.496579 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc38853-26aa-42ac-a57c-637273c7536f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:39.500374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578cfca-2f3a-4c89-8f20-0299fd8c9ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:39.501286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578cfca-2f3a-4c89-8f20-0299fd8c9ab1 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:39.501578 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578cfca-2f3a-4c89-8f20-0299fd8c9ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:39.501856 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578cfca-2f3a-4c89-8f20-0299fd8c9ab1 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:39.506345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578cfca-2f3a-4c89-8f20-0299fd8c9ab1 None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:39.506345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578cfca-2f3a-4c89-8f20-0299fd8c9ab1 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:39.506577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 83/164] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:39 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:39.512484 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578cfca-2f3a-4c89-8f20-0299fd8c9ab1 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:39.541446 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-6fc38853-26aa-42ac-a57c-637273c7536f None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:39.543024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-6fc38853-26aa-42ac-a57c-637273c7536f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:39.566733 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:39.648105 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578cfca-2f3a-4c89-8f20-0299fd8c9ab1 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:39.688780 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b569b-b40b-4f17-91f4-24603221d60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:39.689456 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b569b-b40b-4f17-91f4-24603221d60b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:39.689746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b569b-b40b-4f17-91f4-24603221d60b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:39.690029 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b569b-b40b-4f17-91f4-24603221d60b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:39.691184 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b569b-b40b-4f17-91f4-24603221d60b None admin] RBAC: Adding query filter params (role.id=ba04654a403444e4ae54a1a216888fa5, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a, user.id=e92111952a254ff58d562324db8f63fb) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:39.691524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b569b-b40b-4f17-91f4-24603221d60b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:39.693530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b569b-b40b-4f17-91f4-24603221d60b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:39.700821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 84/165] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:39 2022] GET /identity/v3/role_assignments?user.id=e92111952a254ff58d562324db8f63fb&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:39.714861 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 82/166] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:39.756689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b569b-b40b-4f17-91f4-24603221d60b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:39.893272 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b569b-b40b-4f17-91f4-24603221d60b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:39.907698 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:39.940934 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae32460-a07e-4128-9215-5f68c0bf686a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:39.941759 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae32460-a07e-4128-9215-5f68c0bf686a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:39.942048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae32460-a07e-4128-9215-5f68c0bf686a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:39.942331 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae32460-a07e-4128-9215-5f68c0bf686a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:39.958605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae32460-a07e-4128-9215-5f68c0bf686a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-4ae32460-a07e-4128-9215-5f68c0bf686a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:39.967276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:39.968320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:39.969428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 85/167] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:06:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:39.974468 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc38853-26aa-42ac-a57c-637273c7536f None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:40.047902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:40.116811 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc38853-26aa-42ac-a57c-637273c7536f None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:40.168048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51327243-1ae2-41c8-a6d0-32f53169e74a None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:40.168391 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51327243-1ae2-41c8-a6d0-32f53169e74a None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:40.168645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51327243-1ae2-41c8-a6d0-32f53169e74a None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:40.170948 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 86/168] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:40.173882 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb8985-1c53-4385-90be-3fb37499ae64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:40.174880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8985-1c53-4385-90be-3fb37499ae64 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:40.175168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8985-1c53-4385-90be-3fb37499ae64 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:40.175449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb8985-1c53-4385-90be-3fb37499ae64 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:40.176585 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb8985-1c53-4385-90be-3fb37499ae64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:40.176904 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb8985-1c53-4385-90be-3fb37499ae64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:40.178313 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb8985-1c53-4385-90be-3fb37499ae64 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:40.191772 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 83/169] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:06:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:40.193300 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86efe0d-99a8-4fc0-ade5-b8e54498a50a None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:40.193854 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86efe0d-99a8-4fc0-ade5-b8e54498a50a None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:40.194370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86efe0d-99a8-4fc0-ade5-b8e54498a50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:40.197318 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb8985-1c53-4385-90be-3fb37499ae64 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:40.218388 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-c86efe0d-99a8-4fc0-ade5-b8e54498a50a None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:40.220048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-c86efe0d-99a8-4fc0-ade5-b8e54498a50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:40.375396 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 87/170] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:40.382584 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb8985-1c53-4385-90be-3fb37499ae64 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:40.395137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:40.415014 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86efe0d-99a8-4fc0-ade5-b8e54498a50a None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:40.467343 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c93e1e-4ae1-4240-bcfc-fa07ab5d2275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:40.469639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c93e1e-4ae1-4240-bcfc-fa07ab5d2275 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:40.470910 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c93e1e-4ae1-4240-bcfc-fa07ab5d2275 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:40.471601 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c93e1e-4ae1-4240-bcfc-fa07ab5d2275 None admin] PATH_INFO: `/v3/users/e92111952a254ff58d562324db8f63fb` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:40.474498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c93e1e-4ae1-4240-bcfc-fa07ab5d2275 None admin] RBAC: Authorizing `identity:get_user(user_id=e92111952a254ff58d562324db8f63fb)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:40.481292 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c93e1e-4ae1-4240-bcfc-fa07ab5d2275 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:40.486464 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 84/171] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:40 2022] GET /identity/v3/users/e92111952a254ff58d562324db8f63fb => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:40.491649 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c93e1e-4ae1-4240-bcfc-fa07ab5d2275 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:40.608734 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:40.616996 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86efe0d-99a8-4fc0-ade5-b8e54498a50a None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:40.671935 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afae181-7a82-4cfd-9a9c-b8c4b7e1ed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:40.672706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afae181-7a82-4cfd-9a9c-b8c4b7e1ed12 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:40.672966 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afae181-7a82-4cfd-9a9c-b8c4b7e1ed12 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:40.673242 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afae181-7a82-4cfd-9a9c-b8c4b7e1ed12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:40.684082 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afae181-7a82-4cfd-9a9c-b8c4b7e1ed12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-8afae181-7a82-4cfd-9a9c-b8c4b7e1ed12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:40.692389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:40.693408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:40.694374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 88/172] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:06:40.695627 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb70fb2-f3d3-4a4a-b3c5-433ea2ceba42 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:40.695627 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb70fb2-f3d3-4a4a-b3c5-433ea2ceba42 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:40.695874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb70fb2-f3d3-4a4a-b3c5-433ea2ceba42 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:40.697140 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 89/173] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:40.699421 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c93e1e-4ae1-4240-bcfc-fa07ab5d2275 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:40.702697 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065809b-5050-4bab-8f8f-ac99f532e666 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:40.703849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065809b-5050-4bab-8f8f-ac99f532e666 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:40.704205 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065809b-5050-4bab-8f8f-ac99f532e666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:40.721363 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-8065809b-5050-4bab-8f8f-ac99f532e666 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:40.722646 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-8065809b-5050-4bab-8f8f-ac99f532e666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:40.781642 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58409716-00cc-4bbb-99c7-d382385a08bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:40.782900 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58409716-00cc-4bbb-99c7-d382385a08bd None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:40.782900 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58409716-00cc-4bbb-99c7-d382385a08bd None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:40.782900 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58409716-00cc-4bbb-99c7-d382385a08bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:40.796695 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 90/174] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:40.797597 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065809b-5050-4bab-8f8f-ac99f532e666 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:40.844733 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:40.872771 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58409716-00cc-4bbb-99c7-d382385a08bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:40.875837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58409716-00cc-4bbb-99c7-d382385a08bd None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:40.905431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:40.921886 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065809b-5050-4bab-8f8f-ac99f532e666 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:40.962385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c47350-0cb3-4194-b40d-bb3931be01e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:40.962990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c47350-0cb3-4194-b40d-bb3931be01e6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:40.963281 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c47350-0cb3-4194-b40d-bb3931be01e6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:40.963567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c47350-0cb3-4194-b40d-bb3931be01e6 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:40.964350 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c47350-0cb3-4194-b40d-bb3931be01e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:40.964677 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c47350-0cb3-4194-b40d-bb3931be01e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:40.966229 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c47350-0cb3-4194-b40d-bb3931be01e6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:40.976245 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 91/175] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:40.977979 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109e969-ce4a-4090-abcb-c1b4406af39f None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:40.978337 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109e969-ce4a-4090-abcb-c1b4406af39f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:40.978685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109e969-ce4a-4090-abcb-c1b4406af39f None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:40.980270 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 92/176] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:40.981193 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109e969-ce4a-4090-abcb-c1b4406af39f None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:40.994160 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 85/177] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:40.995044 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58409716-00cc-4bbb-99c7-d382385a08bd None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:41.081427 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:41.103914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58409716-00cc-4bbb-99c7-d382385a08bd None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:41.112099 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109e969-ce4a-4090-abcb-c1b4406af39f None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:41.142408 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18f576-2ab1-42f9-82d3-e717b2632df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:41.143291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18f576-2ab1-42f9-82d3-e717b2632df1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:41.143575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18f576-2ab1-42f9-82d3-e717b2632df1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:41.143853 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18f576-2ab1-42f9-82d3-e717b2632df1 None admin] PATH_INFO: `/v3/users/e92111952a254ff58d562324db8f63fb` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:41.145079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18f576-2ab1-42f9-82d3-e717b2632df1 None admin] RBAC: Authorizing `identity:get_user(user_id=e92111952a254ff58d562324db8f63fb)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:41.146544 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18f576-2ab1-42f9-82d3-e717b2632df1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:41.152513 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 86/178] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:40 2022] GET /identity/v3/users/e92111952a254ff58d562324db8f63fb => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:41.152513 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3563076-b64c-4ea3-91a8-ad1c5d22ea79 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:41.152513 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3563076-b64c-4ea3-91a8-ad1c5d22ea79 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:41.152513 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3563076-b64c-4ea3-91a8-ad1c5d22ea79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:41.154355 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af06927-0e4c-4896-80e2-6f605dd55fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:41.155170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af06927-0e4c-4896-80e2-6f605dd55fb4 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:41.155456 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af06927-0e4c-4896-80e2-6f605dd55fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:41.155733 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af06927-0e4c-4896-80e2-6f605dd55fb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:41.166820 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af06927-0e4c-4896-80e2-6f605dd55fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:41.169671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-e3563076-b64c-4ea3-91a8-ad1c5d22ea79 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:41.171190 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-e3563076-b64c-4ea3-91a8-ad1c5d22ea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-4af06927-0e4c-4896-80e2-6f605dd55fb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:41.175843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:41.176949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 93/179] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:06:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:41.177874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af06927-0e4c-4896-80e2-6f605dd55fb4 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:41.196645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:41.253682 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 87/180] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:41.255419 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325088-c2f9-48e5-a8da-c8d510c19df8 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:41.255768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325088-c2f9-48e5-a8da-c8d510c19df8 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:41.256118 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325088-c2f9-48e5-a8da-c8d510c19df8 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:41.257622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 88/181] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:41.262774 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325088-c2f9-48e5-a8da-c8d510c19df8 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:41.306481 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af06927-0e4c-4896-80e2-6f605dd55fb4 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:41.348876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa07e6b-fc30-401b-9387-8ef47c347893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:41.349771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa07e6b-fc30-401b-9387-8ef47c347893 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:41.350054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa07e6b-fc30-401b-9387-8ef47c347893 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:41.350331 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa07e6b-fc30-401b-9387-8ef47c347893 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:41.351545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa07e6b-fc30-401b-9387-8ef47c347893 None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:41.353052 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa07e6b-fc30-401b-9387-8ef47c347893 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:41.354777 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 94/182] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:41 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:41.360386 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa07e6b-fc30-401b-9387-8ef47c347893 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:41.367330 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:41.404146 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325088-c2f9-48e5-a8da-c8d510c19df8 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:41.454917 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd3d32-2f77-41e5-b34f-9235a3410b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:41.455583 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd3d32-2f77-41e5-b34f-9235a3410b8c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:41.455908 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd3d32-2f77-41e5-b34f-9235a3410b8c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:41.456138 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd3d32-2f77-41e5-b34f-9235a3410b8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:41.456911 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:41.491163 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa07e6b-fc30-401b-9387-8ef47c347893 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:41.540890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d44b9-50e6-450e-8b84-9dae9c2a374b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:41.541973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d44b9-50e6-450e-8b84-9dae9c2a374b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:41.542437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d44b9-50e6-450e-8b84-9dae9c2a374b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:41.542914 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d44b9-50e6-450e-8b84-9dae9c2a374b None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:41.543886 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d44b9-50e6-450e-8b84-9dae9c2a374b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:41.544208 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d44b9-50e6-450e-8b84-9dae9c2a374b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:41.546093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d44b9-50e6-450e-8b84-9dae9c2a374b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:41.550781 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd3d32-2f77-41e5-b34f-9235a3410b8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:41.553145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd3d32-2f77-41e5-b34f-9235a3410b8c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:41.562264 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 95/183] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:06:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:41.563519 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b56cc-1ddd-4fd0-9471-f5ee2fb21f0b None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:41.563817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b56cc-1ddd-4fd0-9471-f5ee2fb21f0b None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:41.564100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599b56cc-1ddd-4fd0-9471-f5ee2fb21f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:41.582706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-599b56cc-1ddd-4fd0-9471-f5ee2fb21f0b None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:41.584163 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-599b56cc-1ddd-4fd0-9471-f5ee2fb21f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:41.645207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:41.673129 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 96/184] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:41.680364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 89/185] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:41.681243 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd3d32-2f77-41e5-b34f-9235a3410b8c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:41.681475 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b56cc-1ddd-4fd0-9471-f5ee2fb21f0b None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:41.760404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:41.800756 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599b56cc-1ddd-4fd0-9471-f5ee2fb21f0b None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:41.806292 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd3d32-2f77-41e5-b34f-9235a3410b8c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:41.839423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcff70-cccb-4345-a1a4-ef0b786eab42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:41.840241 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcff70-cccb-4345-a1a4-ef0b786eab42 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:41.840562 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcff70-cccb-4345-a1a4-ef0b786eab42 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:41.840847 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcff70-cccb-4345-a1a4-ef0b786eab42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:41.847919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbbe73-a489-4801-935f-a0b5728c0566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:41.848777 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbbe73-a489-4801-935f-a0b5728c0566 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:41.849166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbbe73-a489-4801-935f-a0b5728c0566 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:41.849551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbbe73-a489-4801-935f-a0b5728c0566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:41.850702 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbbe73-a489-4801-935f-a0b5728c0566 None admin] RBAC: Adding query filter params (role.id=c9fbfe89c48d458c94f2a8a11652fcf2, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a, user.id=e92111952a254ff58d562324db8f63fb) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:41.851121 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbbe73-a489-4801-935f-a0b5728c0566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:41.852850 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbbe73-a489-4801-935f-a0b5728c0566 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:41.860101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 90/186] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:41 2022] GET /identity/v3/role_assignments?user.id=e92111952a254ff58d562324db8f63fb&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=c9fbfe89c48d458c94f2a8a11652fcf2 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:41.864575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbbe73-a489-4801-935f-a0b5728c0566 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:41.865635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcff70-cccb-4345-a1a4-ef0b786eab42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-bcbcff70-cccb-4345-a1a4-ef0b786eab42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:41.873714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:41.875543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 97/187] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:06:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:41.877317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976f59b-5683-4ad1-bf47-ab139efbcc23 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:41.879268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976f59b-5683-4ad1-bf47-ab139efbcc23 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:41.879268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976f59b-5683-4ad1-bf47-ab139efbcc23 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:41.879268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 98/188] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:41.879268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976f59b-5683-4ad1-bf47-ab139efbcc23 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:41.918839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:42.012704 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbbe73-a489-4801-935f-a0b5728c0566 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:42.023670 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976f59b-5683-4ad1-bf47-ab139efbcc23 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:42.056413 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb92f23-3924-44e9-8fc5-9e556ffc5949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:42.057231 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb92f23-3924-44e9-8fc5-9e556ffc5949 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:42.057511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb92f23-3924-44e9-8fc5-9e556ffc5949 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:42.057787 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb92f23-3924-44e9-8fc5-9e556ffc5949 None admin] PATH_INFO: `/v3/users/172c240c9845422c948a340f9856e6ac` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:42.060897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:42.075280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2190d-9870-4357-afc4-837cb1fcb8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:42.076183 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2190d-9870-4357-afc4-837cb1fcb8e4 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:42.076496 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2190d-9870-4357-afc4-837cb1fcb8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:42.076777 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2190d-9870-4357-afc4-837cb1fcb8e4 None admin] PATH_INFO: `/v3/projects/05f4d9d30ea44bf28334370502d103db` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:42.080632 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2190d-9870-4357-afc4-837cb1fcb8e4 None admin] RBAC: Authorizing `identity:get_project(project_id=05f4d9d30ea44bf28334370502d103db)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:42.081453 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2190d-9870-4357-afc4-837cb1fcb8e4 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:42.082502 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb92f23-3924-44e9-8fc5-9e556ffc5949 None admin] RBAC: Authorizing `identity:get_user(user_id=172c240c9845422c948a340f9856e6ac)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:42.084362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 99/189] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:41 2022] GET /identity/v3/projects/05f4d9d30ea44bf28334370502d103db => generated 312 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:42.084508 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb92f23-3924-44e9-8fc5-9e556ffc5949 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:42.084594 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2190d-9870-4357-afc4-837cb1fcb8e4 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:42.090376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 91/190] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:41 2022] GET /identity/v3/users/172c240c9845422c948a340f9856e6ac => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:42.090376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461107c-3c4a-47f5-968d-61ec3d312f83 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:42.090376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461107c-3c4a-47f5-968d-61ec3d312f83 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:42.090376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461107c-3c4a-47f5-968d-61ec3d312f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:42.116691 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-3461107c-3c4a-47f5-968d-61ec3d312f83 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:42.118402 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-3461107c-3c4a-47f5-968d-61ec3d312f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:42.208433 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 92/191] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:42.209374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461107c-3c4a-47f5-968d-61ec3d312f83 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:42.234363 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2190d-9870-4357-afc4-837cb1fcb8e4 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:42.247810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:42.279193 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ca25e-209e-4487-8dbf-67c621100056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:42.280008 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca25e-209e-4487-8dbf-67c621100056 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:42.280291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca25e-209e-4487-8dbf-67c621100056 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:42.280675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ca25e-209e-4487-8dbf-67c621100056 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:42.281962 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ca25e-209e-4487-8dbf-67c621100056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:42.283470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ca25e-209e-4487-8dbf-67c621100056 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:42.285196 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 100/192] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:42.285993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ca25e-209e-4487-8dbf-67c621100056 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:42.340212 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461107c-3c4a-47f5-968d-61ec3d312f83 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:42.359681 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:42.377132 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028f4de-564a-429e-8e89-0c533dea53c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:42.377811 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028f4de-564a-429e-8e89-0c533dea53c9 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:42.378097 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028f4de-564a-429e-8e89-0c533dea53c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:42.378374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028f4de-564a-429e-8e89-0c533dea53c9 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:42.379166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028f4de-564a-429e-8e89-0c533dea53c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:42.379476 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028f4de-564a-429e-8e89-0c533dea53c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:42.380946 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028f4de-564a-429e-8e89-0c533dea53c9 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:42.390182 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 93/193] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:42.391697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028f4de-564a-429e-8e89-0c533dea53c9 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:42.409842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ca25e-209e-4487-8dbf-67c621100056 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:42.447495 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367169a-95c6-4976-b0cc-12c63755a4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:42.448210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367169a-95c6-4976-b0cc-12c63755a4e3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:42.448521 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367169a-95c6-4976-b0cc-12c63755a4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:42.448804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367169a-95c6-4976-b0cc-12c63755a4e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:42.449643 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367169a-95c6-4976-b0cc-12c63755a4e3 None admin] RBAC: Adding query filter params (role.id=ba04654a403444e4ae54a1a216888fa5, scope.project.id=05f4d9d30ea44bf28334370502d103db, user.id=e92111952a254ff58d562324db8f63fb) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:42.449960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367169a-95c6-4976-b0cc-12c63755a4e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:42.451425 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367169a-95c6-4976-b0cc-12c63755a4e3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:42.458541 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 101/194] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:42 2022] GET /identity/v3/role_assignments?user.id=e92111952a254ff58d562324db8f63fb&scope.project.id=05f4d9d30ea44bf28334370502d103db&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:42.459488 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367169a-95c6-4976-b0cc-12c63755a4e3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:42.496157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:42.517635 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028f4de-564a-429e-8e89-0c533dea53c9 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:42.573715 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eb17b-fa46-46a4-b807-bb6448830f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:42.574596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eb17b-fa46-46a4-b807-bb6448830f66 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:42.575071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eb17b-fa46-46a4-b807-bb6448830f66 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:42.575561 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639eb17b-fa46-46a4-b807-bb6448830f66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:42.595894 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367169a-95c6-4976-b0cc-12c63755a4e3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:42.631844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:42.644712 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e6da4-cde0-480b-a171-75055e2d0bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:42.645560 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e6da4-cde0-480b-a171-75055e2d0bc9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:42.645944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e6da4-cde0-480b-a171-75055e2d0bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:42.646247 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e6da4-cde0-480b-a171-75055e2d0bc9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:42.658963 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e6da4-cde0-480b-a171-75055e2d0bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-e02e6da4-cde0-480b-a171-75055e2d0bc9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:42.667364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:42.668949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:42.670289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:42.675689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639eb17b-fa46-46a4-b807-bb6448830f66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:42.676754 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 102/195] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:42.677484 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639eb17b-fa46-46a4-b807-bb6448830f66 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:42.678473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e6da4-cde0-480b-a171-75055e2d0bc9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:42.808890 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:42.816434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e6da4-cde0-480b-a171-75055e2d0bc9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:42.825937 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 94/196] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:42.830867 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eb17b-fa46-46a4-b807-bb6448830f66 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:42.856068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24b42-e4f0-4185-9732-ac176a45a1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:42.856707 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24b42-e4f0-4185-9732-ac176a45a1cb None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:42.857014 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24b42-e4f0-4185-9732-ac176a45a1cb None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:42.857297 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24b42-e4f0-4185-9732-ac176a45a1cb None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:42.858481 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b24b42-e4f0-4185-9732-ac176a45a1cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:42.861031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b24b42-e4f0-4185-9732-ac176a45a1cb None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:42.937307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639eb17b-fa46-46a4-b807-bb6448830f66 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:42.971692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d6f05-fedb-4280-ac75-ca19d6e4eecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:42.972635 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d6f05-fedb-4280-ac75-ca19d6e4eecf None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:42.972914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d6f05-fedb-4280-ac75-ca19d6e4eecf None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:42.973188 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d6f05-fedb-4280-ac75-ca19d6e4eecf None admin] PATH_INFO: `/v3/users/172c240c9845422c948a340f9856e6ac` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:42.974378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d6f05-fedb-4280-ac75-ca19d6e4eecf None admin] RBAC: Authorizing `identity:get_user(user_id=172c240c9845422c948a340f9856e6ac)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:42.975861 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d6f05-fedb-4280-ac75-ca19d6e4eecf None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:42.977538 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 95/197] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:42 2022] GET /identity/v3/users/172c240c9845422c948a340f9856e6ac => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:42.978299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d6f05-fedb-4280-ac75-ca19d6e4eecf None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:43.001646 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 103/198] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:06:42 2022] POST /identity/v3/users => generated 297 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:43.007009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24b42-e4f0-4185-9732-ac176a45a1cb None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:43.045897 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:43.074565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:43.110672 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d6f05-fedb-4280-ac75-ca19d6e4eecf None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:43.146636 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24b42-e4f0-4185-9732-ac176a45a1cb None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:43.152431 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f913c-7e16-4e5d-93b8-43d3f51d6f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:43.153319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f913c-7e16-4e5d-93b8-43d3f51d6f3e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:43.153407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f913c-7e16-4e5d-93b8-43d3f51d6f3e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:43.153675 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f913c-7e16-4e5d-93b8-43d3f51d6f3e None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:43.154532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f913c-7e16-4e5d-93b8-43d3f51d6f3e None admin] RBAC: Adding query filter params (name=reader) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:43.155589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f913c-7e16-4e5d-93b8-43d3f51d6f3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:43.157230 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f913c-7e16-4e5d-93b8-43d3f51d6f3e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:43.168281 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 96/199] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:43.169141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f913c-7e16-4e5d-93b8-43d3f51d6f3e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:43.193621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880250b-7f8d-4a0c-8734-a2d885bd60f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:43.194518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880250b-7f8d-4a0c-8734-a2d885bd60f7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:43.194800 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880250b-7f8d-4a0c-8734-a2d885bd60f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:43.195074 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880250b-7f8d-4a0c-8734-a2d885bd60f7 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:43.196396 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880250b-7f8d-4a0c-8734-a2d885bd60f7 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:43.197879 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7880250b-7f8d-4a0c-8734-a2d885bd60f7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:43.199696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 104/200] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:43 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:43.202042 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880250b-7f8d-4a0c-8734-a2d885bd60f7 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:43.315013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f913c-7e16-4e5d-93b8-43d3f51d6f3e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:43.338296 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880250b-7f8d-4a0c-8734-a2d885bd60f7 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:43.340490 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:43.367951 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:43.384032 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e16723-3f9d-4c80-a8f4-d7f2fe2e2180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:43.384901 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e16723-3f9d-4c80-a8f4-d7f2fe2e2180 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:43.385187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e16723-3f9d-4c80-a8f4-d7f2fe2e2180 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:43.385468 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e16723-3f9d-4c80-a8f4-d7f2fe2e2180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:43.406949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c49658-f8c1-4881-9784-18c4dd34fefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:43.408244 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c49658-f8c1-4881-9784-18c4dd34fefa None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:43.408752 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c49658-f8c1-4881-9784-18c4dd34fefa None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:43.409215 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c49658-f8c1-4881-9784-18c4dd34fefa None admin] PATH_INFO: `/v3/users/48eaa6ef76d5455891a14a08849eead6` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:43.411314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c49658-f8c1-4881-9784-18c4dd34fefa None admin] RBAC: Authorizing `identity:get_user(user_id=48eaa6ef76d5455891a14a08849eead6)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:43.413730 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c49658-f8c1-4881-9784-18c4dd34fefa None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:43.417082 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 105/201] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:43 2022] GET /identity/v3/users/48eaa6ef76d5455891a14a08849eead6 => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:43.418491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c49658-f8c1-4881-9784-18c4dd34fefa None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:43.509106 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e16723-3f9d-4c80-a8f4-d7f2fe2e2180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:43.511885 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e16723-3f9d-4c80-a8f4-d7f2fe2e2180 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:43.572055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c49658-f8c1-4881-9784-18c4dd34fefa None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:43.622125 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd9a09-11e1-4433-afe4-c9f2208cb488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:43.622852 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd9a09-11e1-4433-afe4-c9f2208cb488 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:43.623407 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd9a09-11e1-4433-afe4-c9f2208cb488 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:43.623758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd9a09-11e1-4433-afe4-c9f2208cb488 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:43.624758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd9a09-11e1-4433-afe4-c9f2208cb488 None admin] RBAC: Adding query filter params (role.id=c9fbfe89c48d458c94f2a8a11652fcf2, scope.project.id=8ff5a20e721548c582f2c7a214733faf, user.id=172c240c9845422c948a340f9856e6ac) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:43.625150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd9a09-11e1-4433-afe4-c9f2208cb488 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:43.626807 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd9a09-11e1-4433-afe4-c9f2208cb488 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:43.633985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 106/202] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:43 2022] GET /identity/v3/role_assignments?user.id=172c240c9845422c948a340f9856e6ac&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=c9fbfe89c48d458c94f2a8a11652fcf2 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:43.634992 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd9a09-11e1-4433-afe4-c9f2208cb488 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:43.658664 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 97/203] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:43.669519 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e16723-3f9d-4c80-a8f4-d7f2fe2e2180 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:43.702411 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:43.708885 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:43.810950 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd9a09-11e1-4433-afe4-c9f2208cb488 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:43.874616 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a17c-a396-4158-8d9c-8b9663aa3076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:43.875540 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a17c-a396-4158-8d9c-8b9663aa3076 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:43.875853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a17c-a396-4158-8d9c-8b9663aa3076 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:43.876135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a17c-a396-4158-8d9c-8b9663aa3076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:43.894070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e16723-3f9d-4c80-a8f4-d7f2fe2e2180 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:43.968159 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c6021-c25e-4f76-b3df-fba84b62906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:43.969709 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6021-c25e-4f76-b3df-fba84b62906b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:43.970078 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6021-c25e-4f76-b3df-fba84b62906b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:43.970440 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c6021-c25e-4f76-b3df-fba84b62906b None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:43.973125 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c6021-c25e-4f76-b3df-fba84b62906b None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:43.974621 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c6021-c25e-4f76-b3df-fba84b62906b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:43.980438 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 98/204] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:43 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:43.993410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c6021-c25e-4f76-b3df-fba84b62906b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:44.000618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0a17c-a396-4158-8d9c-8b9663aa3076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:44.002318 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0a17c-a396-4158-8d9c-8b9663aa3076 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:44.084513 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:44.156142 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:44.200678 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c6021-c25e-4f76-b3df-fba84b62906b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:44.213298 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 107/205] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:44.224022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a17c-a396-4158-8d9c-8b9663aa3076 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:44.262169 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fafd0-1a8d-44ae-beb5-d46212eef093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:44.262845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fafd0-1a8d-44ae-beb5-d46212eef093 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:44.263135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fafd0-1a8d-44ae-beb5-d46212eef093 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:44.263412 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fafd0-1a8d-44ae-beb5-d46212eef093 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:44.264272 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fafd0-1a8d-44ae-beb5-d46212eef093 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=8ff5a20e721548c582f2c7a214733faf, user.id=172c240c9845422c948a340f9856e6ac) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:44.264623 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fafd0-1a8d-44ae-beb5-d46212eef093 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:44.266136 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fafd0-1a8d-44ae-beb5-d46212eef093 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:44.273261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 99/206] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:43 2022] GET /identity/v3/role_assignments?user.id=172c240c9845422c948a340f9856e6ac&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:44.405554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a17c-a396-4158-8d9c-8b9663aa3076 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:44.452084 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f36f1d-13d2-46ad-9dad-fb48df8c3cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:44.453148 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f36f1d-13d2-46ad-9dad-fb48df8c3cac None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:44.453473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f36f1d-13d2-46ad-9dad-fb48df8c3cac None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:44.453752 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f36f1d-13d2-46ad-9dad-fb48df8c3cac None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:44.455101 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f36f1d-13d2-46ad-9dad-fb48df8c3cac None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:44.456639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f36f1d-13d2-46ad-9dad-fb48df8c3cac None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:44.458553 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 108/207] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:44 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:44.465067 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fafd0-1a8d-44ae-beb5-d46212eef093 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:44.627698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fafd0-1a8d-44ae-beb5-d46212eef093 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:44.671501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce72e3-cc9c-4604-bcbf-72a0663c3f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:44.673531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce72e3-cc9c-4604-bcbf-72a0663c3f06 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:44.674404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce72e3-cc9c-4604-bcbf-72a0663c3f06 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:44.675209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce72e3-cc9c-4604-bcbf-72a0663c3f06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:44.677160 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce72e3-cc9c-4604-bcbf-72a0663c3f06 None admin] RBAC: Adding query filter params (role.id=a0e21378e6134985a2ab8a70003fdad9, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a, user.id=48eaa6ef76d5455891a14a08849eead6) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:44.677867 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce72e3-cc9c-4604-bcbf-72a0663c3f06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:44.679658 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce72e3-cc9c-4604-bcbf-72a0663c3f06 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:44.689208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 100/208] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:44 2022] GET /identity/v3/role_assignments?user.id=48eaa6ef76d5455891a14a08849eead6&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=a0e21378e6134985a2ab8a70003fdad9 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:45.755497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791b3a1-c486-4e81-8373-6e9d5f000ecf None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:45.756382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791b3a1-c486-4e81-8373-6e9d5f000ecf None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:45.756382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791b3a1-c486-4e81-8373-6e9d5f000ecf None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:45.757530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 109/209] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:06:45.763658 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564be47-5eba-4cd1-8bf8-5f8f986a7314 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:45.764748 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564be47-5eba-4cd1-8bf8-5f8f986a7314 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:45.764748 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564be47-5eba-4cd1-8bf8-5f8f986a7314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:45.791921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-8564be47-5eba-4cd1-8bf8-5f8f986a7314 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:45.794412 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-8564be47-5eba-4cd1-8bf8-5f8f986a7314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:45.810252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:45.933393 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 101/210] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:45.959192 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791b3a1-c486-4e81-8373-6e9d5f000ecf None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:45.975009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:46.077465 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791b3a1-c486-4e81-8373-6e9d5f000ecf None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:46.120574 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f376da-ab1e-41a2-ba28-05deef82eee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:46.121529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f376da-ab1e-41a2-ba28-05deef82eee9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:46.121814 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f376da-ab1e-41a2-ba28-05deef82eee9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:46.122091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f376da-ab1e-41a2-ba28-05deef82eee9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:46.130510 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d0302-8f02-48d9-9aa2-e01e266c102c None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:46.131545 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d0302-8f02-48d9-9aa2-e01e266c102c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:46.132317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d0302-8f02-48d9-9aa2-e01e266c102c None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:46.135537 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 102/211] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:46.140563 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f376da-ab1e-41a2-ba28-05deef82eee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-82f376da-ab1e-41a2-ba28-05deef82eee9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:46.144011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:46.146075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:46.147905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 110/212] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:06:46.151337 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f376da-ab1e-41a2-ba28-05deef82eee9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:46.152479 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f25ce9-3248-437d-b7cf-a3cf532c7670 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:46.153467 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f25ce9-3248-437d-b7cf-a3cf532c7670 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:46.154299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f25ce9-3248-437d-b7cf-a3cf532c7670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:46.186399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-34f25ce9-3248-437d-b7cf-a3cf532c7670 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:46.189395 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-34f25ce9-3248-437d-b7cf-a3cf532c7670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:46.241036 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:46.289700 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f376da-ab1e-41a2-ba28-05deef82eee9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:46.304524 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 103/213] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:46.306226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905098ed-54b3-48c9-b110-2ef4a6013aee None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:46.306519 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905098ed-54b3-48c9-b110-2ef4a6013aee None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:46.306800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905098ed-54b3-48c9-b110-2ef4a6013aee None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:46.308379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 104/214] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:46.314468 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1622f-f7dd-4248-95d5-27dbbf10c918 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:46.314806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1622f-f7dd-4248-95d5-27dbbf10c918 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:46.315046 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1622f-f7dd-4248-95d5-27dbbf10c918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:46.341441 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-05b1622f-f7dd-4248-95d5-27dbbf10c918 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:46.343393 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-05b1622f-f7dd-4248-95d5-27dbbf10c918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:46.360548 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b88388-1a75-4b94-9197-b72b236d053f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:46.361475 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b88388-1a75-4b94-9197-b72b236d053f None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:46.361909 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b88388-1a75-4b94-9197-b72b236d053f None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:46.362355 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b88388-1a75-4b94-9197-b72b236d053f None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:46.363324 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b88388-1a75-4b94-9197-b72b236d053f None admin] RBAC: Adding query filter params (name=member) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:46.364555 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b88388-1a75-4b94-9197-b72b236d053f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:46.367683 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b88388-1a75-4b94-9197-b72b236d053f None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:46.389530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 111/215] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:46.389530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b88388-1a75-4b94-9197-b72b236d053f None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:46.459986 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 105/216] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:46.461110 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1622f-f7dd-4248-95d5-27dbbf10c918 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:46.464263 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:46.551189 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b88388-1a75-4b94-9197-b72b236d053f None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:46.602839 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ef326-673f-47a3-8f8d-af407fca006e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:46.604013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ef326-673f-47a3-8f8d-af407fca006e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:46.604554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ef326-673f-47a3-8f8d-af407fca006e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:46.605052 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ef326-673f-47a3-8f8d-af407fca006e None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:46.606578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1622f-f7dd-4248-95d5-27dbbf10c918 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:46.606896 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ef326-673f-47a3-8f8d-af407fca006e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:46.608830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ef326-673f-47a3-8f8d-af407fca006e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:46.617210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 112/217] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:46.617210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ef326-673f-47a3-8f8d-af407fca006e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:46.617210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:46.653896 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0068eab-cc4d-4f19-9f82-00b1fcdc973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:46.654889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0068eab-cc4d-4f19-9f82-00b1fcdc973f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:46.655174 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0068eab-cc4d-4f19-9f82-00b1fcdc973f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:46.655449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0068eab-cc4d-4f19-9f82-00b1fcdc973f None admin] PATH_INFO: `/v3/users/7741e15112f64b92b6959796f71ab109` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:46.672668 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0068eab-cc4d-4f19-9f82-00b1fcdc973f None admin] RBAC: Authorizing `identity:get_user(user_id=7741e15112f64b92b6959796f71ab109)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:46.674205 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0068eab-cc4d-4f19-9f82-00b1fcdc973f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:46.675966 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 106/218] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:46 2022] GET /identity/v3/users/7741e15112f64b92b6959796f71ab109 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:46.677491 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0068eab-cc4d-4f19-9f82-00b1fcdc973f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:46.745928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ef326-673f-47a3-8f8d-af407fca006e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:46.751121 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:46.785190 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f068c-f2fb-4b42-b9ca-8ef95059a39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:46.786228 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f068c-f2fb-4b42-b9ca-8ef95059a39c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:46.786347 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f068c-f2fb-4b42-b9ca-8ef95059a39c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:46.786555 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f068c-f2fb-4b42-b9ca-8ef95059a39c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:46.800810 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f068c-f2fb-4b42-b9ca-8ef95059a39c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:46.805339 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0068eab-cc4d-4f19-9f82-00b1fcdc973f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-c50f068c-f2fb-4b42-b9ca-8ef95059a39c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:46.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:46.810618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:46.811623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 113/219] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:06:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:46.813651 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f068c-f2fb-4b42-b9ca-8ef95059a39c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:46.856017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5788f-7900-4440-8fa5-4e154eeb3bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:46.856666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5788f-7900-4440-8fa5-4e154eeb3bdc None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:46.856950 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5788f-7900-4440-8fa5-4e154eeb3bdc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:46.857228 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5788f-7900-4440-8fa5-4e154eeb3bdc None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:46.857960 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5788f-7900-4440-8fa5-4e154eeb3bdc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:46.859518 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5788f-7900-4440-8fa5-4e154eeb3bdc None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:46.889648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:46.933147 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f068c-f2fb-4b42-b9ca-8ef95059a39c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:46.968801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b314da3-6973-46bb-a120-8d972322d180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:46.969532 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314da3-6973-46bb-a120-8d972322d180 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:46.969842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314da3-6973-46bb-a120-8d972322d180 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:46.970122 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b314da3-6973-46bb-a120-8d972322d180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:47.007202 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 107/220] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:06:46 2022] POST /identity/v3/users => generated 297 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:47.008492 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5788f-7900-4440-8fa5-4e154eeb3bdc None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:47.063211 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b314da3-6973-46bb-a120-8d972322d180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:47.065309 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b314da3-6973-46bb-a120-8d972322d180 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:47.151949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:47.181941 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5788f-7900-4440-8fa5-4e154eeb3bdc None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:47.206273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:47.206823 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 114/221] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:47.211974 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b314da3-6973-46bb-a120-8d972322d180 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:47.239552 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6160220-3dd2-44e0-a686-6805e913dc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:47.241994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160220-3dd2-44e0-a686-6805e913dc5d None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:47.242616 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160220-3dd2-44e0-a686-6805e913dc5d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:47.243873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6160220-3dd2-44e0-a686-6805e913dc5d None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:47.244518 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6160220-3dd2-44e0-a686-6805e913dc5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:47.245145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6160220-3dd2-44e0-a686-6805e913dc5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:47.248040 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6160220-3dd2-44e0-a686-6805e913dc5d None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:47.259132 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 108/222] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:06:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:47.265233 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6160220-3dd2-44e0-a686-6805e913dc5d None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:47.399180 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b314da3-6973-46bb-a120-8d972322d180 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:47.412822 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6160220-3dd2-44e0-a686-6805e913dc5d None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:47.459037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a609a1-0bc8-459a-be9c-cfa834a47bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:47.459925 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a609a1-0bc8-459a-be9c-cfa834a47bbf None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:47.460210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a609a1-0bc8-459a-be9c-cfa834a47bbf None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:47.460520 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a609a1-0bc8-459a-be9c-cfa834a47bbf None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:47.461746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a609a1-0bc8-459a-be9c-cfa834a47bbf None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:47.463217 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a609a1-0bc8-459a-be9c-cfa834a47bbf None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:47.463938 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ffb5b-3cbf-4b05-93b4-955b97de134e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:47.465125 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ffb5b-3cbf-4b05-93b4-955b97de134e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:47.465361 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 115/223] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:47 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:47.465825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ffb5b-3cbf-4b05-93b4-955b97de134e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:47.466152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ffb5b-3cbf-4b05-93b4-955b97de134e None admin] PATH_INFO: `/v3/users/172c240c9845422c948a340f9856e6ac` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:47.467544 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ffb5b-3cbf-4b05-93b4-955b97de134e None admin] RBAC: Authorizing `identity:get_user(user_id=172c240c9845422c948a340f9856e6ac)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:47.469777 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ffb5b-3cbf-4b05-93b4-955b97de134e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:47.471530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a609a1-0bc8-459a-be9c-cfa834a47bbf None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:47.471883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 109/224] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:47 2022] GET /identity/v3/users/172c240c9845422c948a340f9856e6ac => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:47.477773 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ffb5b-3cbf-4b05-93b4-955b97de134e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:47.517090 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:47.569810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:47.616024 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a609a1-0bc8-459a-be9c-cfa834a47bbf None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:47.658487 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884397de-978d-4234-8e8e-259fe7a60c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:47.660067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884397de-978d-4234-8e8e-259fe7a60c4d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:47.660582 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884397de-978d-4234-8e8e-259fe7a60c4d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:47.661014 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884397de-978d-4234-8e8e-259fe7a60c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:47.662127 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884397de-978d-4234-8e8e-259fe7a60c4d None admin] RBAC: Adding query filter params (role.id=ba04654a403444e4ae54a1a216888fa5, scope.project.id=8ff5a20e721548c582f2c7a214733faf, user.id=7741e15112f64b92b6959796f71ab109) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:47.662600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884397de-978d-4234-8e8e-259fe7a60c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:47.665112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884397de-978d-4234-8e8e-259fe7a60c4d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:47.673789 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 116/225] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:47 2022] GET /identity/v3/role_assignments?user.id=7741e15112f64b92b6959796f71ab109&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:47.675880 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bc122-c523-449c-b317-c15934d83617 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:47.676429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bc122-c523-449c-b317-c15934d83617 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:47.676922 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bc122-c523-449c-b317-c15934d83617 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:47.679774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 117/226] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:47.690476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad3cd7-4bd2-4e49-b4cf-c5381cfa23a6 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:47.690906 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad3cd7-4bd2-4e49-b4cf-c5381cfa23a6 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:47.691332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad3cd7-4bd2-4e49-b4cf-c5381cfa23a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:47.700275 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ffb5b-3cbf-4b05-93b4-955b97de134e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:47.753910 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-61ad3cd7-4bd2-4e49-b4cf-c5381cfa23a6 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:47.755158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-61ad3cd7-4bd2-4e49-b4cf-c5381cfa23a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:47.805138 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1facf-418a-4f5c-b452-f9683845f398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:47.806587 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1facf-418a-4f5c-b452-f9683845f398 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:47.807135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1facf-418a-4f5c-b452-f9683845f398 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:47.807441 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1facf-418a-4f5c-b452-f9683845f398 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:47.809916 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1facf-418a-4f5c-b452-f9683845f398 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:47.813841 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1facf-418a-4f5c-b452-f9683845f398 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:47.817489 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 110/227] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:47 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:47.826045 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1facf-418a-4f5c-b452-f9683845f398 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:47.917389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 118/228] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:47.955324 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad3cd7-4bd2-4e49-b4cf-c5381cfa23a6 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:47.991350 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:48.067375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1facf-418a-4f5c-b452-f9683845f398 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:48.112377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b3812-ff1c-45ae-9016-a321a14d147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:48.112377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b3812-ff1c-45ae-9016-a321a14d147c None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:48.112377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b3812-ff1c-45ae-9016-a321a14d147c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:48.112377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b3812-ff1c-45ae-9016-a321a14d147c None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf/users/172c240c9845422c948a340f9856e6ac/roles/c9fbfe89c48d458c94f2a8a11652fcf2` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:48.112836 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:48.114528 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad3cd7-4bd2-4e49-b4cf-c5381cfa23a6 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:48.117827 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b3812-ff1c-45ae-9016-a321a14d147c None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5a20e721548c582f2c7a214733faf, user_id=172c240c9845422c948a340f9856e6ac, role_id=c9fbfe89c48d458c94f2a8a11652fcf2)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:48.119358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b3812-ff1c-45ae-9016-a321a14d147c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:48.162490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6042f4-f4b5-42cc-accc-79f1ba4849b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:48.163298 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6042f4-f4b5-42cc-accc-79f1ba4849b0 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:48.163575 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6042f4-f4b5-42cc-accc-79f1ba4849b0 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:48.163846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6042f4-f4b5-42cc-accc-79f1ba4849b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:48.181163 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6042f4-f4b5-42cc-accc-79f1ba4849b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-4f6042f4-f4b5-42cc-accc-79f1ba4849b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:48.185674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:48.187290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:48.188457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 119/229] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:06:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:48.193537 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6042f4-f4b5-42cc-accc-79f1ba4849b0 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:48.206697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 111/230] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:06:47 2022] PUT /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf/users/172c240c9845422c948a340f9856e6ac/roles/c9fbfe89c48d458c94f2a8a11652fcf2 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:48.313515 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:48.334541 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6042f4-f4b5-42cc-accc-79f1ba4849b0 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:48.373330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fb099-eed8-4641-a870-205db7c0e3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:48.374022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fb099-eed8-4641-a870-205db7c0e3aa None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:48.374306 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fb099-eed8-4641-a870-205db7c0e3aa None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:48.374675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fb099-eed8-4641-a870-205db7c0e3aa None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:48.375609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282fb099-eed8-4641-a870-205db7c0e3aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:48.375980 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282fb099-eed8-4641-a870-205db7c0e3aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:48.377692 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282fb099-eed8-4641-a870-205db7c0e3aa None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:48.388394 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 120/231] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:48.394687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b3812-ff1c-45ae-9016-a321a14d147c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:48.533460 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b3812-ff1c-45ae-9016-a321a14d147c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:48.576105 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a9f46-69a0-4ec9-9603-55798ad9095a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:48.577043 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a9f46-69a0-4ec9-9603-55798ad9095a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:48.577339 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a9f46-69a0-4ec9-9603-55798ad9095a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:48.577622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a9f46-69a0-4ec9-9603-55798ad9095a None admin] PATH_INFO: `/v3/users/172c240c9845422c948a340f9856e6ac` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:48.578966 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a9f46-69a0-4ec9-9603-55798ad9095a None admin] RBAC: Authorizing `identity:get_user(user_id=172c240c9845422c948a340f9856e6ac)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:48.580526 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a9f46-69a0-4ec9-9603-55798ad9095a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:48.582388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 112/232] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:48 2022] GET /identity/v3/users/172c240c9845422c948a340f9856e6ac => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:48.588315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fb099-eed8-4641-a870-205db7c0e3aa None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:48.715129 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fb099-eed8-4641-a870-205db7c0e3aa None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:48.752456 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1557e-d077-45cf-a9fe-4cc0f9c90619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:48.753339 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1557e-d077-45cf-a9fe-4cc0f9c90619 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:48.753625 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1557e-d077-45cf-a9fe-4cc0f9c90619 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:48.753902 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1557e-d077-45cf-a9fe-4cc0f9c90619 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:48.755170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1557e-d077-45cf-a9fe-4cc0f9c90619 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:48.756683 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1557e-d077-45cf-a9fe-4cc0f9c90619 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:48.758423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 121/233] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:48 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:48.764111 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a9f46-69a0-4ec9-9603-55798ad9095a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:48.826336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:48.900488 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a9f46-69a0-4ec9-9603-55798ad9095a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:48.938974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11964f3-c922-406d-b281-930e37f40510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:48.939890 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11964f3-c922-406d-b281-930e37f40510 None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:48.940172 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11964f3-c922-406d-b281-930e37f40510 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:48.940477 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11964f3-c922-406d-b281-930e37f40510 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf/users/172c240c9845422c948a340f9856e6ac/roles/ab7a63cec3a744848972fcbb89ecd0ca` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:48.942500 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11964f3-c922-406d-b281-930e37f40510 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5a20e721548c582f2c7a214733faf, user_id=172c240c9845422c948a340f9856e6ac, role_id=ab7a63cec3a744848972fcbb89ecd0ca)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:48.943960 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11964f3-c922-406d-b281-930e37f40510 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:49.069743 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 113/234] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:06:48 2022] PUT /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf/users/172c240c9845422c948a340f9856e6ac/roles/ab7a63cec3a744848972fcbb89ecd0ca => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:50.171122 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d72e2-3afc-4a3e-898e-6ff17da8958e None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:50.172689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d72e2-3afc-4a3e-898e-6ff17da8958e None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:50.173183 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d72e2-3afc-4a3e-898e-6ff17da8958e None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:50.174791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 122/235] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:50.180350 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed73a90-b1fc-42eb-84fa-aa3db0d15b99 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:50.181189 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed73a90-b1fc-42eb-84fa-aa3db0d15b99 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:50.181704 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed73a90-b1fc-42eb-84fa-aa3db0d15b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:50.192300 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6ffc4-84d0-41f3-a083-eaf1a25c1980 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:50.192980 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6ffc4-84d0-41f3-a083-eaf1a25c1980 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:50.193478 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6ffc4-84d0-41f3-a083-eaf1a25c1980 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:50.195098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 123/236] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:06:50.200587 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e66db-5877-404e-b3a7-a1d801edd22e None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:50.200879 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e66db-5877-404e-b3a7-a1d801edd22e None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:50.201169 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e66db-5877-404e-b3a7-a1d801edd22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:50.206601 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-fed73a90-b1fc-42eb-84fa-aa3db0d15b99 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:50.208119 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-fed73a90-b1fc-42eb-84fa-aa3db0d15b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:50.237446 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-445e66db-5877-404e-b3a7-a1d801edd22e None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:50.238649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-445e66db-5877-404e-b3a7-a1d801edd22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:50.260717 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:50.292642 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 114/237] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:50.318352 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed73a90-b1fc-42eb-84fa-aa3db0d15b99 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:50.381751 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:50.405297 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 124/238] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:50.451411 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e66db-5877-404e-b3a7-a1d801edd22e None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:50.507171 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed73a90-b1fc-42eb-84fa-aa3db0d15b99 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:50.566236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc0ccb-5e06-4712-bd5e-09568bbc1708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:50.567516 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc0ccb-5e06-4712-bd5e-09568bbc1708 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:50.568096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc0ccb-5e06-4712-bd5e-09568bbc1708 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:50.568799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc0ccb-5e06-4712-bd5e-09568bbc1708 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:50.570305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc0ccb-5e06-4712-bd5e-09568bbc1708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:50.572123 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc0ccb-5e06-4712-bd5e-09568bbc1708 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:50.574883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 115/239] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:50.576406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a4af5-56b9-4ea6-9478-ab78ea914a32 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:50.576773 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a4af5-56b9-4ea6-9478-ab78ea914a32 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:50.577128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a4af5-56b9-4ea6-9478-ab78ea914a32 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:50.579249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 116/240] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:50.584257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a4af5-56b9-4ea6-9478-ab78ea914a32 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:50.634674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e66db-5877-404e-b3a7-a1d801edd22e None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:50.672175 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaabae-adaa-481c-9284-0068c6f54c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:50.673036 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaabae-adaa-481c-9284-0068c6f54c44 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:50.673322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaabae-adaa-481c-9284-0068c6f54c44 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:50.673601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaabae-adaa-481c-9284-0068c6f54c44 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:50.684887 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaabae-adaa-481c-9284-0068c6f54c44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-5bbaabae-adaa-481c-9284-0068c6f54c44 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:50.692921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:50.694048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:50.694989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 125/241] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:06:50.696270 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d49c47-9967-4180-9307-8b609f84500a None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:50.696270 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d49c47-9967-4180-9307-8b609f84500a None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:50.696490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d49c47-9967-4180-9307-8b609f84500a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:50.702663 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:50.714612 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-36d49c47-9967-4180-9307-8b609f84500a None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:50.719840 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-36d49c47-9967-4180-9307-8b609f84500a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:50.721975 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:50.727019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a4af5-56b9-4ea6-9478-ab78ea914a32 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:50.774019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beed2d6-d8b1-44cf-b263-3f681a0d8bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:50.775406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beed2d6-d8b1-44cf-b263-3f681a0d8bbc None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:50.775759 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beed2d6-d8b1-44cf-b263-3f681a0d8bbc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:50.776114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beed2d6-d8b1-44cf-b263-3f681a0d8bbc None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:50.776950 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beed2d6-d8b1-44cf-b263-3f681a0d8bbc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:50.779192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beed2d6-d8b1-44cf-b263-3f681a0d8bbc None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:50.798518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 126/242] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:50.799888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d49c47-9967-4180-9307-8b609f84500a None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:50.903153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 117/243] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:06:50 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:50.903992 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beed2d6-d8b1-44cf-b263-3f681a0d8bbc None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:50.943090 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d49c47-9967-4180-9307-8b609f84500a None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:50.997195 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93624df1-4463-4db8-ac08-b6637c8bafa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:50.998158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93624df1-4463-4db8-ac08-b6637c8bafa9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:50.998440 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93624df1-4463-4db8-ac08-b6637c8bafa9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:50.998715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93624df1-4463-4db8-ac08-b6637c8bafa9 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:50.999477 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93624df1-4463-4db8-ac08-b6637c8bafa9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:50.999817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93624df1-4463-4db8-ac08-b6637c8bafa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:51.001978 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93624df1-4463-4db8-ac08-b6637c8bafa9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:51.003903 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:51.014333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 127/244] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:51.020488 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93624df1-4463-4db8-ac08-b6637c8bafa9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:51.068079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beed2d6-d8b1-44cf-b263-3f681a0d8bbc None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:51.138144 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ef82a-fc7f-46eb-8ad5-bdf8c1fa7a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:51.139278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef82a-fc7f-46eb-8ad5-bdf8c1fa7a45 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:51.139565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef82a-fc7f-46eb-8ad5-bdf8c1fa7a45 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:51.139847 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef82a-fc7f-46eb-8ad5-bdf8c1fa7a45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:51.162265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ef82a-fc7f-46eb-8ad5-bdf8c1fa7a45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-150ef82a-fc7f-46eb-8ad5-bdf8c1fa7a45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:51.173462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:51.174647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:51.175605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 118/245] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:51.177337 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa6b2b-9405-4ee7-8b6d-7d58a059c92a None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:51.177720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa6b2b-9405-4ee7-8b6d-7d58a059c92a None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:51.177866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa6b2b-9405-4ee7-8b6d-7d58a059c92a None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:51.180244 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 119/246] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:51.185948 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa6b2b-9405-4ee7-8b6d-7d58a059c92a None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:51.191482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93624df1-4463-4db8-ac08-b6637c8bafa9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:51.217271 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:51.245643 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9ab1c-4294-48f6-9071-def838dea0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:51.246530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9ab1c-4294-48f6-9071-def838dea0c1 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:51.246814 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9ab1c-4294-48f6-9071-def838dea0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:51.247090 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9ab1c-4294-48f6-9071-def838dea0c1 None admin] PATH_INFO: `/v3/users/dd677d92744b43c495fe74c593d66ed5` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:51.268508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9ab1c-4294-48f6-9071-def838dea0c1 None admin] RBAC: Authorizing `identity:get_user(user_id=dd677d92744b43c495fe74c593d66ed5)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:51.270378 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9ab1c-4294-48f6-9071-def838dea0c1 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:51.272863 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 128/247] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:51 2022] GET /identity/v3/users/dd677d92744b43c495fe74c593d66ed5 => generated 297 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:51.274400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938041e6-4b3f-4272-bca5-5f603696eb5a None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:51.274929 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938041e6-4b3f-4272-bca5-5f603696eb5a None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:51.276423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938041e6-4b3f-4272-bca5-5f603696eb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:51.295157 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-938041e6-4b3f-4272-bca5-5f603696eb5a None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:51.297061 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-938041e6-4b3f-4272-bca5-5f603696eb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:51.358283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa6b2b-9405-4ee7-8b6d-7d58a059c92a None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:51.361854 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:51.382290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 129/248] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:51.383410 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938041e6-4b3f-4272-bca5-5f603696eb5a None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:51.406362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2dbdf-d137-47b5-9a02-1e8980d8c8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:51.407372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2dbdf-d137-47b5-9a02-1e8980d8c8b3 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:51.407911 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2dbdf-d137-47b5-9a02-1e8980d8c8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:51.408268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2dbdf-d137-47b5-9a02-1e8980d8c8b3 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:51.409147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2dbdf-d137-47b5-9a02-1e8980d8c8b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:51.409536 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2dbdf-d137-47b5-9a02-1e8980d8c8b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:51.411010 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2dbdf-d137-47b5-9a02-1e8980d8c8b3 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:51.423501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 120/249] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:51.424451 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2dbdf-d137-47b5-9a02-1e8980d8c8b3 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:51.521560 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938041e6-4b3f-4272-bca5-5f603696eb5a None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:51.547986 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:51.564873 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e581e55-a71b-41e6-a1c6-0885011d21d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:51.582993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e581e55-a71b-41e6-a1c6-0885011d21d6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:51.582993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e581e55-a71b-41e6-a1c6-0885011d21d6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:51.582993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e581e55-a71b-41e6-a1c6-0885011d21d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:51.583292 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2dbdf-d137-47b5-9a02-1e8980d8c8b3 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:51.626170 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4373e0-52b3-40a8-84bf-637974d0ba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:51.627084 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4373e0-52b3-40a8-84bf-637974d0ba25 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:51.627369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4373e0-52b3-40a8-84bf-637974d0ba25 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:51.628178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4373e0-52b3-40a8-84bf-637974d0ba25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:51.641753 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:51.644159 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4373e0-52b3-40a8-84bf-637974d0ba25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-3f4373e0-52b3-40a8-84bf-637974d0ba25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:51.656209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:51.659020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:06:51.659973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:51.660743 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e581e55-a71b-41e6-a1c6-0885011d21d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:51.660826 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 121/250] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:06:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:51.660826 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4373e0-52b3-40a8-84bf-637974d0ba25 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:51.660961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e581e55-a71b-41e6-a1c6-0885011d21d6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:51.787919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4373e0-52b3-40a8-84bf-637974d0ba25 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:51.793584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 130/251] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:51.794307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e581e55-a71b-41e6-a1c6-0885011d21d6 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:51.835949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab1471-3775-4c56-9358-4a3017c8c072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:51.836877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1471-3775-4c56-9358-4a3017c8c072 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:51.837163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1471-3775-4c56-9358-4a3017c8c072 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:51.837445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1471-3775-4c56-9358-4a3017c8c072 None admin] PATH_INFO: `/v3/users/7741e15112f64b92b6959796f71ab109` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:51.838668 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab1471-3775-4c56-9358-4a3017c8c072 None admin] RBAC: Authorizing `identity:get_user(user_id=7741e15112f64b92b6959796f71ab109)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:51.840149 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab1471-3775-4c56-9358-4a3017c8c072 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:51.842180 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 122/252] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:51 2022] GET /identity/v3/users/7741e15112f64b92b6959796f71ab109 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:51.842981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab1471-3775-4c56-9358-4a3017c8c072 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:51.884632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:51.985688 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:52.011492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e581e55-a71b-41e6-a1c6-0885011d21d6 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:52.058862 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab1471-3775-4c56-9358-4a3017c8c072 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:52.067554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419648f-93db-49c6-8e8b-a886469cf0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:52.068236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419648f-93db-49c6-8e8b-a886469cf0c6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:52.068542 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419648f-93db-49c6-8e8b-a886469cf0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:52.069240 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419648f-93db-49c6-8e8b-a886469cf0c6 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:52.069572 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419648f-93db-49c6-8e8b-a886469cf0c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:52.069920 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419648f-93db-49c6-8e8b-a886469cf0c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:52.071332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419648f-93db-49c6-8e8b-a886469cf0c6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:52.086374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 131/253] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:06:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:52.087311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419648f-93db-49c6-8e8b-a886469cf0c6 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:52.133748 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127330da-7b61-4740-a60a-30e3f8829583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:52.134620 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127330da-7b61-4740-a60a-30e3f8829583 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:52.134900 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127330da-7b61-4740-a60a-30e3f8829583 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:52.135173 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127330da-7b61-4740-a60a-30e3f8829583 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:52.142614 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127330da-7b61-4740-a60a-30e3f8829583 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:52.144120 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127330da-7b61-4740-a60a-30e3f8829583 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:52.146131 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 123/254] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:51 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:52.146733 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127330da-7b61-4740-a60a-30e3f8829583 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:52.254385 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:52.262564 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419648f-93db-49c6-8e8b-a886469cf0c6 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:52.286176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127330da-7b61-4740-a60a-30e3f8829583 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:52.313173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae28e0-ed5b-4868-a434-a611de4e7a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:52.314265 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae28e0-ed5b-4868-a434-a611de4e7a05 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:52.314555 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae28e0-ed5b-4868-a434-a611de4e7a05 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:52.315010 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae28e0-ed5b-4868-a434-a611de4e7a05 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:52.316675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae28e0-ed5b-4868-a434-a611de4e7a05 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:52.318703 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae28e0-ed5b-4868-a434-a611de4e7a05 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:52.321316 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 132/255] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:52 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:52.325307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae28e0-ed5b-4868-a434-a611de4e7a05 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:52.334534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d36b23-cbba-476c-86f6-eb5d89b04384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:52.336043 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36b23-cbba-476c-86f6-eb5d89b04384 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:52.336370 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36b23-cbba-476c-86f6-eb5d89b04384 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:52.336674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d36b23-cbba-476c-86f6-eb5d89b04384 None admin] PATH_INFO: `/v3/users/172c240c9845422c948a340f9856e6ac` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:52.338353 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d36b23-cbba-476c-86f6-eb5d89b04384 None admin] RBAC: Authorizing `identity:get_user(user_id=172c240c9845422c948a340f9856e6ac)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:52.340199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d36b23-cbba-476c-86f6-eb5d89b04384 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:52.342931 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 124/256] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:52 2022] GET /identity/v3/users/172c240c9845422c948a340f9856e6ac => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:52.344416 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263387c2-900c-450d-8c9b-06725d68b283 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:52.344713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263387c2-900c-450d-8c9b-06725d68b283 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:52.345005 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263387c2-900c-450d-8c9b-06725d68b283 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:52.346563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 125/257] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:52.347442 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263387c2-900c-450d-8c9b-06725d68b283 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:52.348822 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:52.451420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae28e0-ed5b-4868-a434-a611de4e7a05 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:52.490932 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353d99f-1e86-4f04-ad5e-7b2d733c29b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:52.491514 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353d99f-1e86-4f04-ad5e-7b2d733c29b8 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:52.491849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353d99f-1e86-4f04-ad5e-7b2d733c29b8 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:52.492132 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353d99f-1e86-4f04-ad5e-7b2d733c29b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:52.493055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353d99f-1e86-4f04-ad5e-7b2d733c29b8 None admin] RBAC: Adding query filter params (role.id=a0e21378e6134985a2ab8a70003fdad9, scope.project.id=8ff5a20e721548c582f2c7a214733faf, user.id=dd677d92744b43c495fe74c593d66ed5) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:52.493379 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353d99f-1e86-4f04-ad5e-7b2d733c29b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:52.494846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353d99f-1e86-4f04-ad5e-7b2d733c29b8 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:52.500476 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263387c2-900c-450d-8c9b-06725d68b283 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:52.501224 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 133/258] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:52 2022] GET /identity/v3/role_assignments?user.id=dd677d92744b43c495fe74c593d66ed5&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=a0e21378e6134985a2ab8a70003fdad9 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:52.510116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353d99f-1e86-4f04-ad5e-7b2d733c29b8 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:52.550730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e400a6c-7291-4b4b-9af0-0df24c640574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:52.555699 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e400a6c-7291-4b4b-9af0-0df24c640574 None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:52.556109 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e400a6c-7291-4b4b-9af0-0df24c640574 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:52.556456 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e400a6c-7291-4b4b-9af0-0df24c640574 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf/users/7741e15112f64b92b6959796f71ab109/roles/ba04654a403444e4ae54a1a216888fa5` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:52.558873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e400a6c-7291-4b4b-9af0-0df24c640574 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5a20e721548c582f2c7a214733faf, user_id=7741e15112f64b92b6959796f71ab109, role_id=ba04654a403444e4ae54a1a216888fa5)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:52.560466 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e400a6c-7291-4b4b-9af0-0df24c640574 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:52.618102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 126/259] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:06:52 2022] PUT /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf/users/7741e15112f64b92b6959796f71ab109/roles/ba04654a403444e4ae54a1a216888fa5 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:52.619145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb67b6d-73c5-4554-ac91-07818cce4c00 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:52.619490 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb67b6d-73c5-4554-ac91-07818cce4c00 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:52.619900 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb67b6d-73c5-4554-ac91-07818cce4c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:52.644755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:52.661396 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-8bb67b6d-73c5-4554-ac91-07818cce4c00 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:52.662725 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-8bb67b6d-73c5-4554-ac91-07818cce4c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:52.685269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:52.735482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353d99f-1e86-4f04-ad5e-7b2d733c29b8 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:52.811844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bad66-8795-491a-b934-a13dfdb05862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:52.813328 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bad66-8795-491a-b934-a13dfdb05862 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:52.813613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bad66-8795-491a-b934-a13dfdb05862 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:52.813889 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54bad66-8795-491a-b934-a13dfdb05862 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:52.828326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 127/260] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:52.875033 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb67b6d-73c5-4554-ac91-07818cce4c00 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:52.931650 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54bad66-8795-491a-b934-a13dfdb05862 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:52.933507 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54bad66-8795-491a-b934-a13dfdb05862 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:53.084369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:53.086489 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb67b6d-73c5-4554-ac91-07818cce4c00 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:53.136652 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 134/261] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:53.142947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bad66-8795-491a-b934-a13dfdb05862 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:53.166554 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:53.179975 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fc59b-c093-415b-ba93-70f6a279cb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:53.180821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fc59b-c093-415b-ba93-70f6a279cb76 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:53.181106 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fc59b-c093-415b-ba93-70f6a279cb76 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:53.181384 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fc59b-c093-415b-ba93-70f6a279cb76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:53.209324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fc59b-c093-415b-ba93-70f6a279cb76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-228fc59b-c093-415b-ba93-70f6a279cb76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:53.217690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:53.218936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:53.220575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 128/262] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:06:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:53.225526 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fc59b-c093-415b-ba93-70f6a279cb76 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:53.408376 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54bad66-8795-491a-b934-a13dfdb05862 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:53.464534 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a38d3-5527-4870-a363-f3264f49ac7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:53.466085 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a38d3-5527-4870-a363-f3264f49ac7f None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:53.466383 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a38d3-5527-4870-a363-f3264f49ac7f None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:53.466967 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a38d3-5527-4870-a363-f3264f49ac7f None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:53.469467 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a38d3-5527-4870-a363-f3264f49ac7f None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:53.471746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a38d3-5527-4870-a363-f3264f49ac7f None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:53.474468 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fc59b-c093-415b-ba93-70f6a279cb76 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:53.477944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 135/263] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:53 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:53.490329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a38d3-5527-4870-a363-f3264f49ac7f None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:53.550711 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12f8ab-853b-4701-adf9-a98136c0a303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:53.551853 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12f8ab-853b-4701-adf9-a98136c0a303 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:53.552318 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12f8ab-853b-4701-adf9-a98136c0a303 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:53.552786 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12f8ab-853b-4701-adf9-a98136c0a303 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:53.553925 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12f8ab-853b-4701-adf9-a98136c0a303 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:53.554399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12f8ab-853b-4701-adf9-a98136c0a303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:53.556883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12f8ab-853b-4701-adf9-a98136c0a303 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:53.574108 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 129/264] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:53.576525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:53.584241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12f8ab-853b-4701-adf9-a98136c0a303 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:53.694345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a38d3-5527-4870-a363-f3264f49ac7f None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:53.751973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:53.752437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec8064-a09f-4b1d-ac9c-55aff04b8dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:53.753354 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec8064-a09f-4b1d-ac9c-55aff04b8dd3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:53.753473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec8064-a09f-4b1d-ac9c-55aff04b8dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:53.753682 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec8064-a09f-4b1d-ac9c-55aff04b8dd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:53.754587 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec8064-a09f-4b1d-ac9c-55aff04b8dd3 None admin] RBAC: Adding query filter params (role.id=c9fbfe89c48d458c94f2a8a11652fcf2, scope.project.id=8ff5a20e721548c582f2c7a214733faf, user.id=172c240c9845422c948a340f9856e6ac) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:53.754899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec8064-a09f-4b1d-ac9c-55aff04b8dd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:53.756457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec8064-a09f-4b1d-ac9c-55aff04b8dd3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:53.770878 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 136/265] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:53 2022] GET /identity/v3/role_assignments?user.id=172c240c9845422c948a340f9856e6ac&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=c9fbfe89c48d458c94f2a8a11652fcf2 => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:53.883558 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12f8ab-853b-4701-adf9-a98136c0a303 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:53.934123 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daf25b-9135-4bcc-913e-22ba4454c431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:53.935007 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daf25b-9135-4bcc-913e-22ba4454c431 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:53.935284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daf25b-9135-4bcc-913e-22ba4454c431 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:53.935553 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daf25b-9135-4bcc-913e-22ba4454c431 None admin] PATH_INFO: `/v3/users/172c240c9845422c948a340f9856e6ac` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:53.941349 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daf25b-9135-4bcc-913e-22ba4454c431 None admin] RBAC: Authorizing `identity:get_user(user_id=172c240c9845422c948a340f9856e6ac)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:53.942880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daf25b-9135-4bcc-913e-22ba4454c431 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:53.945123 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 130/266] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:53 2022] GET /identity/v3/users/172c240c9845422c948a340f9856e6ac => generated 283 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:53.950809 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec8064-a09f-4b1d-ac9c-55aff04b8dd3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:53.975541 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8014b3-7c8b-4b48-9896-d772600a4049 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:53.975868 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8014b3-7c8b-4b48-9896-d772600a4049 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:53.976132 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8014b3-7c8b-4b48-9896-d772600a4049 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:53.977801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 131/267] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:06:53.983234 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aefd72-c4de-44a1-a0b7-51d1c7ab56eb None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:53.983841 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aefd72-c4de-44a1-a0b7-51d1c7ab56eb None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:53.984378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aefd72-c4de-44a1-a0b7-51d1c7ab56eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:54.021608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-44aefd72-c4de-44a1-a0b7-51d1c7ab56eb None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:54.023306 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-44aefd72-c4de-44a1-a0b7-51d1c7ab56eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:54.190795 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 132/268] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:54.193715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec8064-a09f-4b1d-ac9c-55aff04b8dd3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:54.206900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:54.230428 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aefd72-c4de-44a1-a0b7-51d1c7ab56eb None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:54.234528 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520a56c-feba-4555-bdab-a505435892bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:54.235195 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520a56c-feba-4555-bdab-a505435892bd None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:54.236324 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520a56c-feba-4555-bdab-a505435892bd None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:54.236324 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520a56c-feba-4555-bdab-a505435892bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:54.296910 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:54.346636 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520a56c-feba-4555-bdab-a505435892bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:54.348173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520a56c-feba-4555-bdab-a505435892bd None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:54.355509 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aefd72-c4de-44a1-a0b7-51d1c7ab56eb None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:54.401477 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098357da-7495-4930-90a0-5534af2147bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:54.402308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098357da-7495-4930-90a0-5534af2147bc None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:54.402600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098357da-7495-4930-90a0-5534af2147bc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:54.402882 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098357da-7495-4930-90a0-5534af2147bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:54.421163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098357da-7495-4930-90a0-5534af2147bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-098357da-7495-4930-90a0-5534af2147bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:54.431217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:54.434017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:54.436199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 133/269] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:54.440615 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098357da-7495-4930-90a0-5534af2147bc None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:54.536499 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 137/270] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:54.545842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520a56c-feba-4555-bdab-a505435892bd None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:54.568458 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:54.630957 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098357da-7495-4930-90a0-5534af2147bc None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:54.648413 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:54.675497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17525a6-5589-4ce4-a4fd-0fe49519a8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:54.676617 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17525a6-5589-4ce4-a4fd-0fe49519a8da None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:54.677260 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17525a6-5589-4ce4-a4fd-0fe49519a8da None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:54.677750 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17525a6-5589-4ce4-a4fd-0fe49519a8da None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:54.679581 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17525a6-5589-4ce4-a4fd-0fe49519a8da None admin] RBAC: Adding query filter params (name=member) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:54.680119 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17525a6-5589-4ce4-a4fd-0fe49519a8da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:54.681752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17525a6-5589-4ce4-a4fd-0fe49519a8da None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:54.692371 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 134/271] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:54.694509 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdb514-dde8-4e6a-ad0d-7bb7ac918e43 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:54.695013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdb514-dde8-4e6a-ad0d-7bb7ac918e43 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:54.695486 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdb514-dde8-4e6a-ad0d-7bb7ac918e43 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:54.697636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 135/272] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:54.698580 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdb514-dde8-4e6a-ad0d-7bb7ac918e43 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:54.737260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520a56c-feba-4555-bdab-a505435892bd None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:54.793979 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69587192-1663-4872-973c-bcb78cebe613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:54.793979 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69587192-1663-4872-973c-bcb78cebe613 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:54.793979 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69587192-1663-4872-973c-bcb78cebe613 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:54.793979 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69587192-1663-4872-973c-bcb78cebe613 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:54.794475 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69587192-1663-4872-973c-bcb78cebe613 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:54.794475 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69587192-1663-4872-973c-bcb78cebe613 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:54.796896 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 138/273] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:54 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:54.798439 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a3ca-e0ab-4b43-a149-f0f2e183c80c None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:54.798945 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a3ca-e0ab-4b43-a149-f0f2e183c80c None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:54.799441 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42a3ca-e0ab-4b43-a149-f0f2e183c80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:54.817844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-7d42a3ca-e0ab-4b43-a149-f0f2e183c80c None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:54.819491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-7d42a3ca-e0ab-4b43-a149-f0f2e183c80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:54.839617 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdb514-dde8-4e6a-ad0d-7bb7ac918e43 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:54.888037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512bf7d-9551-4f05-970f-02aafcc52d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:54.888980 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512bf7d-9551-4f05-970f-02aafcc52d5f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:54.889287 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512bf7d-9551-4f05-970f-02aafcc52d5f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:54.889571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5512bf7d-9551-4f05-970f-02aafcc52d5f None admin] PATH_INFO: `/v3/users/432ceeaf698041928671dc9fa709f85c` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:54.907624 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512bf7d-9551-4f05-970f-02aafcc52d5f None admin] RBAC: Authorizing `identity:get_user(user_id=432ceeaf698041928671dc9fa709f85c)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:54.909491 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5512bf7d-9551-4f05-970f-02aafcc52d5f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:54.911855 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 136/274] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:54 2022] GET /identity/v3/users/432ceeaf698041928671dc9fa709f85c => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:54.913185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512bf7d-9551-4f05-970f-02aafcc52d5f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:54.914157 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 139/275] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:54.922277 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42a3ca-e0ab-4b43-a149-f0f2e183c80c None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:54.942687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:54.947353 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:55.027735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5512bf7d-9551-4f05-970f-02aafcc52d5f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:55.062774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42a3ca-e0ab-4b43-a149-f0f2e183c80c None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:55.070257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2ac72-69a5-40f3-b0c7-f117bd3fc7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:55.071089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2ac72-69a5-40f3-b0c7-f117bd3fc7d4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:55.071474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2ac72-69a5-40f3-b0c7-f117bd3fc7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:55.071870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2ac72-69a5-40f3-b0c7-f117bd3fc7d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:55.073155 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2ac72-69a5-40f3-b0c7-f117bd3fc7d4 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=8ff5a20e721548c582f2c7a214733faf, user.id=172c240c9845422c948a340f9856e6ac) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:55.073386 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2ac72-69a5-40f3-b0c7-f117bd3fc7d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:55.077449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2ac72-69a5-40f3-b0c7-f117bd3fc7d4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:55.082849 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 137/276] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:54 2022] GET /identity/v3/role_assignments?user.id=172c240c9845422c948a340f9856e6ac&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:55.083980 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2ac72-69a5-40f3-b0c7-f117bd3fc7d4 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:55.120419 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247509bc-5e92-4bb3-9b22-f779654867cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:55.121583 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247509bc-5e92-4bb3-9b22-f779654867cd None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:55.122105 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247509bc-5e92-4bb3-9b22-f779654867cd None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:55.122575 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247509bc-5e92-4bb3-9b22-f779654867cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:55.123775 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247509bc-5e92-4bb3-9b22-f779654867cd None admin] RBAC: Adding query filter params (role.id=ba04654a403444e4ae54a1a216888fa5, user.id=432ceeaf698041928671dc9fa709f85c, scope.system=all) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:55.124426 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247509bc-5e92-4bb3-9b22-f779654867cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:55.126439 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247509bc-5e92-4bb3-9b22-f779654867cd None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:55.133329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 140/277] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:06:54 2022] GET /identity/v3/role_assignments?user.id=432ceeaf698041928671dc9fa709f85c&scope.system=all&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:55.211366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:55.219091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2ac72-69a5-40f3-b0c7-f117bd3fc7d4 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:55.268182 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c442d-94d4-43f0-a17e-79fdf88dcdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:55.269089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c442d-94d4-43f0-a17e-79fdf88dcdc1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:55.269376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c442d-94d4-43f0-a17e-79fdf88dcdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:55.269662 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c442d-94d4-43f0-a17e-79fdf88dcdc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:55.271122 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c442d-94d4-43f0-a17e-79fdf88dcdc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:55.272813 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c442d-94d4-43f0-a17e-79fdf88dcdc1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:55.275031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 138/278] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:55.285894 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247509bc-5e92-4bb3-9b22-f779654867cd None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:55.442479 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:55.475459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247509bc-5e92-4bb3-9b22-f779654867cd None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:55.520651 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef4a56c-0c82-4c87-aa20-400fcac72324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:55.521933 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4a56c-0c82-4c87-aa20-400fcac72324 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:55.522376 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4a56c-0c82-4c87-aa20-400fcac72324 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:55.522717 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef4a56c-0c82-4c87-aa20-400fcac72324 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:55.523482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4a56c-0c82-4c87-aa20-400fcac72324 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:55.525462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef4a56c-0c82-4c87-aa20-400fcac72324 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:55.641462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 141/279] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:06:55 2022] POST /identity/v3/users => generated 293 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:55.733532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b7c1-cb50-4061-97be-951b07f2aa71 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:55.734617 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b7c1-cb50-4061-97be-951b07f2aa71 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:55.734817 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2b7c1-cb50-4061-97be-951b07f2aa71 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:55.736744 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 139/280] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:55.742323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64635e9-927e-41f7-90d4-962ba37ba164 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:55.742747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64635e9-927e-41f7-90d4-962ba37ba164 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:55.743201 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64635e9-927e-41f7-90d4-962ba37ba164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:55.762189 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-c64635e9-927e-41f7-90d4-962ba37ba164 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:55.763411 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-c64635e9-927e-41f7-90d4-962ba37ba164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:55.847057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 142/281] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:55.862205 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4586a-bd48-4494-9cb9-22519733d994 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:55.862543 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4586a-bd48-4494-9cb9-22519733d994 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:55.862765 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4586a-bd48-4494-9cb9-22519733d994 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:55.864622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 140/282] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:55.869870 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1610585-7d6d-4aea-b59b-cfd169cc2433 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:55.870158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1610585-7d6d-4aea-b59b-cfd169cc2433 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:55.870397 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4586a-bd48-4494-9cb9-22519733d994 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:55.870527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1610585-7d6d-4aea-b59b-cfd169cc2433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:55.888639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-c1610585-7d6d-4aea-b59b-cfd169cc2433 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:55.889927 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-c1610585-7d6d-4aea-b59b-cfd169cc2433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:55.966628 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:55.988128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 143/283] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:56.013737 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4586a-bd48-4494-9cb9-22519733d994 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:56.015940 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1610585-7d6d-4aea-b59b-cfd169cc2433 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:56.055850 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c53ef-de0a-47ff-8829-cbb12797bf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:56.057027 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c53ef-de0a-47ff-8829-cbb12797bf02 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:56.057502 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c53ef-de0a-47ff-8829-cbb12797bf02 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:56.057782 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c53ef-de0a-47ff-8829-cbb12797bf02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:56.072371 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c53ef-de0a-47ff-8829-cbb12797bf02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-e24c53ef-de0a-47ff-8829-cbb12797bf02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:56.080953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:56.082057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:56.082991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 141/284] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:56.089311 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c53ef-de0a-47ff-8829-cbb12797bf02 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:56.121598 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:56.190259 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1610585-7d6d-4aea-b59b-cfd169cc2433 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:56.219713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c53ef-de0a-47ff-8829-cbb12797bf02 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:56.248589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942c69a-cb45-46a5-8b20-03987ca787e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:56.250931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c69a-cb45-46a5-8b20-03987ca787e1 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:56.250931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c69a-cb45-46a5-8b20-03987ca787e1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:56.251145 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c69a-cb45-46a5-8b20-03987ca787e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:56.265655 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942c69a-cb45-46a5-8b20-03987ca787e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:56.267912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f294bdb-e25e-4bd4-a0d8-4f37cd9d5e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:56.268881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f294bdb-e25e-4bd4-a0d8-4f37cd9d5e61 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:56.269352 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f294bdb-e25e-4bd4-a0d8-4f37cd9d5e61 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:56.269835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f294bdb-e25e-4bd4-a0d8-4f37cd9d5e61 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:56.270811 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f294bdb-e25e-4bd4-a0d8-4f37cd9d5e61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:56.271319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f294bdb-e25e-4bd4-a0d8-4f37cd9d5e61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:56.272982 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f294bdb-e25e-4bd4-a0d8-4f37cd9d5e61 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-c942c69a-cb45-46a5-8b20-03987ca787e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:56.274369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:56.275392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:06:56.276323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 144/285] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:56.282421 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942c69a-cb45-46a5-8b20-03987ca787e1 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:56.289984 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 142/286] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:56.307421 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f294bdb-e25e-4bd4-a0d8-4f37cd9d5e61 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:56.350568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:56.420993 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:56.448800 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942c69a-cb45-46a5-8b20-03987ca787e1 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:56.451548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f294bdb-e25e-4bd4-a0d8-4f37cd9d5e61 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:56.524048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b4796-7721-450b-a6fa-aa035ae44b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:56.524688 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b4796-7721-450b-a6fa-aa035ae44b42 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:56.525015 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b4796-7721-450b-a6fa-aa035ae44b42 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:56.525251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b4796-7721-450b-a6fa-aa035ae44b42 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:56.526279 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b4796-7721-450b-a6fa-aa035ae44b42 None admin] RBAC: Adding query filter params (name=member) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:56.526594 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b4796-7721-450b-a6fa-aa035ae44b42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:56.528988 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b4796-7721-450b-a6fa-aa035ae44b42 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:56.537497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44408abf-93a4-4adb-994e-f026220640a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:56.538397 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44408abf-93a4-4adb-994e-f026220640a7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:56.538680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44408abf-93a4-4adb-994e-f026220640a7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:56.538957 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44408abf-93a4-4adb-994e-f026220640a7 None admin] PATH_INFO: `/v3/users/dd677d92744b43c495fe74c593d66ed5` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:56.540052 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 145/287] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:56.541016 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44408abf-93a4-4adb-994e-f026220640a7 None admin] RBAC: Authorizing `identity:get_user(user_id=dd677d92744b43c495fe74c593d66ed5)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:56.542385 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44408abf-93a4-4adb-994e-f026220640a7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:56.544372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 143/288] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:56 2022] GET /identity/v3/users/dd677d92744b43c495fe74c593d66ed5 => generated 297 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:56.550421 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b4796-7721-450b-a6fa-aa035ae44b42 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:56.556073 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44408abf-93a4-4adb-994e-f026220640a7 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:56.711596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44408abf-93a4-4adb-994e-f026220640a7 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:56.754919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce752e-8010-46e3-804b-1d3511132e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:56.755795 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce752e-8010-46e3-804b-1d3511132e0d None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:56.756078 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce752e-8010-46e3-804b-1d3511132e0d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:56.756378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ce752e-8010-46e3-804b-1d3511132e0d None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:56.758786 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b4796-7721-450b-a6fa-aa035ae44b42 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:56.765786 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce752e-8010-46e3-804b-1d3511132e0d None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:56.767277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ce752e-8010-46e3-804b-1d3511132e0d None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:56.769313 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 144/289] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:56 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:56.777378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce752e-8010-46e3-804b-1d3511132e0d None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:56.780395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:56.806623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc298b9b-cceb-4cca-9cd9-b27b645ee556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:56.807502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc298b9b-cceb-4cca-9cd9-b27b645ee556 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:56.807786 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc298b9b-cceb-4cca-9cd9-b27b645ee556 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:56.808066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc298b9b-cceb-4cca-9cd9-b27b645ee556 None admin] PATH_INFO: `/v3/users/7741e15112f64b92b6959796f71ab109` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:56.810754 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc298b9b-cceb-4cca-9cd9-b27b645ee556 None admin] RBAC: Authorizing `identity:get_user(user_id=7741e15112f64b92b6959796f71ab109)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:56.812379 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc298b9b-cceb-4cca-9cd9-b27b645ee556 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:56.814192 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 146/290] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:56 2022] GET /identity/v3/users/7741e15112f64b92b6959796f71ab109 => generated 297 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:56.822886 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc298b9b-cceb-4cca-9cd9-b27b645ee556 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:56.832686 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:56.946564 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ce752e-8010-46e3-804b-1d3511132e0d None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:56.993618 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a44545-3460-42d5-913c-1f0cb920549d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:56.994534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a44545-3460-42d5-913c-1f0cb920549d None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:56.994817 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a44545-3460-42d5-913c-1f0cb920549d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:56.995092 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a44545-3460-42d5-913c-1f0cb920549d None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf/users/dd677d92744b43c495fe74c593d66ed5/roles/a0e21378e6134985a2ab8a70003fdad9` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:56.998311 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a44545-3460-42d5-913c-1f0cb920549d None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5a20e721548c582f2c7a214733faf, user_id=dd677d92744b43c495fe74c593d66ed5, role_id=a0e21378e6134985a2ab8a70003fdad9)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:57.000327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a44545-3460-42d5-913c-1f0cb920549d None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:57.016320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc298b9b-cceb-4cca-9cd9-b27b645ee556 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:57.052170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d860cb3-cf7c-4645-b9ec-d3794d19d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:57.052883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d860cb3-cf7c-4645-b9ec-d3794d19d6c7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:57.060648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d860cb3-cf7c-4645-b9ec-d3794d19d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:57.060648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d860cb3-cf7c-4645-b9ec-d3794d19d6c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:57.085922 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 145/291] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:06:56 2022] PUT /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf/users/dd677d92744b43c495fe74c593d66ed5/roles/a0e21378e6134985a2ab8a70003fdad9 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:57.112766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:57.133417 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d860cb3-cf7c-4645-b9ec-d3794d19d6c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:57.134930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d860cb3-cf7c-4645-b9ec-d3794d19d6c7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:57.268384 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 147/292] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:06:57.274379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a44545-3460-42d5-913c-1f0cb920549d None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:57.404804 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:57.429552 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a44545-3460-42d5-913c-1f0cb920549d None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:57.472406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f87f49-f753-4f35-9694-b77c6d9ee847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:57.473689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f87f49-f753-4f35-9694-b77c6d9ee847 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:57.474190 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f87f49-f753-4f35-9694-b77c6d9ee847 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:57.474680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f87f49-f753-4f35-9694-b77c6d9ee847 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:57.476149 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f87f49-f753-4f35-9694-b77c6d9ee847 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:57.477886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f87f49-f753-4f35-9694-b77c6d9ee847 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:57.479918 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 146/293] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:06:57 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:57.486693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d860cb3-cf7c-4645-b9ec-d3794d19d6c7 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:57.605876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:57.631947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d860cb3-cf7c-4645-b9ec-d3794d19d6c7 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:57.679844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba8ed8-65d1-4efd-bb17-cb11108ceb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:57.680493 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba8ed8-65d1-4efd-bb17-cb11108ceb5a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:57.680785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba8ed8-65d1-4efd-bb17-cb11108ceb5a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:57.681067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba8ed8-65d1-4efd-bb17-cb11108ceb5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:57.681929 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba8ed8-65d1-4efd-bb17-cb11108ceb5a None admin] RBAC: Adding query filter params (role.id=ba04654a403444e4ae54a1a216888fa5, scope.project.id=8ff5a20e721548c582f2c7a214733faf, user.id=7741e15112f64b92b6959796f71ab109) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:57.682248 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba8ed8-65d1-4efd-bb17-cb11108ceb5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:57.683777 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba8ed8-65d1-4efd-bb17-cb11108ceb5a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:57.693903 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 148/294] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:06:57 2022] GET /identity/v3/role_assignments?user.id=7741e15112f64b92b6959796f71ab109&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:58.039403 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e9112-2db0-4aae-93ff-355f2eb1265e None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:58.039963 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e9112-2db0-4aae-93ff-355f2eb1265e None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:58.039963 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9e9112-2db0-4aae-93ff-355f2eb1265e None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:58.041679 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 147/295] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:58.054351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca48c1-613b-4dcb-832a-93ef0a258bad None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:58.054872 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca48c1-613b-4dcb-832a-93ef0a258bad None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:58.054996 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca48c1-613b-4dcb-832a-93ef0a258bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:58.078788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-d6ca48c1-613b-4dcb-832a-93ef0a258bad None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:58.080015 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-d6ca48c1-613b-4dcb-832a-93ef0a258bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:58.157732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 149/296] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:58.184543 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e9112-2db0-4aae-93ff-355f2eb1265e None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:58.326987 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9e9112-2db0-4aae-93ff-355f2eb1265e None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:58.366209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c12da-0b7b-4925-8a60-96afb54a4d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:58.367009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c12da-0b7b-4925-8a60-96afb54a4d18 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:58.367294 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c12da-0b7b-4925-8a60-96afb54a4d18 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:58.367570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c12da-0b7b-4925-8a60-96afb54a4d18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:58.378374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c12da-0b7b-4925-8a60-96afb54a4d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-ce5c12da-0b7b-4925-8a60-96afb54a4d18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:58.386240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:58.387307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:06:58.388262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:58.392469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 148/297] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:58.394467 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca48c1-613b-4dcb-832a-93ef0a258bad None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:58.480733 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:58.511978 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca48c1-613b-4dcb-832a-93ef0a258bad None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:58.549938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c11374-bd57-44aa-8ce7-029c13f659f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:58.550610 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c11374-bd57-44aa-8ce7-029c13f659f9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:58.550895 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c11374-bd57-44aa-8ce7-029c13f659f9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:58.551172 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c11374-bd57-44aa-8ce7-029c13f659f9 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:58.551918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c11374-bd57-44aa-8ce7-029c13f659f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:58.552311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c11374-bd57-44aa-8ce7-029c13f659f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:58.553727 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c11374-bd57-44aa-8ce7-029c13f659f9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:58.563101 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 150/298] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:58.568470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c12da-0b7b-4925-8a60-96afb54a4d18 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:58.583302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:58.603390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef45ce5-d7b1-4ba2-8ed0-4885473ca555 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:58.603681 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef45ce5-d7b1-4ba2-8ed0-4885473ca555 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:58.604025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef45ce5-d7b1-4ba2-8ed0-4885473ca555 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:58.605722 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 151/299] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:58.611290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe9b39-a71f-4bfe-9dce-26f9f62b821e None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:58.611598 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe9b39-a71f-4bfe-9dce-26f9f62b821e None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:58.611866 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe9b39-a71f-4bfe-9dce-26f9f62b821e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:58.628982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-07fe9b39-a71f-4bfe-9dce-26f9f62b821e None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:58.630157 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-07fe9b39-a71f-4bfe-9dce-26f9f62b821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:58.700051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c12da-0b7b-4925-8a60-96afb54a4d18 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:58.711464 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 152/300] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:58.712698 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852677e2-d2c3-4ef0-a698-626dcc1ef148 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:58.712987 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852677e2-d2c3-4ef0-a698-626dcc1ef148 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:58.713263 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852677e2-d2c3-4ef0-a698-626dcc1ef148 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:58.716369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 153/301] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:58.720117 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a94ce8-6c0e-4530-bd75-3c0ef4533230 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:58.720432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a94ce8-6c0e-4530-bd75-3c0ef4533230 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:58.720716 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a94ce8-6c0e-4530-bd75-3c0ef4533230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:58.739266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-c0a94ce8-6c0e-4530-bd75-3c0ef4533230 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:58.740873 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-c0a94ce8-6c0e-4530-bd75-3c0ef4533230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:58.746363 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c03ce4-9352-4785-9e13-c8ab523b7f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:58.747244 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c03ce4-9352-4785-9e13-c8ab523b7f03 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:58.747521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c03ce4-9352-4785-9e13-c8ab523b7f03 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:58.747793 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c03ce4-9352-4785-9e13-c8ab523b7f03 None admin] PATH_INFO: `/v3/users/432ceeaf698041928671dc9fa709f85c` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:58.749116 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c03ce4-9352-4785-9e13-c8ab523b7f03 None admin] RBAC: Authorizing `identity:get_user(user_id=432ceeaf698041928671dc9fa709f85c)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:58.751160 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c03ce4-9352-4785-9e13-c8ab523b7f03 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:58.752805 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 149/302] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:58 2022] GET /identity/v3/users/432ceeaf698041928671dc9fa709f85c => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:58.753574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c03ce4-9352-4785-9e13-c8ab523b7f03 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:58.816247 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:58.825064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 154/303] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:06:58.825995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a94ce8-6c0e-4530-bd75-3c0ef4533230 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:58.853207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:58.881409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c03ce4-9352-4785-9e13-c8ab523b7f03 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:58.919080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542d953-55ac-4ec0-b2fe-3ef39c8745cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:58.919869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542d953-55ac-4ec0-b2fe-3ef39c8745cb None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:58.920148 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542d953-55ac-4ec0-b2fe-3ef39c8745cb None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:58.920824 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542d953-55ac-4ec0-b2fe-3ef39c8745cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:58.922076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542d953-55ac-4ec0-b2fe-3ef39c8745cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:58.923520 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542d953-55ac-4ec0-b2fe-3ef39c8745cb None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:58.928381 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 150/304] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:06:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:58.928381 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542d953-55ac-4ec0-b2fe-3ef39c8745cb None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:58.934003 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a94ce8-6c0e-4530-bd75-3c0ef4533230 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:58.979816 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b0de5-fb51-49ea-8838-12bc800a16ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:58.981298 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b0de5-fb51-49ea-8838-12bc800a16ba None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:58.981778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b0de5-fb51-49ea-8838-12bc800a16ba None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:58.982379 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b0de5-fb51-49ea-8838-12bc800a16ba None admin] PATH_INFO: `/v3/system/users/432ceeaf698041928671dc9fa709f85c/roles/ba04654a403444e4ae54a1a216888fa5` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:58.984719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b0de5-fb51-49ea-8838-12bc800a16ba None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=432ceeaf698041928671dc9fa709f85c, role_id=ba04654a403444e4ae54a1a216888fa5)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:58.987022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b0de5-fb51-49ea-8838-12bc800a16ba None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:59.027366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542d953-55ac-4ec0-b2fe-3ef39c8745cb None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:59.061968 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4672c-a1bb-4110-b6ca-acd2e27ccfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:59.063335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4672c-a1bb-4110-b6ca-acd2e27ccfb8 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:59.063623 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4672c-a1bb-4110-b6ca-acd2e27ccfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:59.063903 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4672c-a1bb-4110-b6ca-acd2e27ccfb8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:59.074311 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc4672c-a1bb-4110-b6ca-acd2e27ccfb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-1bc4672c-a1bb-4110-b6ca-acd2e27ccfb8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:06:59.082565 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:06:59.083881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:06:59.085319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:06:59.088676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 151/305] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:06:59.089323 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4672c-a1bb-4110-b6ca-acd2e27ccfb8 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:59.092093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 155/306] 10.222.0.88 () {58 vars in 1334 bytes} [Thu May 19 20:06:58 2022] PUT /identity/v3/system/users/432ceeaf698041928671dc9fa709f85c/roles/ba04654a403444e4ae54a1a216888fa5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:06:59.093795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b0de5-fb51-49ea-8838-12bc800a16ba None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:59.111341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:59.178344 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:59.230466 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b0de5-fb51-49ea-8838-12bc800a16ba None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:59.238745 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4672c-a1bb-4110-b6ca-acd2e27ccfb8 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:59.283846 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d92b28-1dab-45f1-a023-fed74f2d999b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:59.284372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1508ecb-6e59-4c7b-ba7f-4046e2649864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:59.285068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1508ecb-6e59-4c7b-ba7f-4046e2649864 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:59.285350 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1508ecb-6e59-4c7b-ba7f-4046e2649864 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:59.285627 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1508ecb-6e59-4c7b-ba7f-4046e2649864 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:59.285937 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d92b28-1dab-45f1-a023-fed74f2d999b None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:59.286388 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1508ecb-6e59-4c7b-ba7f-4046e2649864 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:59.286657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d92b28-1dab-45f1-a023-fed74f2d999b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:59.286767 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1508ecb-6e59-4c7b-ba7f-4046e2649864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:59.287256 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d92b28-1dab-45f1-a023-fed74f2d999b None admin] PATH_INFO: `/v3/groups` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:59.288201 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d92b28-1dab-45f1-a023-fed74f2d999b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:59.288315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1508ecb-6e59-4c7b-ba7f-4046e2649864 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:59.290107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d92b28-1dab-45f1-a023-fed74f2d999b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:59.304365 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 156/307] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:06:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:59.306626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1508ecb-6e59-4c7b-ba7f-4046e2649864 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:59.377579 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 152/308] 10.222.0.88 () {60 vars in 1134 bytes} [Thu May 19 20:06:59 2022] POST /identity/v3/groups => generated 229 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:06:59.425092 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1508ecb-6e59-4c7b-ba7f-4046e2649864 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:59.449523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:59.466533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d17c24-e1b4-4641-bd0d-2dd5f547a8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:59.467495 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d17c24-e1b4-4641-bd0d-2dd5f547a8d2 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:59.467778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d17c24-e1b4-4641-bd0d-2dd5f547a8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:59.468052 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d17c24-e1b4-4641-bd0d-2dd5f547a8d2 None admin] PATH_INFO: `/v3/users/edb3ef1e4888407e880710ae18438732` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:59.484251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d17c24-e1b4-4641-bd0d-2dd5f547a8d2 None admin] RBAC: Authorizing `identity:get_user(user_id=edb3ef1e4888407e880710ae18438732)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:59.485876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d17c24-e1b4-4641-bd0d-2dd5f547a8d2 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:59.488400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 157/309] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:06:59 2022] GET /identity/v3/users/edb3ef1e4888407e880710ae18438732 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:06:59.494287 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d92b28-1dab-45f1-a023-fed74f2d999b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:59.581337 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:06:59.620309 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d92b28-1dab-45f1-a023-fed74f2d999b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:06:59.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ad91b-044b-459c-85be-08c57580cffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:06:59.657901 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ad91b-044b-459c-85be-08c57580cffd None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:59.660384 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ad91b-044b-459c-85be-08c57580cffd None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:59.660384 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ad91b-044b-459c-85be-08c57580cffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:59.660384 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ad91b-044b-459c-85be-08c57580cffd None admin] RBAC: Adding query filter params (role.id=a0e21378e6134985a2ab8a70003fdad9, user.id=edb3ef1e4888407e880710ae18438732, scope.system=all) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:06:59.663780 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ad91b-044b-459c-85be-08c57580cffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:06:59.663780 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ad91b-044b-459c-85be-08c57580cffd None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:06:59.668796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 153/310] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:06:59 2022] GET /identity/v3/role_assignments?user.id=edb3ef1e4888407e880710ae18438732&scope.system=all&role.id=a0e21378e6134985a2ab8a70003fdad9 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:06:59.744229 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65417d3-af6b-4d27-bf2e-f1a9205935c7 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:59.744611 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65417d3-af6b-4d27-bf2e-f1a9205935c7 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:59.744840 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65417d3-af6b-4d27-bf2e-f1a9205935c7 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:59.746537 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 158/311] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:06:59.752361 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8a7e1-02f9-44d9-8ad6-db1844012914 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:06:59.753103 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8a7e1-02f9-44d9-8ad6-db1844012914 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:06:59.753103 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8a7e1-02f9-44d9-8ad6-db1844012914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:06:59.772425 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-b0a8a7e1-02f9-44d9-8ad6-db1844012914 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:06:59.773690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-b0a8a7e1-02f9-44d9-8ad6-db1844012914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:06:59.854511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 154/312] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:06:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:06:59.877781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65417d3-af6b-4d27-bf2e-f1a9205935c7 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:06:59.990295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65417d3-af6b-4d27-bf2e-f1a9205935c7 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:00.031515 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27961027-3fee-4f41-96f5-56781bde4158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:00.032326 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27961027-3fee-4f41-96f5-56781bde4158 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:00.032700 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27961027-3fee-4f41-96f5-56781bde4158 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:00.032977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27961027-3fee-4f41-96f5-56781bde4158 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:00.044457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27961027-3fee-4f41-96f5-56781bde4158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-27961027-3fee-4f41-96f5-56781bde4158 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:00.052898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:00.053982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:00.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 159/313] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:06:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:00.062471 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8a7e1-02f9-44d9-8ad6-db1844012914 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:00.134798 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:00.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8a7e1-02f9-44d9-8ad6-db1844012914 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:00.216785 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf6981-6ea8-4dcb-9bc8-32cd2026cc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:00.217850 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf6981-6ea8-4dcb-9bc8-32cd2026cc0f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:00.218898 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf6981-6ea8-4dcb-9bc8-32cd2026cc0f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:00.219391 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf6981-6ea8-4dcb-9bc8-32cd2026cc0f None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:00.220420 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf6981-6ea8-4dcb-9bc8-32cd2026cc0f None admin] RBAC: Adding query filter params (name=reader) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:00.220978 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf6981-6ea8-4dcb-9bc8-32cd2026cc0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:00.222584 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdf6981-6ea8-4dcb-9bc8-32cd2026cc0f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:00.232175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 155/314] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:07:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:00.238626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27961027-3fee-4f41-96f5-56781bde4158 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:00.283309 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:00.350185 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27961027-3fee-4f41-96f5-56781bde4158 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:00.386257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9ae1c-df2b-4370-b525-7d3772c076ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:00.387561 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9ae1c-df2b-4370-b525-7d3772c076ec None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:00.388085 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9ae1c-df2b-4370-b525-7d3772c076ec None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:00.388605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9ae1c-df2b-4370-b525-7d3772c076ec None admin] PATH_INFO: `/v3/users/dd677d92744b43c495fe74c593d66ed5` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:00.390118 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9ae1c-df2b-4370-b525-7d3772c076ec None admin] RBAC: Authorizing `identity:get_user(user_id=dd677d92744b43c495fe74c593d66ed5)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:00.391803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9ae1c-df2b-4370-b525-7d3772c076ec None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:00.394035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 160/315] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:07:00 2022] GET /identity/v3/users/dd677d92744b43c495fe74c593d66ed5 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:00.399887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf6981-6ea8-4dcb-9bc8-32cd2026cc0f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:00.509071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf6981-6ea8-4dcb-9bc8-32cd2026cc0f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:00.535197 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:00.547333 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546e04e-ac16-46cc-8f93-a731cd8c23c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:00.551685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e04e-ac16-46cc-8f93-a731cd8c23c4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:00.551685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e04e-ac16-46cc-8f93-a731cd8c23c4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:00.551685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4546e04e-ac16-46cc-8f93-a731cd8c23c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:00.619885 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546e04e-ac16-46cc-8f93-a731cd8c23c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:00.621436 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4546e04e-ac16-46cc-8f93-a731cd8c23c4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:00.732538 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 156/316] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:00.740172 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9ae1c-df2b-4370-b525-7d3772c076ec None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:00.851465 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9ae1c-df2b-4370-b525-7d3772c076ec None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:00.862048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:00.889295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0682ede-773b-46e8-b59c-047ddb966fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:00.890608 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0682ede-773b-46e8-b59c-047ddb966fa2 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:00.891313 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0682ede-773b-46e8-b59c-047ddb966fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:00.891818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0682ede-773b-46e8-b59c-047ddb966fa2 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:00.893282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0682ede-773b-46e8-b59c-047ddb966fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:00.894970 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0682ede-773b-46e8-b59c-047ddb966fa2 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:00.897291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 161/317] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:00 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:00.905451 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546e04e-ac16-46cc-8f93-a731cd8c23c4 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:00.931788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:01.015222 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4546e04e-ac16-46cc-8f93-a731cd8c23c4 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:01.052787 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f289f63-4601-4503-bf02-00d7ec61ff5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:01.053884 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f289f63-4601-4503-bf02-00d7ec61ff5d None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:01.054170 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f289f63-4601-4503-bf02-00d7ec61ff5d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:01.054449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f289f63-4601-4503-bf02-00d7ec61ff5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:01.055282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f289f63-4601-4503-bf02-00d7ec61ff5d None admin] RBAC: Adding query filter params (role.id=a0e21378e6134985a2ab8a70003fdad9, scope.project.id=8ff5a20e721548c582f2c7a214733faf, user.id=dd677d92744b43c495fe74c593d66ed5) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:01.055593 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f289f63-4601-4503-bf02-00d7ec61ff5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:01.057083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f289f63-4601-4503-bf02-00d7ec61ff5d None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:01.063622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 157/318] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:00 2022] GET /identity/v3/role_assignments?user.id=dd677d92744b43c495fe74c593d66ed5&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=a0e21378e6134985a2ab8a70003fdad9 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:01.522218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac63263-9828-46bf-9879-d5971758db83 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:01.522803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac63263-9828-46bf-9879-d5971758db83 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:01.522803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac63263-9828-46bf-9879-d5971758db83 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:01.524360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 162/319] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:01.529631 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487be500-5ab2-4ee0-aab5-e2d482ef4e09 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:01.529922 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487be500-5ab2-4ee0-aab5-e2d482ef4e09 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:01.530203 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487be500-5ab2-4ee0-aab5-e2d482ef4e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:01.550512 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-487be500-5ab2-4ee0-aab5-e2d482ef4e09 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:01.551717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-487be500-5ab2-4ee0-aab5-e2d482ef4e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:01.625733 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 158/320] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:01.649894 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac63263-9828-46bf-9879-d5971758db83 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:01.762540 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac63263-9828-46bf-9879-d5971758db83 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:01.800270 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2556ae4-42f8-49b0-8fdc-c1928a747674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:01.801139 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2556ae4-42f8-49b0-8fdc-c1928a747674 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:01.801430 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2556ae4-42f8-49b0-8fdc-c1928a747674 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:01.801710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2556ae4-42f8-49b0-8fdc-c1928a747674 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:01.815329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2556ae4-42f8-49b0-8fdc-c1928a747674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-e2556ae4-42f8-49b0-8fdc-c1928a747674 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:01.828640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:01.829685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:01.830725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 163/321] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:01.831511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487be500-5ab2-4ee0-aab5-e2d482ef4e09 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:01.847320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1e2a5-2968-41d6-b321-ac78c2179b27 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:01.847616 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1e2a5-2968-41d6-b321-ac78c2179b27 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:01.847900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1e2a5-2968-41d6-b321-ac78c2179b27 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:01.849312 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 164/322] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:07:01.851035 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:01.854529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb98d6-feec-4fc9-a820-c5867bce2381 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:01.857551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb98d6-feec-4fc9-a820-c5867bce2381 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:01.857551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb98d6-feec-4fc9-a820-c5867bce2381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:01.872687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-c2cb98d6-feec-4fc9-a820-c5867bce2381 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:01.874027 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-c2cb98d6-feec-4fc9-a820-c5867bce2381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:01.906329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:01.941373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487be500-5ab2-4ee0-aab5-e2d482ef4e09 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:01.950781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 165/323] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:07:01.974427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb98d6-feec-4fc9-a820-c5867bce2381 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:01.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906ab99-a640-4b7d-9254-cc625cae3290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:01.979938 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906ab99-a640-4b7d-9254-cc625cae3290 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:01.980223 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906ab99-a640-4b7d-9254-cc625cae3290 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:01.980531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906ab99-a640-4b7d-9254-cc625cae3290 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:01.986608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9906ab99-a640-4b7d-9254-cc625cae3290 None admin] RBAC: Adding query filter params (name=member) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:01.986608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9906ab99-a640-4b7d-9254-cc625cae3290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:01.986608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9906ab99-a640-4b7d-9254-cc625cae3290 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:01.991561 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 159/324] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:07:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:01.993062 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851be307-d8cc-43e3-be57-8fd21784cb1f None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:01.993531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851be307-d8cc-43e3-be57-8fd21784cb1f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:01.993818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851be307-d8cc-43e3-be57-8fd21784cb1f None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:01.995064 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 160/325] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:01.997257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851be307-d8cc-43e3-be57-8fd21784cb1f None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:02.081078 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:02.082582 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb98d6-feec-4fc9-a820-c5867bce2381 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:02.105953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851be307-d8cc-43e3-be57-8fd21784cb1f None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:02.117461 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96308c38-20d4-4980-95af-3ffade01f62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:02.118264 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96308c38-20d4-4980-95af-3ffade01f62d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:02.118546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96308c38-20d4-4980-95af-3ffade01f62d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:02.118828 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96308c38-20d4-4980-95af-3ffade01f62d None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:02.119989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96308c38-20d4-4980-95af-3ffade01f62d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:02.121505 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96308c38-20d4-4980-95af-3ffade01f62d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:02.123155 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 166/326] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:02.124476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8b247-d300-4d13-8ed1-888da69337f8 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:02.125270 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8b247-d300-4d13-8ed1-888da69337f8 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:02.125553 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8b247-d300-4d13-8ed1-888da69337f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:02.141697 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-bad8b247-d300-4d13-8ed1-888da69337f8 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:02.141904 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3aacc-5ee7-4a0b-81dd-f4509d03ea82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:02.142767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3aacc-5ee7-4a0b-81dd-f4509d03ea82 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:02.142916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-bad8b247-d300-4d13-8ed1-888da69337f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:02.143026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3aacc-5ee7-4a0b-81dd-f4509d03ea82 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:02.143310 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3aacc-5ee7-4a0b-81dd-f4509d03ea82 None admin] PATH_INFO: `/v3/users/432ceeaf698041928671dc9fa709f85c` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:02.144516 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3aacc-5ee7-4a0b-81dd-f4509d03ea82 None admin] RBAC: Authorizing `identity:get_user(user_id=432ceeaf698041928671dc9fa709f85c)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:02.145947 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3aacc-5ee7-4a0b-81dd-f4509d03ea82 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:02.147709 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 161/327] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:07:01 2022] GET /identity/v3/users/432ceeaf698041928671dc9fa709f85c => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:02.148478 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3aacc-5ee7-4a0b-81dd-f4509d03ea82 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:02.172215 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:02.213881 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 167/328] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:02.214754 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8b247-d300-4d13-8ed1-888da69337f8 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:02.257631 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3aacc-5ee7-4a0b-81dd-f4509d03ea82 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:02.291567 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9e546-3d4b-47fd-b9b5-20cbac6b390b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:02.292244 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9e546-3d4b-47fd-b9b5-20cbac6b390b None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:02.292561 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9e546-3d4b-47fd-b9b5-20cbac6b390b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:02.292844 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f9e546-3d4b-47fd-b9b5-20cbac6b390b None admin] PATH_INFO: `/v3/groups` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:02.293553 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9e546-3d4b-47fd-b9b5-20cbac6b390b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:02.294995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f9e546-3d4b-47fd-b9b5-20cbac6b390b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:02.321492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8b247-d300-4d13-8ed1-888da69337f8 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:02.354240 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6254877-2105-46ee-8e74-29142a6bd5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:02.354820 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6254877-2105-46ee-8e74-29142a6bd5f0 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:02.355101 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6254877-2105-46ee-8e74-29142a6bd5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:02.355373 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6254877-2105-46ee-8e74-29142a6bd5f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:02.356224 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6254877-2105-46ee-8e74-29142a6bd5f0 None admin] RBAC: Adding query filter params (role.id=ba04654a403444e4ae54a1a216888fa5, user.id=432ceeaf698041928671dc9fa709f85c, scope.system=all) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:02.356562 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6254877-2105-46ee-8e74-29142a6bd5f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:02.358092 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6254877-2105-46ee-8e74-29142a6bd5f0 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:02.370215 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 168/329] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:07:02 2022] GET /identity/v3/role_assignments?user.id=432ceeaf698041928671dc9fa709f85c&scope.system=all&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 510 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:02.371004 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6254877-2105-46ee-8e74-29142a6bd5f0 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:02.396541 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:02.424240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 162/330] 10.222.0.88 () {60 vars in 1134 bytes} [Thu May 19 20:07:02 2022] POST /identity/v3/groups => generated 226 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:07:02.491243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6254877-2105-46ee-8e74-29142a6bd5f0 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:02.530300 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4820d06-6c40-4e2d-8c70-fff5e485e87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:02.531847 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4820d06-6c40-4e2d-8c70-fff5e485e87e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:02.532132 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4820d06-6c40-4e2d-8c70-fff5e485e87e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:02.532453 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4820d06-6c40-4e2d-8c70-fff5e485e87e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:02.545065 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4820d06-6c40-4e2d-8c70-fff5e485e87e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-f4820d06-6c40-4e2d-8c70-fff5e485e87e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:02.554079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:02.555234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:02.561838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 169/331] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:02.562629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9e546-3d4b-47fd-b9b5-20cbac6b390b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:02.573217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:02.675980 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f9e546-3d4b-47fd-b9b5-20cbac6b390b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:02.710549 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05757d96-f6b1-4dde-ba34-1d3f940df139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:02.711224 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05757d96-f6b1-4dde-ba34-1d3f940df139 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:02.711515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05757d96-f6b1-4dde-ba34-1d3f940df139 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:02.711814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05757d96-f6b1-4dde-ba34-1d3f940df139 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:02.712591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05757d96-f6b1-4dde-ba34-1d3f940df139 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:02.712896 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05757d96-f6b1-4dde-ba34-1d3f940df139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:02.714301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05757d96-f6b1-4dde-ba34-1d3f940df139 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:02.724277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 163/332] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:07:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:02.730003 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4820d06-6c40-4e2d-8c70-fff5e485e87e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:02.818552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:02.840053 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4820d06-6c40-4e2d-8c70-fff5e485e87e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:02.879404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90eee7-c760-45d0-b9c4-216afceb84b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:02.880595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90eee7-c760-45d0-b9c4-216afceb84b7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:02.881084 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90eee7-c760-45d0-b9c4-216afceb84b7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:02.881366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90eee7-c760-45d0-b9c4-216afceb84b7 None admin] PATH_INFO: `/v3/users/edb3ef1e4888407e880710ae18438732` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:02.884249 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90eee7-c760-45d0-b9c4-216afceb84b7 None admin] RBAC: Authorizing `identity:get_user(user_id=edb3ef1e4888407e880710ae18438732)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:02.885778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90eee7-c760-45d0-b9c4-216afceb84b7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:02.887655 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 170/333] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:07:02 2022] GET /identity/v3/users/edb3ef1e4888407e880710ae18438732 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:02.893426 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05757d96-f6b1-4dde-ba34-1d3f940df139 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:02.972359 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:03.010363 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05757d96-f6b1-4dde-ba34-1d3f940df139 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:03.049260 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8758f18-8a03-4ed8-92b7-497c10e40d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:03.050142 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8758f18-8a03-4ed8-92b7-497c10e40d69 None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:03.050424 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8758f18-8a03-4ed8-92b7-497c10e40d69 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:03.050702 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8758f18-8a03-4ed8-92b7-497c10e40d69 None admin] PATH_INFO: `/v3/system/users/edb3ef1e4888407e880710ae18438732/roles/a0e21378e6134985a2ab8a70003fdad9` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:03.052226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8758f18-8a03-4ed8-92b7-497c10e40d69 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=edb3ef1e4888407e880710ae18438732, role_id=a0e21378e6134985a2ab8a70003fdad9)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:03.053659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8758f18-8a03-4ed8-92b7-497c10e40d69 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:03.132003 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 164/334] 10.222.0.88 () {58 vars in 1334 bytes} [Thu May 19 20:07:02 2022] PUT /identity/v3/system/users/edb3ef1e4888407e880710ae18438732/roles/a0e21378e6134985a2ab8a70003fdad9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:07:05.608190 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812bb09-9cd9-4c73-ab29-b8fd4f98483d None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:05.609855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812bb09-9cd9-4c73-ab29-b8fd4f98483d None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:05.610363 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812bb09-9cd9-4c73-ab29-b8fd4f98483d None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:05.612411 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 171/335] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:05.617828 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3850cb-0064-431b-999f-15b8de33434c None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:05.618350 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3850cb-0064-431b-999f-15b8de33434c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:05.618446 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3850cb-0064-431b-999f-15b8de33434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:05.640175 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec695c85-f26f-4168-a3a4-690ea1b2cb62 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:05.640838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec695c85-f26f-4168-a3a4-690ea1b2cb62 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:05.641354 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec695c85-f26f-4168-a3a4-690ea1b2cb62 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:05.643206 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-1e3850cb-0064-431b-999f-15b8de33434c None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:05.643387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 172/336] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:05.644822 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-1e3850cb-0064-431b-999f-15b8de33434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:05.648581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc15ced-a91b-4bb8-8295-a484bd3b9792 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:05.649141 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc15ced-a91b-4bb8-8295-a484bd3b9792 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:05.649643 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc15ced-a91b-4bb8-8295-a484bd3b9792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:05.678021 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-fcc15ced-a91b-4bb8-8295-a484bd3b9792 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:05.679589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-fcc15ced-a91b-4bb8-8295-a484bd3b9792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:05.801751 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 165/337] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:05.834611 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 173/338] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:05.842700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3850cb-0064-431b-999f-15b8de33434c None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:05.871420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc15ced-a91b-4bb8-8295-a484bd3b9792 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:05.948952 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:06.020244 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:06.023296 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3850cb-0064-431b-999f-15b8de33434c None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:06.048171 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc15ced-a91b-4bb8-8295-a484bd3b9792 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:06.107989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8f669-45dd-43cd-83f7-bf8c075314d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:06.109914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8f669-45dd-43cd-83f7-bf8c075314d2 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:06.110513 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8f669-45dd-43cd-83f7-bf8c075314d2 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:06.111033 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8f669-45dd-43cd-83f7-bf8c075314d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:06.113020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631cc3a-3baa-4143-ac70-398afa981112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:06.114576 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631cc3a-3baa-4143-ac70-398afa981112 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:06.115084 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631cc3a-3baa-4143-ac70-398afa981112 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:06.115586 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631cc3a-3baa-4143-ac70-398afa981112 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:06.126537 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8f669-45dd-43cd-83f7-bf8c075314d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-36c8f669-45dd-43cd-83f7-bf8c075314d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:06.135800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:06.136965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:06.137990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:06.140442 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6631cc3a-3baa-4143-ac70-398afa981112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:06.141955 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 166/339] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:06.143464 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40765700-01a5-457d-8b91-68b7df363906 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:06.143761 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40765700-01a5-457d-8b91-68b7df363906 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:06.144045 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40765700-01a5-457d-8b91-68b7df363906 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:06.145642 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 167/340] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:06.149928 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40765700-01a5-457d-8b91-68b7df363906 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-6631cc3a-3baa-4143-ac70-398afa981112 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:06.156362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:06.157470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:06.158406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:06.161674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 174/341] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:07:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:06.163309 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc7e2f-48ac-4414-a41a-e1d8e3462084 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:06.163825 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc7e2f-48ac-4414-a41a-e1d8e3462084 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:06.164451 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc7e2f-48ac-4414-a41a-e1d8e3462084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:06.197454 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-0ddc7e2f-48ac-4414-a41a-e1d8e3462084 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:06.199289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-0ddc7e2f-48ac-4414-a41a-e1d8e3462084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:06.267975 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40765700-01a5-457d-8b91-68b7df363906 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:06.304729 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:06.315963 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 175/342] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:06.316906 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc7e2f-48ac-4414-a41a-e1d8e3462084 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:06.324705 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a5113-81e8-47e4-bc66-588f231c84fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:06.325408 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a5113-81e8-47e4-bc66-588f231c84fc None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:06.325692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a5113-81e8-47e4-bc66-588f231c84fc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:06.325969 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a5113-81e8-47e4-bc66-588f231c84fc None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:06.326864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a5113-81e8-47e4-bc66-588f231c84fc None admin] RBAC: Adding query filter params (name=member) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:06.327178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a5113-81e8-47e4-bc66-588f231c84fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:06.338097 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a5113-81e8-47e4-bc66-588f231c84fc None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:06.359563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 168/343] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:07:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:06.360535 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a5113-81e8-47e4-bc66-588f231c84fc None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:06.480002 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:06.542686 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc7e2f-48ac-4414-a41a-e1d8e3462084 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:06.560657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a5113-81e8-47e4-bc66-588f231c84fc None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:06.587209 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d65e3-cb07-432d-bb6f-11ae18171b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:06.588289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d65e3-cb07-432d-bb6f-11ae18171b8d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:06.588944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d65e3-cb07-432d-bb6f-11ae18171b8d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:06.589435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d65e3-cb07-432d-bb6f-11ae18171b8d None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:06.590523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d65e3-cb07-432d-bb6f-11ae18171b8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:06.591117 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d65e3-cb07-432d-bb6f-11ae18171b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:06.592949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d65e3-cb07-432d-bb6f-11ae18171b8d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:06.603321 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 176/344] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:06.608766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d65e3-cb07-432d-bb6f-11ae18171b8d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:06.609370 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38014a1-e3ba-47d1-ac2e-51f75259fde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:06.610438 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38014a1-e3ba-47d1-ac2e-51f75259fde7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:06.610684 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38014a1-e3ba-47d1-ac2e-51f75259fde7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:06.610960 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38014a1-e3ba-47d1-ac2e-51f75259fde7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:06.624290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38014a1-e3ba-47d1-ac2e-51f75259fde7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-c38014a1-e3ba-47d1-ac2e-51f75259fde7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:06.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:06.634049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:06.635137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:06.639398 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 169/345] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:06.641585 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85933d3-bc64-45a9-9ad6-36ed420c1447 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:06.641883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85933d3-bc64-45a9-9ad6-36ed420c1447 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:06.642215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85933d3-bc64-45a9-9ad6-36ed420c1447 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:06.648097 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 170/346] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:06.648097 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85933d3-bc64-45a9-9ad6-36ed420c1447 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:06.718042 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:06.753369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d65e3-cb07-432d-bb6f-11ae18171b8d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:06.787004 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85933d3-bc64-45a9-9ad6-36ed420c1447 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:06.787686 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:06.793760 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba0e98-56f7-4209-b706-5e2bbcd1f63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:06.794427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba0e98-56f7-4209-b706-5e2bbcd1f63c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:06.794711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba0e98-56f7-4209-b706-5e2bbcd1f63c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:06.794988 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba0e98-56f7-4209-b706-5e2bbcd1f63c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:06.828801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f3ffb-ff13-4290-92d0-37f4195e9a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:06.830155 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f3ffb-ff13-4290-92d0-37f4195e9a99 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:06.830700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f3ffb-ff13-4290-92d0-37f4195e9a99 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:06.831192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f3ffb-ff13-4290-92d0-37f4195e9a99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:06.876180 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba0e98-56f7-4209-b706-5e2bbcd1f63c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:06.877702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba0e98-56f7-4209-b706-5e2bbcd1f63c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:06.924662 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f3ffb-ff13-4290-92d0-37f4195e9a99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:06.926165 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f3ffb-ff13-4290-92d0-37f4195e9a99 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:06.993416 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:07.027134 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 177/347] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:07.028591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba0e98-56f7-4209-b706-5e2bbcd1f63c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:07.074792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:07.076515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 171/348] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:07.078653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab2afa-7909-4765-a48e-3270badf9e5f None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:07.078998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab2afa-7909-4765-a48e-3270badf9e5f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:07.079210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab2afa-7909-4765-a48e-3270badf9e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:07.100784 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-94ab2afa-7909-4765-a48e-3270badf9e5f None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:07.102086 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-94ab2afa-7909-4765-a48e-3270badf9e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:07.172684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba0e98-56f7-4209-b706-5e2bbcd1f63c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:07.200812 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 172/349] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:07.203123 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e03b8-dd25-406f-b891-b9b361bfb129 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:07.203411 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e03b8-dd25-406f-b891-b9b361bfb129 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:07.203691 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e03b8-dd25-406f-b891-b9b361bfb129 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:07.205293 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 173/350] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:07.205893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e03b8-dd25-406f-b891-b9b361bfb129 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:07.223537 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa947ca-9e17-4692-a116-8c4ec0bc0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:07.224420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa947ca-9e17-4692-a116-8c4ec0bc0979 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:07.224785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa947ca-9e17-4692-a116-8c4ec0bc0979 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:07.225146 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa947ca-9e17-4692-a116-8c4ec0bc0979 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:07.226430 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa947ca-9e17-4692-a116-8c4ec0bc0979 None admin] RBAC: Adding query filter params (name=member) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:07.226430 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa947ca-9e17-4692-a116-8c4ec0bc0979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:07.227936 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa947ca-9e17-4692-a116-8c4ec0bc0979 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:07.244372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 178/351] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:07:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:07.244372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa947ca-9e17-4692-a116-8c4ec0bc0979 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:07.310998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:07.341801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e03b8-dd25-406f-b891-b9b361bfb129 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:07.362919 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:07.377551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa947ca-9e17-4692-a116-8c4ec0bc0979 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:07.390970 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ab1ac-4268-4188-8f7a-0b69dea7bfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:07.391851 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab1ac-4268-4188-8f7a-0b69dea7bfb6 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:07.392134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab1ac-4268-4188-8f7a-0b69dea7bfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:07.392471 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ab1ac-4268-4188-8f7a-0b69dea7bfb6 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:07.394884 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ab1ac-4268-4188-8f7a-0b69dea7bfb6 None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:07.396575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ab1ac-4268-4188-8f7a-0b69dea7bfb6 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:07.399163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 174/352] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:07 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:07.399559 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ac175-06db-4014-ac14-af73bbc84064 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:07.399912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ac175-06db-4014-ac14-af73bbc84064 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:07.400193 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ac175-06db-4014-ac14-af73bbc84064 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:07.401598 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 175/353] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:07.402671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5067c-f904-452e-a9c4-3cf12d9b42a4 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:07.402976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5067c-f904-452e-a9c4-3cf12d9b42a4 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:07.403211 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d5067c-f904-452e-a9c4-3cf12d9b42a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:07.422261 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb83bd0-1680-41d5-ab50-46bd9b0942e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:07.423173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb83bd0-1680-41d5-ab50-46bd9b0942e9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:07.423459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb83bd0-1680-41d5-ab50-46bd9b0942e9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:07.423737 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb83bd0-1680-41d5-ab50-46bd9b0942e9 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:07.425077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb83bd0-1680-41d5-ab50-46bd9b0942e9 None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:07.425811 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-d7d5067c-f904-452e-a9c4-3cf12d9b42a4 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:07.427563 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb83bd0-1680-41d5-ab50-46bd9b0942e9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:07.427700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-d7d5067c-f904-452e-a9c4-3cf12d9b42a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:07.432420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 179/354] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:07 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:07.432420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb83bd0-1680-41d5-ab50-46bd9b0942e9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:07.543641 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 176/355] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:07.551957 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5067c-f904-452e-a9c4-3cf12d9b42a4 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:07.572028 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb83bd0-1680-41d5-ab50-46bd9b0942e9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:07.615792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54655d7f-fb72-44ee-802a-9eaf349342f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:07.619988 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655d7f-fb72-44ee-802a-9eaf349342f0 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:07.619988 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655d7f-fb72-44ee-802a-9eaf349342f0 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:07.619988 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54655d7f-fb72-44ee-802a-9eaf349342f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:07.636028 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54655d7f-fb72-44ee-802a-9eaf349342f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-54655d7f-fb72-44ee-802a-9eaf349342f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:07.644116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:07.645348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:07.646288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 180/356] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:07.647143 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54655d7f-fb72-44ee-802a-9eaf349342f0 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:07.653588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:07.679412 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:07.696023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d5067c-f904-452e-a9c4-3cf12d9b42a4 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:07.743864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d8ad0-ded2-4d57-a42d-e5a08a73752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:07.744729 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8ad0-ded2-4d57-a42d-e5a08a73752e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:07.745017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8ad0-ded2-4d57-a42d-e5a08a73752e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:07.745350 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8ad0-ded2-4d57-a42d-e5a08a73752e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:07.782052 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54655d7f-fb72-44ee-802a-9eaf349342f0 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:07.831557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c57fd-b378-4822-8a9d-ee9a1381a6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:07.832368 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c57fd-b378-4822-8a9d-ee9a1381a6a2 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:07.832655 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c57fd-b378-4822-8a9d-ee9a1381a6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:07.832929 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c57fd-b378-4822-8a9d-ee9a1381a6a2 None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:07.841306 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c57fd-b378-4822-8a9d-ee9a1381a6a2 None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:07.842843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c57fd-b378-4822-8a9d-ee9a1381a6a2 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:07.844182 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d8ad0-ded2-4d57-a42d-e5a08a73752e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:07.845771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 181/357] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:07 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:07.845911 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d8ad0-ded2-4d57-a42d-e5a08a73752e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:07.846845 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3f126-4045-4b91-a11a-77ef9d010a03 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:07.847472 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3f126-4045-4b91-a11a-77ef9d010a03 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:07.847472 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3f126-4045-4b91-a11a-77ef9d010a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:07.869700 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-09f3f126-4045-4b91-a11a-77ef9d010a03 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:07.871234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-09f3f126-4045-4b91-a11a-77ef9d010a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:07.940765 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:07.970293 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 182/358] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:07.974367 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3f126-4045-4b91-a11a-77ef9d010a03 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:07.988837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:07.992640 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 177/359] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:07.992640 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d8ad0-ded2-4d57-a42d-e5a08a73752e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:08.108077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3f126-4045-4b91-a11a-77ef9d010a03 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:08.139469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d8ad0-ded2-4d57-a42d-e5a08a73752e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:08.153136 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a5b86-24c8-4aad-833e-c37da2132986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:08.154425 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a5b86-24c8-4aad-833e-c37da2132986 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:08.154885 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a5b86-24c8-4aad-833e-c37da2132986 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:08.155261 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a5b86-24c8-4aad-833e-c37da2132986 None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:08.157992 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a5b86-24c8-4aad-833e-c37da2132986 None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:08.160497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a5b86-24c8-4aad-833e-c37da2132986 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:08.163094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 183/360] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:07 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:08.166708 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a5b86-24c8-4aad-833e-c37da2132986 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:08.189136 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206e64c-f257-405a-83e7-1b9e053f0f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:08.189969 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206e64c-f257-405a-83e7-1b9e053f0f3e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:08.190253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206e64c-f257-405a-83e7-1b9e053f0f3e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:08.190525 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206e64c-f257-405a-83e7-1b9e053f0f3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:08.207600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206e64c-f257-405a-83e7-1b9e053f0f3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-f206e64c-f257-405a-83e7-1b9e053f0f3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:08.216102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:08.218835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:08.220976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 178/361] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:07:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:08.222528 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206e64c-f257-405a-83e7-1b9e053f0f3e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:08.274274 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:08.300971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:08.329546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a5b86-24c8-4aad-833e-c37da2132986 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:08.373670 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206e64c-f257-405a-83e7-1b9e053f0f3e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:08.379003 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37327eeb-9276-4d53-8f23-f5a048856854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:08.379669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327eeb-9276-4d53-8f23-f5a048856854 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:08.379949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327eeb-9276-4d53-8f23-f5a048856854 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:08.380221 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37327eeb-9276-4d53-8f23-f5a048856854 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:08.381058 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37327eeb-9276-4d53-8f23-f5a048856854 None admin] RBAC: Adding query filter params (name=reader) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:08.381356 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37327eeb-9276-4d53-8f23-f5a048856854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:08.383413 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37327eeb-9276-4d53-8f23-f5a048856854 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:08.393753 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 184/362] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:07:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:08.394892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37327eeb-9276-4d53-8f23-f5a048856854 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:08.415096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d712a-2245-4a95-91e5-112468b0b197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:08.415898 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d712a-2245-4a95-91e5-112468b0b197 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:08.416099 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d712a-2245-4a95-91e5-112468b0b197 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:08.416398 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d712a-2245-4a95-91e5-112468b0b197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:08.417250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d712a-2245-4a95-91e5-112468b0b197 None admin] RBAC: Adding query filter params (group.id=0e122408b3fc49a282523b4d0f77ae01, role.id=ba04654a403444e4ae54a1a216888fa5, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:08.417572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d712a-2245-4a95-91e5-112468b0b197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:08.419052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d712a-2245-4a95-91e5-112468b0b197 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:08.425970 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 179/363] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:07:08 2022] GET /identity/v3/role_assignments?group.id=0e122408b3fc49a282523b4d0f77ae01&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:08.427121 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d712a-2245-4a95-91e5-112468b0b197 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:08.541336 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37327eeb-9276-4d53-8f23-f5a048856854 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:08.552932 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:08.559677 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d712a-2245-4a95-91e5-112468b0b197 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:08.582920 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0814ef-7aa3-4195-86c1-9739b458ab36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:08.583869 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0814ef-7aa3-4195-86c1-9739b458ab36 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:08.584158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0814ef-7aa3-4195-86c1-9739b458ab36 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:08.584476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0814ef-7aa3-4195-86c1-9739b458ab36 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:08.585784 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0814ef-7aa3-4195-86c1-9739b458ab36 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:08.587331 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0814ef-7aa3-4195-86c1-9739b458ab36 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:08.589174 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 185/364] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:08 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:08.589984 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0814ef-7aa3-4195-86c1-9739b458ab36 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:08.606500 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12e84f-7282-41d8-9778-8c5c23d54324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:08.607659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12e84f-7282-41d8-9778-8c5c23d54324 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:08.608126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12e84f-7282-41d8-9778-8c5c23d54324 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:08.608625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12e84f-7282-41d8-9778-8c5c23d54324 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:08.609887 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:08.622524 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12e84f-7282-41d8-9778-8c5c23d54324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-6c12e84f-7282-41d8-9778-8c5c23d54324 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:08.630696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:08.631740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:08.632753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 180/365] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:07:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:08.634275 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12e84f-7282-41d8-9778-8c5c23d54324 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:08.711097 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0814ef-7aa3-4195-86c1-9739b458ab36 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:08.749489 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e54c8-5007-4d26-8ce9-32945bd0d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:08.750031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12e84f-7282-41d8-9778-8c5c23d54324 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:08.750483 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e54c8-5007-4d26-8ce9-32945bd0d91a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:08.750483 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e54c8-5007-4d26-8ce9-32945bd0d91a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:08.750755 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e54c8-5007-4d26-8ce9-32945bd0d91a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:08.751518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e54c8-5007-4d26-8ce9-32945bd0d91a None admin] RBAC: Adding query filter params (group.id=0e122408b3fc49a282523b4d0f77ae01, role.id=c9fbfe89c48d458c94f2a8a11652fcf2, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:08.751835 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e54c8-5007-4d26-8ce9-32945bd0d91a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:08.753350 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e54c8-5007-4d26-8ce9-32945bd0d91a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:08.759084 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 186/366] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:07:08 2022] GET /identity/v3/role_assignments?group.id=0e122408b3fc49a282523b4d0f77ae01&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=c9fbfe89c48d458c94f2a8a11652fcf2 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:08.759913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e54c8-5007-4d26-8ce9-32945bd0d91a None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:08.803393 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90878b-23b6-4488-95db-88beb32f0b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:08.804516 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90878b-23b6-4488-95db-88beb32f0b2f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:08.805035 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90878b-23b6-4488-95db-88beb32f0b2f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:08.805642 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90878b-23b6-4488-95db-88beb32f0b2f None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:08.806780 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90878b-23b6-4488-95db-88beb32f0b2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:08.807335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90878b-23b6-4488-95db-88beb32f0b2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:08.809200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90878b-23b6-4488-95db-88beb32f0b2f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:08.824954 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 181/367] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:08.826314 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90878b-23b6-4488-95db-88beb32f0b2f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:08.852462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:08.879875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:08.908591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e54c8-5007-4d26-8ce9-32945bd0d91a None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:08.953146 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd95a74-db80-4dca-a634-b69745ac8356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:08.954406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd95a74-db80-4dca-a634-b69745ac8356 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:08.954885 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd95a74-db80-4dca-a634-b69745ac8356 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:08.955338 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd95a74-db80-4dca-a634-b69745ac8356 None admin] PATH_INFO: `/v3/users/edb3ef1e4888407e880710ae18438732` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:08.958374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd95a74-db80-4dca-a634-b69745ac8356 None admin] RBAC: Authorizing `identity:get_user(user_id=edb3ef1e4888407e880710ae18438732)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:08.959897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd95a74-db80-4dca-a634-b69745ac8356 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:08.961670 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 187/368] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:07:08 2022] GET /identity/v3/users/edb3ef1e4888407e880710ae18438732 => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:08.962474 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd95a74-db80-4dca-a634-b69745ac8356 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:08.983219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90878b-23b6-4488-95db-88beb32f0b2f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:09.027632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcbd02-81e3-48d2-b1b1-8c305713e74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:09.028909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcbd02-81e3-48d2-b1b1-8c305713e74f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:09.029264 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcbd02-81e3-48d2-b1b1-8c305713e74f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:09.029918 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dcbd02-81e3-48d2-b1b1-8c305713e74f None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:09.031728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcbd02-81e3-48d2-b1b1-8c305713e74f None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:09.033298 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dcbd02-81e3-48d2-b1b1-8c305713e74f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:09.035904 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 182/369] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:08 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:09.036866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcbd02-81e3-48d2-b1b1-8c305713e74f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:09.099255 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd95a74-db80-4dca-a634-b69745ac8356 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:09.141825 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283a361-b32b-454c-af60-078c2f46008d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:09.142919 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283a361-b32b-454c-af60-078c2f46008d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:09.143442 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283a361-b32b-454c-af60-078c2f46008d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:09.144044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283a361-b32b-454c-af60-078c2f46008d None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:09.145063 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283a361-b32b-454c-af60-078c2f46008d None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:09.145593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283a361-b32b-454c-af60-078c2f46008d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:09.147325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283a361-b32b-454c-af60-078c2f46008d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:09.157038 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:09.159291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 188/370] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:09.161321 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283a361-b32b-454c-af60-078c2f46008d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:09.178162 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dcbd02-81e3-48d2-b1b1-8c305713e74f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:09.178524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:09.219030 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484faedf-f351-48a9-ae40-00ff3b6b3e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:09.219877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484faedf-f351-48a9-ae40-00ff3b6b3e8e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:09.220319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484faedf-f351-48a9-ae40-00ff3b6b3e8e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:09.220732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484faedf-f351-48a9-ae40-00ff3b6b3e8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:09.302013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283a361-b32b-454c-af60-078c2f46008d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:09.302683 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484faedf-f351-48a9-ae40-00ff3b6b3e8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:09.304459 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484faedf-f351-48a9-ae40-00ff3b6b3e8e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:09.348271 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d38e4-d774-426a-a6a1-e5ffdbae4e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:09.349728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d38e4-d774-426a-a6a1-e5ffdbae4e33 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:09.350195 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d38e4-d774-426a-a6a1-e5ffdbae4e33 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:09.350640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d38e4-d774-426a-a6a1-e5ffdbae4e33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:09.351642 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d38e4-d774-426a-a6a1-e5ffdbae4e33 None admin] RBAC: Adding query filter params (role.id=a0e21378e6134985a2ab8a70003fdad9, user.id=edb3ef1e4888407e880710ae18438732, scope.system=all) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:09.352569 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d38e4-d774-426a-a6a1-e5ffdbae4e33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:09.354579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d38e4-d774-426a-a6a1-e5ffdbae4e33 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:09.371824 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 189/371] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:07:09 2022] GET /identity/v3/role_assignments?user.id=edb3ef1e4888407e880710ae18438732&scope.system=all&role.id=a0e21378e6134985a2ab8a70003fdad9 => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:09.372990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d38e4-d774-426a-a6a1-e5ffdbae4e33 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:09.438859 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:09.470574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 183/372] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:09.472030 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484faedf-f351-48a9-ae40-00ff3b6b3e8e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:09.491476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:09.594721 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d38e4-d774-426a-a6a1-e5ffdbae4e33 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:09.627796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484faedf-f351-48a9-ae40-00ff3b6b3e8e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:09.670157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b5127-981d-4d58-9836-8fb71d31dafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:09.672374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b5127-981d-4d58-9836-8fb71d31dafc None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:09.672886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b5127-981d-4d58-9836-8fb71d31dafc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:09.673284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06295e04-749f-4b22-957a-bd44d27cda06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:09.673671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b5127-981d-4d58-9836-8fb71d31dafc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:09.673962 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06295e04-749f-4b22-957a-bd44d27cda06 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:09.674253 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06295e04-749f-4b22-957a-bd44d27cda06 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:09.674534 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06295e04-749f-4b22-957a-bd44d27cda06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:09.675389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06295e04-749f-4b22-957a-bd44d27cda06 None admin] RBAC: Adding query filter params (group.id=0e122408b3fc49a282523b4d0f77ae01, role.id=ba04654a403444e4ae54a1a216888fa5, scope.project.id=8ff5a20e721548c582f2c7a214733faf) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:09.675709 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06295e04-749f-4b22-957a-bd44d27cda06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:09.677357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06295e04-749f-4b22-957a-bd44d27cda06 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:09.685861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 190/373] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:07:09 2022] GET /identity/v3/role_assignments?group.id=0e122408b3fc49a282523b4d0f77ae01&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 260 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:09.686726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06295e04-749f-4b22-957a-bd44d27cda06 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:09.790067 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:09.821670 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b5127-981d-4d58-9836-8fb71d31dafc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:09.827588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b5127-981d-4d58-9836-8fb71d31dafc None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:09.922438 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06295e04-749f-4b22-957a-bd44d27cda06 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:09.969372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:09.981148 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64109c79-0d4b-4ba1-ae50-d16c82588343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:09.987567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64109c79-0d4b-4ba1-ae50-d16c82588343 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:09.987567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64109c79-0d4b-4ba1-ae50-d16c82588343 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:09.987567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64109c79-0d4b-4ba1-ae50-d16c82588343 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:09.987567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64109c79-0d4b-4ba1-ae50-d16c82588343 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:09.989880 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64109c79-0d4b-4ba1-ae50-d16c82588343 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:09.992230 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 191/374] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:09 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:09.998590 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64109c79-0d4b-4ba1-ae50-d16c82588343 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:10.050999 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 184/375] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:10.057633 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b5127-981d-4d58-9836-8fb71d31dafc None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:10.204871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64109c79-0d4b-4ba1-ae50-d16c82588343 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:10.233552 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:10.255553 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b5127-981d-4d58-9836-8fb71d31dafc None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:10.280128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01824574-296c-4a09-bce8-e2ee160d0894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:10.281699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01824574-296c-4a09-bce8-e2ee160d0894 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:10.281999 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01824574-296c-4a09-bce8-e2ee160d0894 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:10.282278 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01824574-296c-4a09-bce8-e2ee160d0894 None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:10.283519 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01824574-296c-4a09-bce8-e2ee160d0894 None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:10.285384 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01824574-296c-4a09-bce8-e2ee160d0894 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:10.290897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 192/376] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:09 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:10.297010 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01824574-296c-4a09-bce8-e2ee160d0894 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:10.311036 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9b80d-f06d-444b-b176-dd034a75d8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:10.312215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9b80d-f06d-444b-b176-dd034a75d8c6 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:10.312534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9b80d-f06d-444b-b176-dd034a75d8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:10.313113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9b80d-f06d-444b-b176-dd034a75d8c6 None admin] PATH_INFO: `/v3/projects/4f2a61498743436aab0f28a2590bd946` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:10.315573 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9b80d-f06d-444b-b176-dd034a75d8c6 None admin] RBAC: Authorizing `identity:get_project(project_id=4f2a61498743436aab0f28a2590bd946)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:10.317815 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9b80d-f06d-444b-b176-dd034a75d8c6 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:10.320078 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 185/377] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:10 2022] GET /identity/v3/projects/4f2a61498743436aab0f28a2590bd946 => generated 344 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:10.326786 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9b80d-f06d-444b-b176-dd034a75d8c6 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:10.422991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:10.490325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01824574-296c-4a09-bce8-e2ee160d0894 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:10.523589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9b80d-f06d-444b-b176-dd034a75d8c6 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:10.537146 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2f823-9ca3-4823-9be2-9c18739065ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:10.537836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2f823-9ca3-4823-9be2-9c18739065ca None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:10.538127 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2f823-9ca3-4823-9be2-9c18739065ca None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:10.538408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2f823-9ca3-4823-9be2-9c18739065ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:10.539253 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2f823-9ca3-4823-9be2-9c18739065ca None admin] RBAC: Adding query filter params (group.id=0e122408b3fc49a282523b4d0f77ae01, role.id=c9fbfe89c48d458c94f2a8a11652fcf2, scope.project.id=8ff5a20e721548c582f2c7a214733faf) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:10.539571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2f823-9ca3-4823-9be2-9c18739065ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:10.541077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2f823-9ca3-4823-9be2-9c18739065ca None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:10.548636 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 193/378] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:07:10 2022] GET /identity/v3/role_assignments?group.id=0e122408b3fc49a282523b4d0f77ae01&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=c9fbfe89c48d458c94f2a8a11652fcf2 => generated 260 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:10.581952 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0eb774-c89d-481d-9a69-a4d3c4e680cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:10.582753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0eb774-c89d-481d-9a69-a4d3c4e680cb None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:10.583039 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0eb774-c89d-481d-9a69-a4d3c4e680cb None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:10.583319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0eb774-c89d-481d-9a69-a4d3c4e680cb None admin] PATH_INFO: `/v3/groups/ef98e694996341f9aa5a0c64f3897a86` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:10.599758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0eb774-c89d-481d-9a69-a4d3c4e680cb None admin] RBAC: Authorizing `identity:get_group(group_id=ef98e694996341f9aa5a0c64f3897a86)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:10.601328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0eb774-c89d-481d-9a69-a4d3c4e680cb None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:10.603585 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 186/379] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:10 2022] GET /identity/v3/groups/ef98e694996341f9aa5a0c64f3897a86 => generated 229 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:10.610028 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2f823-9ca3-4823-9be2-9c18739065ca None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:10.776293 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2f823-9ca3-4823-9be2-9c18739065ca None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:10.810772 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:10.820594 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f505e8-e114-4df6-a6b9-c13663b68f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:10.823762 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f505e8-e114-4df6-a6b9-c13663b68f75 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:10.823762 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f505e8-e114-4df6-a6b9-c13663b68f75 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:10.823762 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f505e8-e114-4df6-a6b9-c13663b68f75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:10.823762 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f505e8-e114-4df6-a6b9-c13663b68f75 None admin] RBAC: Adding query filter params (group.id=ef98e694996341f9aa5a0c64f3897a86, role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=4f2a61498743436aab0f28a2590bd946) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:10.823762 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f505e8-e114-4df6-a6b9-c13663b68f75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:10.824547 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f505e8-e114-4df6-a6b9-c13663b68f75 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:10.832201 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 194/380] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:07:10 2022] GET /identity/v3/role_assignments?group.id=ef98e694996341f9aa5a0c64f3897a86&scope.project.id=4f2a61498743436aab0f28a2590bd946&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:11.137317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485dd48-2c26-4fcf-a8cf-7c47eb3b44de None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:11.137871 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485dd48-2c26-4fcf-a8cf-7c47eb3b44de None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:11.137871 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485dd48-2c26-4fcf-a8cf-7c47eb3b44de None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:11.139666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 187/381] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:11.146676 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d0eef-4616-458d-8bf7-e0a8a686c06c None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:11.147226 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d0eef-4616-458d-8bf7-e0a8a686c06c None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:11.147226 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d0eef-4616-458d-8bf7-e0a8a686c06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:11.177723 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-ff8d0eef-4616-458d-8bf7-e0a8a686c06c None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:11.179837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-ff8d0eef-4616-458d-8bf7-e0a8a686c06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:11.250991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334aef5-148f-4c1d-904b-8b22370fd0c6 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:11.251348 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334aef5-148f-4c1d-904b-8b22370fd0c6 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:11.251552 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334aef5-148f-4c1d-904b-8b22370fd0c6 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:11.252980 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 188/382] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:07:11.258738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c56d6-e14d-41fa-a20a-e8d364bb336e None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:11.259161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c56d6-e14d-41fa-a20a-e8d364bb336e None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:11.259544 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c56d6-e14d-41fa-a20a-e8d364bb336e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:11.290481 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-9a1c56d6-e14d-41fa-a20a-e8d364bb336e None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:11.291741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-9a1c56d6-e14d-41fa-a20a-e8d364bb336e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:11.319124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:11.325291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 195/383] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:11.348236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d0eef-4616-458d-8bf7-e0a8a686c06c None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:11.490269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 189/384] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:11.520872 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d0eef-4616-458d-8bf7-e0a8a686c06c None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:11.555998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c56d6-e14d-41fa-a20a-e8d364bb336e None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:11.557483 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:11.570493 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d0b3b-af10-45b6-a2aa-6679d93b604e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:11.571294 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0b3b-af10-45b6-a2aa-6679d93b604e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:11.571573 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0b3b-af10-45b6-a2aa-6679d93b604e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:11.571845 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0b3b-af10-45b6-a2aa-6679d93b604e None admin] PATH_INFO: `/v3/roles/member` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:11.584844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d0b3b-af10-45b6-a2aa-6679d93b604e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-d01d0b3b-af10-45b6-a2aa-6679d93b604e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:11.594818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:11.595973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:11.596973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 196/385] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:11.604872 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d0b3b-af10-45b6-a2aa-6679d93b604e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:11.763022 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c56d6-e14d-41fa-a20a-e8d364bb336e None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:11.811212 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d0b3b-af10-45b6-a2aa-6679d93b604e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:11.835429 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12e9e2-66c3-4d43-abe3-57c74beac561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:11.837513 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12e9e2-66c3-4d43-abe3-57c74beac561 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:11.838226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12e9e2-66c3-4d43-abe3-57c74beac561 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:11.840225 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12e9e2-66c3-4d43-abe3-57c74beac561 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:11.863899 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe12e9e2-66c3-4d43-abe3-57c74beac561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-fe12e9e2-66c3-4d43-abe3-57c74beac561 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:11.875286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:11.876505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:11.877493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:11.888167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 190/386] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:07:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:11.891019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c85c1-3ef9-4532-ae69-1250a1592fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:11.892063 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c85c1-3ef9-4532-ae69-1250a1592fe2 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:11.892655 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c85c1-3ef9-4532-ae69-1250a1592fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:11.893173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c85c1-3ef9-4532-ae69-1250a1592fe2 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:11.894283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12e9e2-66c3-4d43-abe3-57c74beac561 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:11.895282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c85c1-3ef9-4532-ae69-1250a1592fe2 None admin] RBAC: Adding query filter params (name=member) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:11.896213 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c85c1-3ef9-4532-ae69-1250a1592fe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:11.898999 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c85c1-3ef9-4532-ae69-1250a1592fe2 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:11.917865 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 197/387] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:07:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:11.919032 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:11.925154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c85c1-3ef9-4532-ae69-1250a1592fe2 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:11.977465 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:12.056493 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c85c1-3ef9-4532-ae69-1250a1592fe2 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:12.092756 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec7dcf-ef5b-40a5-8a04-92b8693be08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:12.094207 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec7dcf-ef5b-40a5-8a04-92b8693be08b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:12.094494 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec7dcf-ef5b-40a5-8a04-92b8693be08b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:12.094774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec7dcf-ef5b-40a5-8a04-92b8693be08b None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:12.096013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec7dcf-ef5b-40a5-8a04-92b8693be08b None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:12.097530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ec7dcf-ef5b-40a5-8a04-92b8693be08b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:12.099504 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 198/388] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:11 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:12.105082 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec7dcf-ef5b-40a5-8a04-92b8693be08b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:12.115327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12e9e2-66c3-4d43-abe3-57c74beac561 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:12.169353 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e767d8-c8cb-4f44-8853-46cce41280bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:12.170089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e767d8-c8cb-4f44-8853-46cce41280bc None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:12.170338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e767d8-c8cb-4f44-8853-46cce41280bc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:12.170582 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e767d8-c8cb-4f44-8853-46cce41280bc None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:12.171669 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e767d8-c8cb-4f44-8853-46cce41280bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:12.171983 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e767d8-c8cb-4f44-8853-46cce41280bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:12.173431 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e767d8-c8cb-4f44-8853-46cce41280bc None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:12.199709 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 191/389] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:12.203422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e767d8-c8cb-4f44-8853-46cce41280bc None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:12.316494 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec7dcf-ef5b-40a5-8a04-92b8693be08b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:12.330567 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:12.359826 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:12.394404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f1abf-898b-4d32-a35d-0bad62e47f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:12.395309 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f1abf-898b-4d32-a35d-0bad62e47f0e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:12.395599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f1abf-898b-4d32-a35d-0bad62e47f0e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:12.395878 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611f1abf-898b-4d32-a35d-0bad62e47f0e None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:12.399531 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f1abf-898b-4d32-a35d-0bad62e47f0e None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:12.402936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e767d8-c8cb-4f44-8853-46cce41280bc None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:12.404918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611f1abf-898b-4d32-a35d-0bad62e47f0e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:12.406756 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 199/390] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:12 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:12.415562 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f1abf-898b-4d32-a35d-0bad62e47f0e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:12.487204 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cc76e-9191-471a-bdfa-867e26d7f799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:12.488089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cc76e-9191-471a-bdfa-867e26d7f799 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:12.488402 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cc76e-9191-471a-bdfa-867e26d7f799 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:12.488685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632cc76e-9191-471a-bdfa-867e26d7f799 None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:12.494364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cc76e-9191-471a-bdfa-867e26d7f799 None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:12.495974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632cc76e-9191-471a-bdfa-867e26d7f799 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:12.498077 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 192/391] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:12 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:12.508553 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cc76e-9191-471a-bdfa-867e26d7f799 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:12.632145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632cc76e-9191-471a-bdfa-867e26d7f799 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:12.661645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611f1abf-898b-4d32-a35d-0bad62e47f0e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:12.679703 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2379e-333d-4931-9beb-873dc97ef365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:12.680720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2379e-333d-4931-9beb-873dc97ef365 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:12.681017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2379e-333d-4931-9beb-873dc97ef365 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:12.681301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2379e-333d-4931-9beb-873dc97ef365 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:12.683362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2379e-333d-4931-9beb-873dc97ef365 None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:12.684982 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2379e-333d-4931-9beb-873dc97ef365 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:12.687042 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 193/392] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:12 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:12.693555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2379e-333d-4931-9beb-873dc97ef365 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:12.712805 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:12.730380 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab6a5f-1aef-44f8-b85a-5c6c611c5e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:12.733745 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab6a5f-1aef-44f8-b85a-5c6c611c5e87 None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:12.734038 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab6a5f-1aef-44f8-b85a-5c6c611c5e87 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:12.735337 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab6a5f-1aef-44f8-b85a-5c6c611c5e87 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/groups/0e122408b3fc49a282523b4d0f77ae01/roles/ba04654a403444e4ae54a1a216888fa5` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:12.739554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab6a5f-1aef-44f8-b85a-5c6c611c5e87 None admin] RBAC: Authorizing `identity:create_grant(project_id=003a311d04e34ac4ab05c59a55d5a47a, group_id=0e122408b3fc49a282523b4d0f77ae01, role_id=ba04654a403444e4ae54a1a216888fa5)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:12.741077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab6a5f-1aef-44f8-b85a-5c6c611c5e87 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:12.818004 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 200/393] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:07:12 2022] PUT /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/groups/0e122408b3fc49a282523b4d0f77ae01/roles/ba04654a403444e4ae54a1a216888fa5 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:07:12.819396 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8a576-8fe6-4171-aa60-c43d8eefe669 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:12.819691 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8a576-8fe6-4171-aa60-c43d8eefe669 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:12.819977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8a576-8fe6-4171-aa60-c43d8eefe669 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:12.829206 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 201/394] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:12.837916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4572dd2-f933-42cd-a8dc-d8c9122ae79b None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:12.838255 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4572dd2-f933-42cd-a8dc-d8c9122ae79b None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:12.838570 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4572dd2-f933-42cd-a8dc-d8c9122ae79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:12.868924 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2379e-333d-4931-9beb-873dc97ef365 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:12.893859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-f4572dd2-f933-42cd-a8dc-d8c9122ae79b None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:12.895278 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-f4572dd2-f933-42cd-a8dc-d8c9122ae79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:12.917752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065ce15-9c33-4877-90af-a01bcfd7bded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:12.919222 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065ce15-9c33-4877-90af-a01bcfd7bded None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:12.919859 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065ce15-9c33-4877-90af-a01bcfd7bded None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:12.920388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065ce15-9c33-4877-90af-a01bcfd7bded None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/groups/0e122408b3fc49a282523b4d0f77ae01/roles/c9fbfe89c48d458c94f2a8a11652fcf2` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:12.923674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065ce15-9c33-4877-90af-a01bcfd7bded None admin] RBAC: Authorizing `identity:create_grant(project_id=003a311d04e34ac4ab05c59a55d5a47a, group_id=0e122408b3fc49a282523b4d0f77ae01, role_id=c9fbfe89c48d458c94f2a8a11652fcf2)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:12.926149 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065ce15-9c33-4877-90af-a01bcfd7bded None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:12.964969 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:13.010887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 194/395] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:07:12 2022] PUT /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a/groups/0e122408b3fc49a282523b4d0f77ae01/roles/c9fbfe89c48d458c94f2a8a11652fcf2 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:07:13.014043 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 202/396] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:13.053005 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065ce15-9c33-4877-90af-a01bcfd7bded None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:13.164138 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:13.220576 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065ce15-9c33-4877-90af-a01bcfd7bded None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:13.273292 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2ee92-35e2-4583-9f0b-c6ea68761589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:13.274107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ee92-35e2-4583-9f0b-c6ea68761589 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:13.274389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ee92-35e2-4583-9f0b-c6ea68761589 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:13.274664 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2ee92-35e2-4583-9f0b-c6ea68761589 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:13.287533 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2ee92-35e2-4583-9f0b-c6ea68761589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-d7f2ee92-35e2-4583-9f0b-c6ea68761589 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:13.295692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:13.297630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:13.298589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 195/397] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:13.305720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4572dd2-f933-42cd-a8dc-d8c9122ae79b None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:13.401671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d8927-da6f-4273-a9ae-996d3629ff53 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:13.402039 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d8927-da6f-4273-a9ae-996d3629ff53 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:13.402230 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d8927-da6f-4273-a9ae-996d3629ff53 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:13.404478 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 196/398] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:13.410327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089597d2-101c-4ae9-9977-50ccb72bdcdd None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:13.410668 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089597d2-101c-4ae9-9977-50ccb72bdcdd None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:13.410903 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089597d2-101c-4ae9-9977-50ccb72bdcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:13.431464 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4572dd2-f933-42cd-a8dc-d8c9122ae79b None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:13.432605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-089597d2-101c-4ae9-9977-50ccb72bdcdd None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:13.433989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-089597d2-101c-4ae9-9977-50ccb72bdcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:13.472102 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6871bf6-2e54-4efe-8e2e-add1fc0c9cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:13.472857 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6871bf6-2e54-4efe-8e2e-add1fc0c9cc3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:13.473144 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6871bf6-2e54-4efe-8e2e-add1fc0c9cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:13.473422 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6871bf6-2e54-4efe-8e2e-add1fc0c9cc3 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:13.474212 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6871bf6-2e54-4efe-8e2e-add1fc0c9cc3 None admin] RBAC: Adding query filter params (name=member) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:13.474529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6871bf6-2e54-4efe-8e2e-add1fc0c9cc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:13.476006 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6871bf6-2e54-4efe-8e2e-add1fc0c9cc3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:13.487219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 203/399] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:07:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:13.493964 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6871bf6-2e54-4efe-8e2e-add1fc0c9cc3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:13.572747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:13.610682 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 197/400] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:07:13.648428 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089597d2-101c-4ae9-9977-50ccb72bdcdd None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:13.649128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6871bf6-2e54-4efe-8e2e-add1fc0c9cc3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:13.714655 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dfc3a-3eaf-4053-9049-73b5a775f624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:13.715454 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dfc3a-3eaf-4053-9049-73b5a775f624 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:13.715743 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dfc3a-3eaf-4053-9049-73b5a775f624 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:13.716027 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dfc3a-3eaf-4053-9049-73b5a775f624 None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:13.717321 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801dfc3a-3eaf-4053-9049-73b5a775f624 None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:13.719387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801dfc3a-3eaf-4053-9049-73b5a775f624 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:13.721455 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 204/401] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:13 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:13.726874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dfc3a-3eaf-4053-9049-73b5a775f624 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:13.754791 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:13.808583 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089597d2-101c-4ae9-9977-50ccb72bdcdd None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:13.849330 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51560ac-f0ff-4f4a-9f7e-15f644f5aeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:13.850226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51560ac-f0ff-4f4a-9f7e-15f644f5aeb7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:13.850512 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51560ac-f0ff-4f4a-9f7e-15f644f5aeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:13.850790 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51560ac-f0ff-4f4a-9f7e-15f644f5aeb7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:13.862729 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51560ac-f0ff-4f4a-9f7e-15f644f5aeb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:13.869789 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:13.873933 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dfc3a-3eaf-4053-9049-73b5a775f624 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-d51560ac-f0ff-4f4a-9f7e-15f644f5aeb7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:13.877227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:13.878331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:13.879689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:13.892636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 198/402] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:07:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:13.892636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02aa6-ff37-4017-ab93-b8a03255960c None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:13.892636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02aa6-ff37-4017-ab93-b8a03255960c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:13.892636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02aa6-ff37-4017-ab93-b8a03255960c None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:13.892636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 199/403] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:13.894594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb18db4-ca3c-4e77-a050-f4bf4405a5ef None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:13.894889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb18db4-ca3c-4e77-a050-f4bf4405a5ef None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:13.895173 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb18db4-ca3c-4e77-a050-f4bf4405a5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:13.917219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-ecb18db4-ca3c-4e77-a050-f4bf4405a5ef None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:13.919863 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-ecb18db4-ca3c-4e77-a050-f4bf4405a5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:13.927523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3575d-78c6-4cf6-9445-e37b7f71e4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:13.928816 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3575d-78c6-4cf6-9445-e37b7f71e4fd None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:13.929929 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3575d-78c6-4cf6-9445-e37b7f71e4fd None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:13.930448 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3575d-78c6-4cf6-9445-e37b7f71e4fd None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:13.932536 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3575d-78c6-4cf6-9445-e37b7f71e4fd None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:13.935134 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3575d-78c6-4cf6-9445-e37b7f71e4fd None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:13.939247 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 205/404] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:13 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:13.940796 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3575d-78c6-4cf6-9445-e37b7f71e4fd None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:14.008582 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 200/405] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:14.009962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb18db4-ca3c-4e77-a050-f4bf4405a5ef None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:14.075719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:14.088909 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3575d-78c6-4cf6-9445-e37b7f71e4fd None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:14.132326 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe6b59-78cf-4023-a07c-b65fe26f2841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:14.133711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe6b59-78cf-4023-a07c-b65fe26f2841 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:14.134213 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe6b59-78cf-4023-a07c-b65fe26f2841 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:14.134756 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe6b59-78cf-4023-a07c-b65fe26f2841 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:14.136157 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe6b59-78cf-4023-a07c-b65fe26f2841 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:14.136880 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe6b59-78cf-4023-a07c-b65fe26f2841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:14.139612 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe6b59-78cf-4023-a07c-b65fe26f2841 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:14.141107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb18db4-ca3c-4e77-a050-f4bf4405a5ef None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:14.164222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 206/406] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:14.166068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe6b59-78cf-4023-a07c-b65fe26f2841 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:14.182170 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106f213-5bd4-42e6-9f0d-20c83e2cdc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:14.183534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f213-5bd4-42e6-9f0d-20c83e2cdc05 None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:14.183998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f213-5bd4-42e6-9f0d-20c83e2cdc05 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:14.184474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106f213-5bd4-42e6-9f0d-20c83e2cdc05 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf/groups/0e122408b3fc49a282523b4d0f77ae01/roles/ba04654a403444e4ae54a1a216888fa5` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:14.186998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106f213-5bd4-42e6-9f0d-20c83e2cdc05 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5a20e721548c582f2c7a214733faf, group_id=0e122408b3fc49a282523b4d0f77ae01, role_id=ba04654a403444e4ae54a1a216888fa5)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:14.189211 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106f213-5bd4-42e6-9f0d-20c83e2cdc05 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:14.205074 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:14.283584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe6b59-78cf-4023-a07c-b65fe26f2841 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:14.292391 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 201/407] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:07:14 2022] PUT /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf/groups/0e122408b3fc49a282523b4d0f77ae01/roles/ba04654a403444e4ae54a1a216888fa5 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:07:14.292391 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106f213-5bd4-42e6-9f0d-20c83e2cdc05 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:14.364110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c71ad-4da5-4779-98d4-d0189cda68fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:14.364981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c71ad-4da5-4779-98d4-d0189cda68fa None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:14.365266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c71ad-4da5-4779-98d4-d0189cda68fa None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:14.365543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c71ad-4da5-4779-98d4-d0189cda68fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:14.379781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c71ad-4da5-4779-98d4-d0189cda68fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-670c71ad-4da5-4779-98d4-d0189cda68fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:14.387949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:14.389162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:14.390177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 207/408] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:07:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:14.397802 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c71ad-4da5-4779-98d4-d0189cda68fa None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:14.522659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106f213-5bd4-42e6-9f0d-20c83e2cdc05 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:14.534970 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:14.586252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f0d45-9a22-446f-a9d1-4e251dcd9ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:14.587533 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f0d45-9a22-446f-a9d1-4e251dcd9ac9 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:14.588075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f0d45-9a22-446f-a9d1-4e251dcd9ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:14.588700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f0d45-9a22-446f-a9d1-4e251dcd9ac9 None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:14.590327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f0d45-9a22-446f-a9d1-4e251dcd9ac9 None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:14.592052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f0d45-9a22-446f-a9d1-4e251dcd9ac9 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:14.593938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:14.596656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 202/409] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:14 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:14.602556 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f0d45-9a22-446f-a9d1-4e251dcd9ac9 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:14.605305 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c71ad-4da5-4779-98d4-d0189cda68fa None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:14.660030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082a060-802e-44f6-acd0-15d6babef7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:14.661151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082a060-802e-44f6-acd0-15d6babef7c9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:14.662880 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082a060-802e-44f6-acd0-15d6babef7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:14.663402 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082a060-802e-44f6-acd0-15d6babef7c9 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:14.664421 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082a060-802e-44f6-acd0-15d6babef7c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:14.665651 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082a060-802e-44f6-acd0-15d6babef7c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:14.667325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082a060-802e-44f6-acd0-15d6babef7c9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:14.689605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 208/410] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:14.697299 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082a060-802e-44f6-acd0-15d6babef7c9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:14.769457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f0d45-9a22-446f-a9d1-4e251dcd9ac9 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:14.841478 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1baa8-1aa6-4f58-a3fa-c0895eb5ae14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:14.842739 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1baa8-1aa6-4f58-a3fa-c0895eb5ae14 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:14.843807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1baa8-1aa6-4f58-a3fa-c0895eb5ae14 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:14.844306 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1baa8-1aa6-4f58-a3fa-c0895eb5ae14 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:14.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1baa8-1aa6-4f58-a3fa-c0895eb5ae14 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:14.847766 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1baa8-1aa6-4f58-a3fa-c0895eb5ae14 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:14.856300 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 203/411] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:14 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:14.857527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1baa8-1aa6-4f58-a3fa-c0895eb5ae14 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:14.903504 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082a060-802e-44f6-acd0-15d6babef7c9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:14.936758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:14.968632 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf01b98-cd80-4600-891a-8c4585f775c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:14.969502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf01b98-cd80-4600-891a-8c4585f775c1 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:14.969786 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf01b98-cd80-4600-891a-8c4585f775c1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:14.970068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf01b98-cd80-4600-891a-8c4585f775c1 None admin] PATH_INFO: `/v3/groups/ef98e694996341f9aa5a0c64f3897a86` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:14.971970 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf01b98-cd80-4600-891a-8c4585f775c1 None admin] RBAC: Authorizing `identity:get_group(group_id=ef98e694996341f9aa5a0c64f3897a86)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:14.973470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf01b98-cd80-4600-891a-8c4585f775c1 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:14.976187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 209/412] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:14 2022] GET /identity/v3/groups/ef98e694996341f9aa5a0c64f3897a86 => generated 229 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:14.982159 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf01b98-cd80-4600-891a-8c4585f775c1 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:15.008782 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:15.055824 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1baa8-1aa6-4f58-a3fa-c0895eb5ae14 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:15.095611 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf01b98-cd80-4600-891a-8c4585f775c1 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:15.133843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d31b0-b4f7-4158-a71b-81c21294177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:15.134769 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d31b0-b4f7-4158-a71b-81c21294177d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:15.135058 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d31b0-b4f7-4158-a71b-81c21294177d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:15.135340 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d31b0-b4f7-4158-a71b-81c21294177d None admin] PATH_INFO: `/v3/projects/4f2a61498743436aab0f28a2590bd946` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:15.136620 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d31b0-b4f7-4158-a71b-81c21294177d None admin] RBAC: Authorizing `identity:get_project(project_id=4f2a61498743436aab0f28a2590bd946)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:15.138981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d31b0-b4f7-4158-a71b-81c21294177d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:15.146137 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 210/413] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:14 2022] GET /identity/v3/projects/4f2a61498743436aab0f28a2590bd946 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:15.148055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d31b0-b4f7-4158-a71b-81c21294177d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:15.185215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b8224-50d3-4368-8b5a-102a00b960ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:15.186149 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b8224-50d3-4368-8b5a-102a00b960ee None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:15.186436 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b8224-50d3-4368-8b5a-102a00b960ee None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:15.186740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b8224-50d3-4368-8b5a-102a00b960ee None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf/groups/0e122408b3fc49a282523b4d0f77ae01/roles/c9fbfe89c48d458c94f2a8a11652fcf2` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:15.190397 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b8224-50d3-4368-8b5a-102a00b960ee None admin] RBAC: Authorizing `identity:create_grant(project_id=8ff5a20e721548c582f2c7a214733faf, group_id=0e122408b3fc49a282523b4d0f77ae01, role_id=c9fbfe89c48d458c94f2a8a11652fcf2)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:15.192326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b8224-50d3-4368-8b5a-102a00b960ee None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:15.255761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:15.268992 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d31b0-b4f7-4158-a71b-81c21294177d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:15.299450 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 204/414] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:07:14 2022] PUT /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf/groups/0e122408b3fc49a282523b4d0f77ae01/roles/c9fbfe89c48d458c94f2a8a11652fcf2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:07:15.308163 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87721d26-7886-4b1a-9395-afae75c30732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:15.309208 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87721d26-7886-4b1a-9395-afae75c30732 None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:15.309498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87721d26-7886-4b1a-9395-afae75c30732 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:15.309777 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87721d26-7886-4b1a-9395-afae75c30732 None admin] PATH_INFO: `/v3/projects/4f2a61498743436aab0f28a2590bd946/groups/ef98e694996341f9aa5a0c64f3897a86/roles/ab7a63cec3a744848972fcbb89ecd0ca` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:15.312199 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87721d26-7886-4b1a-9395-afae75c30732 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f2a61498743436aab0f28a2590bd946, group_id=ef98e694996341f9aa5a0c64f3897a86, role_id=ab7a63cec3a744848972fcbb89ecd0ca)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:15.313792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87721d26-7886-4b1a-9395-afae75c30732 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:15.409185 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 211/415] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:07:15 2022] PUT /identity/v3/projects/4f2a61498743436aab0f28a2590bd946/groups/ef98e694996341f9aa5a0c64f3897a86/roles/ab7a63cec3a744848972fcbb89ecd0ca => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:07:15.685025 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4951bc9-265e-4cdd-b027-49a119f3cca3 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:15.685469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4951bc9-265e-4cdd-b027-49a119f3cca3 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:15.685969 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4951bc9-265e-4cdd-b027-49a119f3cca3 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:15.688968 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 205/416] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:15.694816 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60e0d-4d4d-44e9-827d-2093f2c53c25 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:15.695810 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60e0d-4d4d-44e9-827d-2093f2c53c25 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:15.696327 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec60e0d-4d4d-44e9-827d-2093f2c53c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:15.723961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-9ec60e0d-4d4d-44e9-827d-2093f2c53c25 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:15.725708 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-9ec60e0d-4d4d-44e9-827d-2093f2c53c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:15.801239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69fa12-29ae-49e9-bb66-822b1aac7849 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:15.801590 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69fa12-29ae-49e9-bb66-822b1aac7849 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:15.801796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69fa12-29ae-49e9-bb66-822b1aac7849 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:15.803468 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 206/417] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:15.808925 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43114c0-abcc-4731-a591-c27974bfbc5c None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:15.809268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43114c0-abcc-4731-a591-c27974bfbc5c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:15.809494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43114c0-abcc-4731-a591-c27974bfbc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:15.821041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 212/418] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:15.838101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-e43114c0-abcc-4731-a591-c27974bfbc5c None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:15.839359 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-e43114c0-abcc-4731-a591-c27974bfbc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:15.871925 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec60e0d-4d4d-44e9-827d-2093f2c53c25 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:15.970210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 207/419] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:16.010290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43114c0-abcc-4731-a591-c27974bfbc5c None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:16.024822 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:16.052515 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec60e0d-4d4d-44e9-827d-2093f2c53c25 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:16.068362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:16.109600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675959d-ad80-43b0-aa76-8d40ab119906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:16.110462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675959d-ad80-43b0-aa76-8d40ab119906 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:16.116476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675959d-ad80-43b0-aa76-8d40ab119906 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:16.116476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675959d-ad80-43b0-aa76-8d40ab119906 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:16.123729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e675959d-ad80-43b0-aa76-8d40ab119906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-e675959d-ad80-43b0-aa76-8d40ab119906 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:16.133060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:16.134133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:16.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 213/420] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:16.144105 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675959d-ad80-43b0-aa76-8d40ab119906 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:16.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43114c0-abcc-4731-a591-c27974bfbc5c None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:16.290463 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed256d-aa15-4c29-8e73-7b3de45c2a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:16.291302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed256d-aa15-4c29-8e73-7b3de45c2a64 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:16.291590 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed256d-aa15-4c29-8e73-7b3de45c2a64 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:16.295259 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed256d-aa15-4c29-8e73-7b3de45c2a64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:16.321415 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed256d-aa15-4c29-8e73-7b3de45c2a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-01ed256d-aa15-4c29-8e73-7b3de45c2a64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:16.341037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:16.342207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:16.343764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:16.348366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 208/421] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:07:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:16.352466 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed256d-aa15-4c29-8e73-7b3de45c2a64 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:16.374957 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675959d-ad80-43b0-aa76-8d40ab119906 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:16.428315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb81d7-9bc2-44dd-9668-ef2d77786326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:16.429046 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb81d7-9bc2-44dd-9668-ef2d77786326 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:16.429330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb81d7-9bc2-44dd-9668-ef2d77786326 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:16.429625 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb81d7-9bc2-44dd-9668-ef2d77786326 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:16.430370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb81d7-9bc2-44dd-9668-ef2d77786326 None admin] RBAC: Adding query filter params (name=member) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:16.430686 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb81d7-9bc2-44dd-9668-ef2d77786326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:16.432121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb81d7-9bc2-44dd-9668-ef2d77786326 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:16.445484 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:16.451294 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 214/422] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:07:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:16.460999 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb81d7-9bc2-44dd-9668-ef2d77786326 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:16.556802 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:16.570075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed256d-aa15-4c29-8e73-7b3de45c2a64 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:16.671237 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90ed23-3307-43af-9a3b-ff57b0875dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:16.671933 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90ed23-3307-43af-9a3b-ff57b0875dcd None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:16.672214 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90ed23-3307-43af-9a3b-ff57b0875dcd None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:16.672597 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90ed23-3307-43af-9a3b-ff57b0875dcd None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:16.673366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90ed23-3307-43af-9a3b-ff57b0875dcd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:16.673676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90ed23-3307-43af-9a3b-ff57b0875dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:16.675096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90ed23-3307-43af-9a3b-ff57b0875dcd None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:16.687593 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 209/423] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:16.694669 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90ed23-3307-43af-9a3b-ff57b0875dcd None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:16.700173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb81d7-9bc2-44dd-9668-ef2d77786326 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:16.784548 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1775053-70e3-4be0-9e32-cb3ae505ede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:16.785406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1775053-70e3-4be0-9e32-cb3ae505ede2 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:16.785691 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1775053-70e3-4be0-9e32-cb3ae505ede2 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:16.785968 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1775053-70e3-4be0-9e32-cb3ae505ede2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:16.908420 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90ed23-3307-43af-9a3b-ff57b0875dcd None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:16.928297 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:16.937958 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1775053-70e3-4be0-9e32-cb3ae505ede2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:16.939767 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1775053-70e3-4be0-9e32-cb3ae505ede2 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:16.985411 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21caa5e3-e69c-429d-a98f-93e95d7df78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:16.986250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa5e3-e69c-429d-a98f-93e95d7df78c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:16.986579 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa5e3-e69c-429d-a98f-93e95d7df78c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:16.986894 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21caa5e3-e69c-429d-a98f-93e95d7df78c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:17.065306 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:17.121997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 215/424] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:07:17.124283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc7156-7821-4c53-a64a-e11011cdbecf None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:17.124618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc7156-7821-4c53-a64a-e11011cdbecf None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:17.125118 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc7156-7821-4c53-a64a-e11011cdbecf None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:17.130086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 216/425] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:17.130086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc7156-7821-4c53-a64a-e11011cdbecf None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:17.157609 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21caa5e3-e69c-429d-a98f-93e95d7df78c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:17.159906 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21caa5e3-e69c-429d-a98f-93e95d7df78c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:17.264463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:17.269643 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc7156-7821-4c53-a64a-e11011cdbecf None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:17.303106 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 210/426] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:17.306032 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a14e52-6dbe-4382-880a-b5321cd74f5b None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:17.306886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a14e52-6dbe-4382-880a-b5321cd74f5b None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:17.307706 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a14e52-6dbe-4382-880a-b5321cd74f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:17.325657 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e6474-8496-4894-854f-a15e258e7ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:17.326560 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e6474-8496-4894-854f-a15e258e7ae6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:17.326872 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e6474-8496-4894-854f-a15e258e7ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:17.327180 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e6474-8496-4894-854f-a15e258e7ae6 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:17.329087 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e6474-8496-4894-854f-a15e258e7ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:17.330591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e6474-8496-4894-854f-a15e258e7ae6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:17.333533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 217/427] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:17 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:17.334450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e6474-8496-4894-854f-a15e258e7ae6 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:17.341421 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-63a14e52-6dbe-4382-880a-b5321cd74f5b None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:17.344418 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-63a14e52-6dbe-4382-880a-b5321cd74f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:17.406563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:17.437235 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 211/428] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:17.438741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a14e52-6dbe-4382-880a-b5321cd74f5b None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:17.482158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e6474-8496-4894-854f-a15e258e7ae6 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:17.527466 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfbfc0-8c89-4e0f-b8c7-3e489c1519c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:17.528883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfbfc0-8c89-4e0f-b8c7-3e489c1519c0 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:17.529412 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfbfc0-8c89-4e0f-b8c7-3e489c1519c0 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:17.529905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfbfc0-8c89-4e0f-b8c7-3e489c1519c0 None admin] PATH_INFO: `/v3/projects/003a311d04e34ac4ab05c59a55d5a47a` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:17.531588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfbfc0-8c89-4e0f-b8c7-3e489c1519c0 None admin] RBAC: Authorizing `identity:get_project(project_id=003a311d04e34ac4ab05c59a55d5a47a)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:17.533689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfbfc0-8c89-4e0f-b8c7-3e489c1519c0 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:17.540708 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 218/429] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:17 2022] GET /identity/v3/projects/003a311d04e34ac4ab05c59a55d5a47a => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:17.541951 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfbfc0-8c89-4e0f-b8c7-3e489c1519c0 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:17.588769 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:17.596753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a14e52-6dbe-4382-880a-b5321cd74f5b None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:17.643179 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea1f06-5db0-47b8-81e0-96c14b0462e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:17.644546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1f06-5db0-47b8-81e0-96c14b0462e0 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:17.644546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1f06-5db0-47b8-81e0-96c14b0462e0 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:17.644881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea1f06-5db0-47b8-81e0-96c14b0462e0 None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:17.646319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea1f06-5db0-47b8-81e0-96c14b0462e0 None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:17.648181 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea1f06-5db0-47b8-81e0-96c14b0462e0 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:17.650710 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 212/430] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:17 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:17.652029 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea1f06-5db0-47b8-81e0-96c14b0462e0 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:17.682922 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfbfc0-8c89-4e0f-b8c7-3e489c1519c0 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:17.730853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3069917-c5c6-4953-ab88-e789ec65cb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:17.732872 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3069917-c5c6-4953-ab88-e789ec65cb46 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:17.733372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3069917-c5c6-4953-ab88-e789ec65cb46 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:17.733846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3069917-c5c6-4953-ab88-e789ec65cb46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:17.735774 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:17.751275 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3069917-c5c6-4953-ab88-e789ec65cb46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-a3069917-c5c6-4953-ab88-e789ec65cb46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:17.761286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:17.762529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:07:17.763774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:17.771066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 219/431] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:17.772219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3069917-c5c6-4953-ab88-e789ec65cb46 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:17.803687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea1f06-5db0-47b8-81e0-96c14b0462e0 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:17.848872 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9eec47-cc6b-46c2-a470-e679f7188050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:17.850145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9eec47-cc6b-46c2-a470-e679f7188050 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:17.850593 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9eec47-cc6b-46c2-a470-e679f7188050 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:17.851029 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9eec47-cc6b-46c2-a470-e679f7188050 None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:17.852715 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9eec47-cc6b-46c2-a470-e679f7188050 None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:17.854488 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9eec47-cc6b-46c2-a470-e679f7188050 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:17.856810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 213/432] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:17 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:17.857899 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9eec47-cc6b-46c2-a470-e679f7188050 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:17.901194 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:17.908074 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3069917-c5c6-4953-ab88-e789ec65cb46 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:17.955669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876154a3-0212-4d1c-83e7-f186606bd295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:17.956366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876154a3-0212-4d1c-83e7-f186606bd295 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:17.956725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876154a3-0212-4d1c-83e7-f186606bd295 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:17.957009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876154a3-0212-4d1c-83e7-f186606bd295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:17.957851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876154a3-0212-4d1c-83e7-f186606bd295 None admin] RBAC: Adding query filter params (group.id=0e122408b3fc49a282523b4d0f77ae01, role.id=ba04654a403444e4ae54a1a216888fa5, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:17.958170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876154a3-0212-4d1c-83e7-f186606bd295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:17.959653 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876154a3-0212-4d1c-83e7-f186606bd295 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:17.966501 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 220/433] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:07:17 2022] GET /identity/v3/role_assignments?group.id=0e122408b3fc49a282523b4d0f77ae01&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:17.967366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876154a3-0212-4d1c-83e7-f186606bd295 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:17.979423 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9eec47-cc6b-46c2-a470-e679f7188050 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:18.008679 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:18.028895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681be04-be40-415f-be50-f9a02667282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:18.030163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681be04-be40-415f-be50-f9a02667282e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:18.030506 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681be04-be40-415f-be50-f9a02667282e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:18.030791 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681be04-be40-415f-be50-f9a02667282e None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:18.031667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681be04-be40-415f-be50-f9a02667282e None admin] RBAC: Adding query filter params (name=member) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:18.032613 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681be04-be40-415f-be50-f9a02667282e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:18.034113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681be04-be40-415f-be50-f9a02667282e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:18.048568 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 214/434] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:07:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:18.054939 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525190b9-f9e1-4ccf-9912-476febe2c500 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:18.057058 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525190b9-f9e1-4ccf-9912-476febe2c500 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:18.057429 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525190b9-f9e1-4ccf-9912-476febe2c500 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:18.064788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 215/435] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:18.064788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525190b9-f9e1-4ccf-9912-476febe2c500 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:18.164151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876154a3-0212-4d1c-83e7-f186606bd295 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:18.214312 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6af34-f9b7-41f1-8ca5-fdd6761e6434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:18.215019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6af34-f9b7-41f1-8ca5-fdd6761e6434 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:18.215688 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6af34-f9b7-41f1-8ca5-fdd6761e6434 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:18.215688 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6af34-f9b7-41f1-8ca5-fdd6761e6434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:18.216806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6af34-f9b7-41f1-8ca5-fdd6761e6434 None admin] RBAC: Adding query filter params (group.id=0e122408b3fc49a282523b4d0f77ae01, role.id=c9fbfe89c48d458c94f2a8a11652fcf2, scope.project.id=003a311d04e34ac4ab05c59a55d5a47a) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:18.217038 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6af34-f9b7-41f1-8ca5-fdd6761e6434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:18.219079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6af34-f9b7-41f1-8ca5-fdd6761e6434 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:18.226610 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 221/436] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:07:17 2022] GET /identity/v3/role_assignments?group.id=0e122408b3fc49a282523b4d0f77ae01&scope.project.id=003a311d04e34ac4ab05c59a55d5a47a&role.id=c9fbfe89c48d458c94f2a8a11652fcf2 => generated 611 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:18.228007 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50f6f-cda2-45dd-895e-2920a9dd122e None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:18.228293 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50f6f-cda2-45dd-895e-2920a9dd122e None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:18.228633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af50f6f-cda2-45dd-895e-2920a9dd122e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:18.233373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525190b9-f9e1-4ccf-9912-476febe2c500 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:18.258944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-1af50f6f-cda2-45dd-895e-2920a9dd122e None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:18.260573 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-1af50f6f-cda2-45dd-895e-2920a9dd122e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:18.285897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:18.297033 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14607fd-8855-4c48-9644-469f0d8c264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:18.306017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14607fd-8855-4c48-9644-469f0d8c264c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:18.306017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14607fd-8855-4c48-9644-469f0d8c264c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:18.306017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14607fd-8855-4c48-9644-469f0d8c264c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:18.361710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 222/437] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:18.363537 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69674cd6-148c-4cf8-a7b9-fb8bbb598409 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:18.364145 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69674cd6-148c-4cf8-a7b9-fb8bbb598409 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:18.364678 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69674cd6-148c-4cf8-a7b9-fb8bbb598409 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:18.366916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 223/438] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:18.377308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e213de7-45bf-47aa-a554-c20403b2b34c None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:18.377308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e213de7-45bf-47aa-a554-c20403b2b34c None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:18.377308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e213de7-45bf-47aa-a554-c20403b2b34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:18.387249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:18.399483 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-8e213de7-45bf-47aa-a554-c20403b2b34c None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:18.401190 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-8e213de7-45bf-47aa-a554-c20403b2b34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:18.415138 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14607fd-8855-4c48-9644-469f0d8c264c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:18.416699 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14607fd-8855-4c48-9644-469f0d8c264c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:18.497670 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 224/439] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:18.498357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e213de7-45bf-47aa-a554-c20403b2b34c None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:18.575343 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 216/440] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:18.577020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14607fd-8855-4c48-9644-469f0d8c264c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:18.603759 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:18.622870 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e213de7-45bf-47aa-a554-c20403b2b34c None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:18.661461 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e9c07-e760-4f13-b444-0f353cad0f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:18.662801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e9c07-e760-4f13-b444-0f353cad0f10 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:18.662801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e9c07-e760-4f13-b444-0f353cad0f10 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:18.663091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e9c07-e760-4f13-b444-0f353cad0f10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:18.672644 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:18.679326 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e9c07-e760-4f13-b444-0f353cad0f10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-f78e9c07-e760-4f13-b444-0f353cad0f10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:18.686170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:18.687875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 225/441] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:07:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:18.690160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e9c07-e760-4f13-b444-0f353cad0f10 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:18.704534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14607fd-8855-4c48-9644-469f0d8c264c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:18.752540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2d088-9ffe-48fe-b571-9743681c84f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:18.753439 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2d088-9ffe-48fe-b571-9743681c84f7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:18.753725 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2d088-9ffe-48fe-b571-9743681c84f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:18.754019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2d088-9ffe-48fe-b571-9743681c84f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:18.771673 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2d088-9ffe-48fe-b571-9743681c84f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-baf2d088-9ffe-48fe-b571-9743681c84f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:18.780113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:18.782191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 217/442] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:07:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:18.783326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2d088-9ffe-48fe-b571-9743681c84f7 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:18.823735 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e9c07-e760-4f13-b444-0f353cad0f10 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:18.862600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989ec9b-15e5-4abe-8963-4f86095dd1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:18.863353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989ec9b-15e5-4abe-8963-4f86095dd1bc None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:18.863632 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989ec9b-15e5-4abe-8963-4f86095dd1bc None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:18.863906 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989ec9b-15e5-4abe-8963-4f86095dd1bc None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:18.865220 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989ec9b-15e5-4abe-8963-4f86095dd1bc None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:18.866694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989ec9b-15e5-4abe-8963-4f86095dd1bc None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:18.868492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 226/443] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:18 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:18.869291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989ec9b-15e5-4abe-8963-4f86095dd1bc None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:18.889700 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:18.895195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2d088-9ffe-48fe-b571-9743681c84f7 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:18.932644 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba75a33-1deb-48a0-8bb9-6d5cc2bdf90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:18.933313 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba75a33-1deb-48a0-8bb9-6d5cc2bdf90f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:18.933594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba75a33-1deb-48a0-8bb9-6d5cc2bdf90f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:18.933983 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba75a33-1deb-48a0-8bb9-6d5cc2bdf90f None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:18.934741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba75a33-1deb-48a0-8bb9-6d5cc2bdf90f None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:18.935050 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba75a33-1deb-48a0-8bb9-6d5cc2bdf90f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:18.937163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba75a33-1deb-48a0-8bb9-6d5cc2bdf90f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:18.945953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 218/444] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:18.946663 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba75a33-1deb-48a0-8bb9-6d5cc2bdf90f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:18.951590 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:18.977066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989ec9b-15e5-4abe-8963-4f86095dd1bc None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:19.012079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae02fc2-ac39-4885-8a81-a61b64536940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:19.012776 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae02fc2-ac39-4885-8a81-a61b64536940 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:19.013075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae02fc2-ac39-4885-8a81-a61b64536940 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:19.013357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae02fc2-ac39-4885-8a81-a61b64536940 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:19.014106 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae02fc2-ac39-4885-8a81-a61b64536940 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:19.014414 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae02fc2-ac39-4885-8a81-a61b64536940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:19.016473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae02fc2-ac39-4885-8a81-a61b64536940 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:19.024771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 227/445] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:19.025620 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae02fc2-ac39-4885-8a81-a61b64536940 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:19.055407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba75a33-1deb-48a0-8bb9-6d5cc2bdf90f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:19.093003 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2055d-e7af-43f6-868a-b64f1250414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:19.093866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2055d-e7af-43f6-868a-b64f1250414b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:19.094149 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2055d-e7af-43f6-868a-b64f1250414b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:19.094430 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2055d-e7af-43f6-868a-b64f1250414b None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:19.095622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2055d-e7af-43f6-868a-b64f1250414b None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:19.097114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2055d-e7af-43f6-868a-b64f1250414b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:19.098743 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 219/446] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:18 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:19.104403 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2055d-e7af-43f6-868a-b64f1250414b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:19.108838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:19.134235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae02fc2-ac39-4885-8a81-a61b64536940 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:19.169766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d0849-30cf-424e-b5f1-632497e4be76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:19.170570 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d0849-30cf-424e-b5f1-632497e4be76 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:19.170864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d0849-30cf-424e-b5f1-632497e4be76 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:19.171955 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d0849-30cf-424e-b5f1-632497e4be76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:19.177356 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:19.215893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2055d-e7af-43f6-868a-b64f1250414b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:19.255333 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14c6a0-26b0-48ba-a55f-0675844b9865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:19.255996 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14c6a0-26b0-48ba-a55f-0675844b9865 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:19.256277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14c6a0-26b0-48ba-a55f-0675844b9865 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:19.256603 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14c6a0-26b0-48ba-a55f-0675844b9865 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:19.260956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d0849-30cf-424e-b5f1-632497e4be76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:19.262565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d0849-30cf-424e-b5f1-632497e4be76 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:19.332697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14c6a0-26b0-48ba-a55f-0675844b9865 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:19.334460 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14c6a0-26b0-48ba-a55f-0675844b9865 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:19.349483 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:19.379352 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 228/447] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:19.380717 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d0849-30cf-424e-b5f1-632497e4be76 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:19.409454 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:19.446567 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 220/448] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:19.447440 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14c6a0-26b0-48ba-a55f-0675844b9865 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:19.502991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d0849-30cf-424e-b5f1-632497e4be76 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:19.546257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58b906-1ee7-438d-a19d-144fb323de24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:19.546925 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58b906-1ee7-438d-a19d-144fb323de24 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:19.547208 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58b906-1ee7-438d-a19d-144fb323de24 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:19.547484 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58b906-1ee7-438d-a19d-144fb323de24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:19.548315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58b906-1ee7-438d-a19d-144fb323de24 None admin] RBAC: Adding query filter params (group.id=0e122408b3fc49a282523b4d0f77ae01, role.id=ba04654a403444e4ae54a1a216888fa5, scope.project.id=8ff5a20e721548c582f2c7a214733faf) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:19.548673 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58b906-1ee7-438d-a19d-144fb323de24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:19.550121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58b906-1ee7-438d-a19d-144fb323de24 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:19.556656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 229/449] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:07:19 2022] GET /identity/v3/role_assignments?group.id=0e122408b3fc49a282523b4d0f77ae01&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=ba04654a403444e4ae54a1a216888fa5 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:19.557491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58b906-1ee7-438d-a19d-144fb323de24 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:19.558835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14c6a0-26b0-48ba-a55f-0675844b9865 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:19.608034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9378c9-9f5d-4132-a402-00c9d6779d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:19.608831 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:19.609752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9378c9-9f5d-4132-a402-00c9d6779d69 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:19.609886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9378c9-9f5d-4132-a402-00c9d6779d69 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:19.610053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9378c9-9f5d-4132-a402-00c9d6779d69 None admin] PATH_INFO: `/v3/projects/4f2a61498743436aab0f28a2590bd946` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:19.611368 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9378c9-9f5d-4132-a402-00c9d6779d69 None admin] RBAC: Authorizing `identity:get_project(project_id=4f2a61498743436aab0f28a2590bd946)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:19.612895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9378c9-9f5d-4132-a402-00c9d6779d69 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:19.614703 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 221/450] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:19 2022] GET /identity/v3/projects/4f2a61498743436aab0f28a2590bd946 => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:19.620821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9378c9-9f5d-4132-a402-00c9d6779d69 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:19.672852 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:19.682908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58b906-1ee7-438d-a19d-144fb323de24 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:19.724467 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e863c-c176-4078-9ea0-309ec50578a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:19.725377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e863c-c176-4078-9ea0-309ec50578a5 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:19.725675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e863c-c176-4078-9ea0-309ec50578a5 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:19.725938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e863c-c176-4078-9ea0-309ec50578a5 None admin] PATH_INFO: `/v3/projects/8ff5a20e721548c582f2c7a214733faf` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:19.727229 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e863c-c176-4078-9ea0-309ec50578a5 None admin] RBAC: Authorizing `identity:get_project(project_id=8ff5a20e721548c582f2c7a214733faf)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:19.728773 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e863c-c176-4078-9ea0-309ec50578a5 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:19.730562 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 230/451] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:07:19 2022] GET /identity/v3/projects/8ff5a20e721548c582f2c7a214733faf => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:19.736327 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e863c-c176-4078-9ea0-309ec50578a5 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:19.744806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9378c9-9f5d-4132-a402-00c9d6779d69 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:19.792972 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78516b37-1f4c-4d44-be49-7eb3646b967e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:19.794030 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78516b37-1f4c-4d44-be49-7eb3646b967e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:19.797654 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78516b37-1f4c-4d44-be49-7eb3646b967e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:19.797654 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78516b37-1f4c-4d44-be49-7eb3646b967e None admin] PATH_INFO: `/v3/groups/ef98e694996341f9aa5a0c64f3897a86` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:19.797654 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78516b37-1f4c-4d44-be49-7eb3646b967e None admin] RBAC: Authorizing `identity:get_group(group_id=ef98e694996341f9aa5a0c64f3897a86)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:19.798495 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78516b37-1f4c-4d44-be49-7eb3646b967e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:19.800587 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 222/452] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:19 2022] GET /identity/v3/groups/ef98e694996341f9aa5a0c64f3897a86 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:19.806705 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78516b37-1f4c-4d44-be49-7eb3646b967e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:19.862863 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e863c-c176-4078-9ea0-309ec50578a5 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:19.871583 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:19.909668 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a5479-4d16-4354-ae82-7c863c67d04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:19.910951 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a5479-4d16-4354-ae82-7c863c67d04b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:19.911234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a5479-4d16-4354-ae82-7c863c67d04b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:19.911541 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a5479-4d16-4354-ae82-7c863c67d04b None admin] PATH_INFO: `/v3/groups/0e122408b3fc49a282523b4d0f77ae01` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:19.912841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a5479-4d16-4354-ae82-7c863c67d04b None admin] RBAC: Authorizing `identity:get_group(group_id=0e122408b3fc49a282523b4d0f77ae01)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:19.914716 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18a5479-4d16-4354-ae82-7c863c67d04b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:19.916109 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 231/453] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:07:19 2022] GET /identity/v3/groups/0e122408b3fc49a282523b4d0f77ae01 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:19.922598 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a5479-4d16-4354-ae82-7c863c67d04b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:19.934402 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78516b37-1f4c-4d44-be49-7eb3646b967e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:19.945858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:19.984791 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a21a919-71e6-4809-be9a-ee6f7cbac32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:19.985358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21a919-71e6-4809-be9a-ee6f7cbac32b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:19.985822 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21a919-71e6-4809-be9a-ee6f7cbac32b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:19.986284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a21a919-71e6-4809-be9a-ee6f7cbac32b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:19.987782 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21a919-71e6-4809-be9a-ee6f7cbac32b None admin] RBAC: Adding query filter params (group.id=ef98e694996341f9aa5a0c64f3897a86, role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=4f2a61498743436aab0f28a2590bd946) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:19.988313 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21a919-71e6-4809-be9a-ee6f7cbac32b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:19.990949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a21a919-71e6-4809-be9a-ee6f7cbac32b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:19.997950 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 223/454] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:07:19 2022] GET /identity/v3/role_assignments?group.id=ef98e694996341f9aa5a0c64f3897a86&scope.project.id=4f2a61498743436aab0f28a2590bd946&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:20.054747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a5479-4d16-4354-ae82-7c863c67d04b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:20.094264 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b84cc4-e9fd-4b4a-870c-ba0e12930378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:20.094933 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b84cc4-e9fd-4b4a-870c-ba0e12930378 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:20.095216 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b84cc4-e9fd-4b4a-870c-ba0e12930378 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:20.095489 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b84cc4-e9fd-4b4a-870c-ba0e12930378 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:20.096351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b84cc4-e9fd-4b4a-870c-ba0e12930378 None admin] RBAC: Adding query filter params (group.id=0e122408b3fc49a282523b4d0f77ae01, role.id=c9fbfe89c48d458c94f2a8a11652fcf2, scope.project.id=8ff5a20e721548c582f2c7a214733faf) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:20.096671 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b84cc4-e9fd-4b4a-870c-ba0e12930378 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:20.098142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b84cc4-e9fd-4b4a-870c-ba0e12930378 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:20.105255 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 232/455] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:07:19 2022] GET /identity/v3/role_assignments?group.id=0e122408b3fc49a282523b4d0f77ae01&scope.project.id=8ff5a20e721548c582f2c7a214733faf&role.id=c9fbfe89c48d458c94f2a8a11652fcf2 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:22.922835 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d761d7-ed13-4d0d-b167-474fdb18433c None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:22.923911 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d761d7-ed13-4d0d-b167-474fdb18433c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:22.924440 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d761d7-ed13-4d0d-b167-474fdb18433c None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:22.926345 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 224/456] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:22.931823 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e19f0-a015-4c36-8a02-242f0a1396c5 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:22.932772 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e19f0-a015-4c36-8a02-242f0a1396c5 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:22.933355 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e19f0-a015-4c36-8a02-242f0a1396c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:22.957486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-812e19f0-a015-4c36-8a02-242f0a1396c5 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:22.959049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-812e19f0-a015-4c36-8a02-242f0a1396c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:22.997848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:23.050086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 233/457] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:23.125337 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d761d7-ed13-4d0d-b167-474fdb18433c None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:23.282886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d761d7-ed13-4d0d-b167-474fdb18433c None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:23.319503 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d891974-d3be-4d99-8fa7-6b63b5d3b59c None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:23.320149 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d891974-d3be-4d99-8fa7-6b63b5d3b59c None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:23.320702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d891974-d3be-4d99-8fa7-6b63b5d3b59c None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:23.322625 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 234/458] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:23.328124 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df34ff-7514-4654-9543-4cd35bfc5d73 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:23.328796 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df34ff-7514-4654-9543-4cd35bfc5d73 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:23.329309 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df34ff-7514-4654-9543-4cd35bfc5d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:23.338227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f52a9-56d3-4c3f-8fcf-b00060943958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:23.339423 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f52a9-56d3-4c3f-8fcf-b00060943958 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:23.339934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f52a9-56d3-4c3f-8fcf-b00060943958 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:23.340461 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f52a9-56d3-4c3f-8fcf-b00060943958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:23.361047 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-70df34ff-7514-4654-9543-4cd35bfc5d73 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:23.362849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-70df34ff-7514-4654-9543-4cd35bfc5d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:23.363880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f52a9-56d3-4c3f-8fcf-b00060943958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:23.365647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f52a9-56d3-4c3f-8fcf-b00060943958 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-e36f52a9-56d3-4c3f-8fcf-b00060943958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:23.389226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:23.390377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:23.391396 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:23.391396 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:23.391396 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:23.391396 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:23.393756 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 225/459] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:23.399838 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c010cc-3924-40a9-af7d-76246b05d9b6 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:23.400660 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c010cc-3924-40a9-af7d-76246b05d9b6 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:23.401170 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c010cc-3924-40a9-af7d-76246b05d9b6 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:23.402847 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 226/460] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:23.403833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c010cc-3924-40a9-af7d-76246b05d9b6 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:23.433500 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:23.464617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 235/461] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:23.466130 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ae405-95cd-4bab-b5a6-4354fe03ace9 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:23.466423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ae405-95cd-4bab-b5a6-4354fe03ace9 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:23.466706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ae405-95cd-4bab-b5a6-4354fe03ace9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:23.489051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-192ae405-95cd-4bab-b5a6-4354fe03ace9 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:23.490456 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-192ae405-95cd-4bab-b5a6-4354fe03ace9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:23.529219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c010cc-3924-40a9-af7d-76246b05d9b6 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:23.572419 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f9430-441a-41ef-879f-e8ddba872761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:23.573187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f9430-441a-41ef-879f-e8ddba872761 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:23.573578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f9430-441a-41ef-879f-e8ddba872761 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:23.574150 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f9430-441a-41ef-879f-e8ddba872761 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:23.575125 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f9430-441a-41ef-879f-e8ddba872761 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:23.575598 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f9430-441a-41ef-879f-e8ddba872761 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:23.577974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f9430-441a-41ef-879f-e8ddba872761 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:23.580375 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 236/462] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:23.581790 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ae405-95cd-4bab-b5a6-4354fe03ace9 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:23.607360 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 227/463] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:23.609821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f9430-441a-41ef-879f-e8ddba872761 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:23.620972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:23.732845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:23.735884 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ae405-95cd-4bab-b5a6-4354fe03ace9 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:23.752092 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f9430-441a-41ef-879f-e8ddba872761 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:23.794307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3d153-ace3-45bc-8392-7b3a7e13e291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:23.795127 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3d153-ace3-45bc-8392-7b3a7e13e291 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:23.795414 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3d153-ace3-45bc-8392-7b3a7e13e291 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:23.795694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3d153-ace3-45bc-8392-7b3a7e13e291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:23.801145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88d3bd-1df0-4e8f-ad82-ce64dc105ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:23.802277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88d3bd-1df0-4e8f-ad82-ce64dc105ac6 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:23.802793 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88d3bd-1df0-4e8f-ad82-ce64dc105ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:23.803300 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88d3bd-1df0-4e8f-ad82-ce64dc105ac6 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:23.804987 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88d3bd-1df0-4e8f-ad82-ce64dc105ac6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:23.807660 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b88d3bd-1df0-4e8f-ad82-ce64dc105ac6 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:23.808051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3d153-ace3-45bc-8392-7b3a7e13e291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:23.809583 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3d153-ace3-45bc-8392-7b3a7e13e291 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-99a3d153-ace3-45bc-8392-7b3a7e13e291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:23.836630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:23.837771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:23.839086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:23.839086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:23.839086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:23.839086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:23.839086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 237/464] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:23.839086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3d153-ace3-45bc-8392-7b3a7e13e291 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:23.928649 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 228/465] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:07:23 2022] POST /identity/v3/users => generated 246 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:07:23.930357 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88d3bd-1df0-4e8f-ad82-ce64dc105ac6 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:23.958117 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:23.988269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3d153-ace3-45bc-8392-7b3a7e13e291 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:24.035593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f7210-3e08-4f70-9274-5d1064e139bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:24.036495 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7210-3e08-4f70-9274-5d1064e139bd None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:24.036945 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7210-3e08-4f70-9274-5d1064e139bd None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:24.037229 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7210-3e08-4f70-9274-5d1064e139bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:24.050538 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f7210-3e08-4f70-9274-5d1064e139bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:24.052190 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f7210-3e08-4f70-9274-5d1064e139bd None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-506f7210-3e08-4f70-9274-5d1064e139bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:24.073981 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:24.075060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:24.076080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:24.076080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:24.076080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:24.076080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:24.076329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 238/466] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:24.080735 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd03df-ed9b-43d5-b464-cfb11aa08a67 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:24.081237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd03df-ed9b-43d5-b464-cfb11aa08a67 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:24.081789 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd03df-ed9b-43d5-b464-cfb11aa08a67 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:24.087588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 239/467] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:24.087588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd03df-ed9b-43d5-b464-cfb11aa08a67 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:24.130381 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88d3bd-1df0-4e8f-ad82-ce64dc105ac6 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:24.178323 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f1b18-5b7f-4f9e-a103-b6c7336d140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:24.178998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f1b18-5b7f-4f9e-a103-b6c7336d140f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:24.179334 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f1b18-5b7f-4f9e-a103-b6c7336d140f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:24.179622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f1b18-5b7f-4f9e-a103-b6c7336d140f None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:24.180434 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f1b18-5b7f-4f9e-a103-b6c7336d140f None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:24.181529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f1b18-5b7f-4f9e-a103-b6c7336d140f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:24.182550 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f1b18-5b7f-4f9e-a103-b6c7336d140f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:24.193262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:24.197125 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 229/468] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:24.198504 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132999f-214f-4ff7-8aad-d2ddafa5459c None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:24.198802 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132999f-214f-4ff7-8aad-d2ddafa5459c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:24.199086 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132999f-214f-4ff7-8aad-d2ddafa5459c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:24.220483 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-5132999f-214f-4ff7-8aad-d2ddafa5459c None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:24.221773 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-5132999f-214f-4ff7-8aad-d2ddafa5459c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:24.231115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd03df-ed9b-43d5-b464-cfb11aa08a67 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:24.266260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:24.271940 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20197a-642a-4382-a51f-de52691c3dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:24.273284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20197a-642a-4382-a51f-de52691c3dbe None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:24.273284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20197a-642a-4382-a51f-de52691c3dbe None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:24.273284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20197a-642a-4382-a51f-de52691c3dbe None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:24.273919 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20197a-642a-4382-a51f-de52691c3dbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:24.274285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20197a-642a-4382-a51f-de52691c3dbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:24.275687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20197a-642a-4382-a51f-de52691c3dbe None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:24.291032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 240/469] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:24.291883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20197a-642a-4382-a51f-de52691c3dbe None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:24.316514 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 230/470] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:24.317376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132999f-214f-4ff7-8aad-d2ddafa5459c None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:24.423606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20197a-642a-4382-a51f-de52691c3dbe None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:24.442106 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132999f-214f-4ff7-8aad-d2ddafa5459c None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:24.462963 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803cccc-abc0-4089-a155-7dcd5d2ee67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:24.463644 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803cccc-abc0-4089-a155-7dcd5d2ee67b None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:24.463928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803cccc-abc0-4089-a155-7dcd5d2ee67b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:24.464306 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803cccc-abc0-4089-a155-7dcd5d2ee67b None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:24.465055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803cccc-abc0-4089-a155-7dcd5d2ee67b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:24.466519 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803cccc-abc0-4089-a155-7dcd5d2ee67b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:24.473478 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:24.480167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35373a-69d6-4ec6-be6f-913af0192a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:24.481010 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35373a-69d6-4ec6-be6f-913af0192a93 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:24.481363 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35373a-69d6-4ec6-be6f-913af0192a93 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:24.481778 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35373a-69d6-4ec6-be6f-913af0192a93 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:24.482596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35373a-69d6-4ec6-be6f-913af0192a93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:24.484229 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35373a-69d6-4ec6-be6f-913af0192a93 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:24.556497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 241/471] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:07:24 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:07:24.556497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803cccc-abc0-4089-a155-7dcd5d2ee67b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:24.589705 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 231/472] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:07:24 2022] POST /identity/v3/users => generated 248 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:07:24.683945 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:24.768709 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803cccc-abc0-4089-a155-7dcd5d2ee67b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:24.784682 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009abee-143e-4d16-bf95-85baeed8c504 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:24.785232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009abee-143e-4d16-bf95-85baeed8c504 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:24.785711 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009abee-143e-4d16-bf95-85baeed8c504 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:24.788781 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 232/473] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:24.809650 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3a2aa-e313-48b8-a455-6b7cbc734a7b None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:24.810183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3a2aa-e313-48b8-a455-6b7cbc734a7b None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:24.810714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3a2aa-e313-48b8-a455-6b7cbc734a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:24.830626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b451-549e-4942-8994-d1aee4a84aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:24.845390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b451-549e-4942-8994-d1aee4a84aba None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:24.845390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b451-549e-4942-8994-d1aee4a84aba None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:24.845390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b451-549e-4942-8994-d1aee4a84aba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:24.857444 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8b451-549e-4942-8994-d1aee4a84aba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:24.861887 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8b451-549e-4942-8994-d1aee4a84aba None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:24.881997 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-66a3a2aa-e313-48b8-a455-6b7cbc734a7b None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:24.883890 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-66a3a2aa-e313-48b8-a455-6b7cbc734a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-c1c8b451-549e-4942-8994-d1aee4a84aba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:24.896892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:24.898037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:24.899113 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:24.899113 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:24.899113 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:24.899113 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:24.899594 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 242/474] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:24.908887 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b451-549e-4942-8994-d1aee4a84aba None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:25.002102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 233/475] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:25.028604 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3a2aa-e313-48b8-a455-6b7cbc734a7b None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:25.042774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:25.048795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b451-549e-4942-8994-d1aee4a84aba None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:25.094171 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae7cfb-c97d-4b13-87c7-3cb2b9d0348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:25.094870 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7cfb-c97d-4b13-87c7-3cb2b9d0348f None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:25.095158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7cfb-c97d-4b13-87c7-3cb2b9d0348f None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:25.095560 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae7cfb-c97d-4b13-87c7-3cb2b9d0348f None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:25.096188 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae7cfb-c97d-4b13-87c7-3cb2b9d0348f None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:25.096534 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae7cfb-c97d-4b13-87c7-3cb2b9d0348f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:25.097964 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae7cfb-c97d-4b13-87c7-3cb2b9d0348f None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:25.116384 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 243/476] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:25.119198 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae7cfb-c97d-4b13-87c7-3cb2b9d0348f None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:25.175142 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3a2aa-e313-48b8-a455-6b7cbc734a7b None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:25.198094 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:25.243264 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae0667-db82-459f-963b-3ad2c2aa2cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:25.244469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae0667-db82-459f-963b-3ad2c2aa2cc5 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:25.244986 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae0667-db82-459f-963b-3ad2c2aa2cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:25.245476 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae0667-db82-459f-963b-3ad2c2aa2cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:25.246486 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae0667-db82-459f-963b-3ad2c2aa2cc5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:25.248117 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae0667-db82-459f-963b-3ad2c2aa2cc5 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:25.274356 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae7cfb-c97d-4b13-87c7-3cb2b9d0348f None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:25.291086 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.sql.core [None req-42ae0667-db82-459f-963b-3ad2c2aa2cc5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:07:25.291086 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:07:25.291086 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [parameters: {'id': 'ffc773422016437ead069d165a3dd615', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:07:25.291086 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=144624) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-42ae0667-db82-459f-963b-3ad2c2aa2cc5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application conn.query(q) May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application result.read() May 19 20:07:25.317949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return ref.to_dict() May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:07:25.319100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application next(self.gen) May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application yield resource May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application next(self.gen) May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise self.value May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application session.commit() May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:07:25.320286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self.session.flush() May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application compat.raise_( May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:07:25.321358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:07:25.322383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application conn.query(q) May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application result.read() May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:07:25.323341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [parameters: {'id': 'ffc773422016437ead069d165a3dd615', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:25.324276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:07:25.327919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:25.330721 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 234/477] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:07:25 2022] POST /identity/v3/roles => generated 144 bytes in 291 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:25.330721 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae0667-db82-459f-963b-3ad2c2aa2cc5 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:25.337262 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac72eb4-752f-425f-8491-85fd5c91dd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:25.338779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac72eb4-752f-425f-8491-85fd5c91dd85 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:25.345443 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac72eb4-752f-425f-8491-85fd5c91dd85 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:25.346047 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac72eb4-752f-425f-8491-85fd5c91dd85 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:25.347133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac72eb4-752f-425f-8491-85fd5c91dd85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:25.349169 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac72eb4-752f-425f-8491-85fd5c91dd85 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:25.462383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae0667-db82-459f-963b-3ad2c2aa2cc5 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:25.473090 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 244/478] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:07:25 2022] POST /identity/v3/users => generated 249 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:07:25.505529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93745b7-653f-4d4d-8757-65a6e31e73fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:25.506901 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93745b7-653f-4d4d-8757-65a6e31e73fe None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:25.507431 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93745b7-653f-4d4d-8757-65a6e31e73fe None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:25.507924 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93745b7-653f-4d4d-8757-65a6e31e73fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:25.529425 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93745b7-653f-4d4d-8757-65a6e31e73fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-d93745b7-653f-4d4d-8757-65a6e31e73fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:25.537819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:25.539053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:25.540023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 235/479] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:07:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:25.544859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac72eb4-752f-425f-8491-85fd5c91dd85 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:25.579621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:25.670965 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac72eb4-752f-425f-8491-85fd5c91dd85 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:25.711352 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b03a10e-ac8c-4588-a7ea-c5ca6118c0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:25.712051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03a10e-ac8c-4588-a7ea-c5ca6118c0aa None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:25.712353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03a10e-ac8c-4588-a7ea-c5ca6118c0aa None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:25.712645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b03a10e-ac8c-4588-a7ea-c5ca6118c0aa None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:25.713444 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03a10e-ac8c-4588-a7ea-c5ca6118c0aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:25.713757 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03a10e-ac8c-4588-a7ea-c5ca6118c0aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:25.715227 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b03a10e-ac8c-4588-a7ea-c5ca6118c0aa None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:25.725959 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 245/480] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:26.778277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589bdf2-a22e-4c8a-8f9a-985954ecc875 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:26.779369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589bdf2-a22e-4c8a-8f9a-985954ecc875 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:26.779888 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589bdf2-a22e-4c8a-8f9a-985954ecc875 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:26.782532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 236/481] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:26.788098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab4567-efe3-4154-9c35-efc1b7b0996d None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:26.793423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab4567-efe3-4154-9c35-efc1b7b0996d None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:26.793423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab4567-efe3-4154-9c35-efc1b7b0996d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:26.811436 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-1eab4567-efe3-4154-9c35-efc1b7b0996d None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:26.813072 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-1eab4567-efe3-4154-9c35-efc1b7b0996d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:26.898408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 246/482] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:26.947200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589bdf2-a22e-4c8a-8f9a-985954ecc875 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:27.007001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:27.112178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589bdf2-a22e-4c8a-8f9a-985954ecc875 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:27.173091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fd69c-f5ac-4bb2-afc7-5701b7779ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:27.174482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fd69c-f5ac-4bb2-afc7-5701b7779ea4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:27.174990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fd69c-f5ac-4bb2-afc7-5701b7779ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:27.175487 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fd69c-f5ac-4bb2-afc7-5701b7779ea4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:27.194660 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fd69c-f5ac-4bb2-afc7-5701b7779ea4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-2c7fd69c-f5ac-4bb2-afc7-5701b7779ea4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:27.202974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:27.204219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:27.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:27.208502 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 237/483] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:27.211200 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab4567-efe3-4154-9c35-efc1b7b0996d None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:27.236799 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:27.319497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab4567-efe3-4154-9c35-efc1b7b0996d None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:27.357409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fe907-510a-4262-afd6-3419f86e5a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:27.358084 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fe907-510a-4262-afd6-3419f86e5a77 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:27.358428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fe907-510a-4262-afd6-3419f86e5a77 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:27.359262 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fe907-510a-4262-afd6-3419f86e5a77 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:27.360103 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fe907-510a-4262-afd6-3419f86e5a77 None admin] RBAC: Adding query filter params (name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:27.360451 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fe907-510a-4262-afd6-3419f86e5a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:27.361933 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fe907-510a-4262-afd6-3419f86e5a77 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:27.376201 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 247/484] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:27.377620 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fd69c-f5ac-4bb2-afc7-5701b7779ea4 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:27.508075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:27.515952 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fd69c-f5ac-4bb2-afc7-5701b7779ea4 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:27.566751 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477aabd-5e73-4a35-a6ba-ba9a31c1cee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:27.567626 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477aabd-5e73-4a35-a6ba-ba9a31c1cee2 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:27.567918 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477aabd-5e73-4a35-a6ba-ba9a31c1cee2 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:27.568198 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477aabd-5e73-4a35-a6ba-ba9a31c1cee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:27.586175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477aabd-5e73-4a35-a6ba-ba9a31c1cee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:27.587723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477aabd-5e73-4a35-a6ba-ba9a31c1cee2 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-c477aabd-5e73-4a35-a6ba-ba9a31c1cee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:27.613695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:27.615964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:27.618260 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:27.618260 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:27.618260 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:27.618260 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:27.620394 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 238/485] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:27.621973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fe907-510a-4262-afd6-3419f86e5a77 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:27.648366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2395425-2b97-45d5-9295-b5c3bd64bde2 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:27.648921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2395425-2b97-45d5-9295-b5c3bd64bde2 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:27.649382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2395425-2b97-45d5-9295-b5c3bd64bde2 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:27.651027 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 239/486] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:07:27.657587 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2714ab-bad5-44fb-b041-55be3423b346 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:27.658227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2714ab-bad5-44fb-b041-55be3423b346 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:27.658687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2714ab-bad5-44fb-b041-55be3423b346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:27.689158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-7d2714ab-bad5-44fb-b041-55be3423b346 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:27.690663 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-7d2714ab-bad5-44fb-b041-55be3423b346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:27.732820 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:27.769032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fe907-510a-4262-afd6-3419f86e5a77 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:27.809391 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d939f-6819-46a4-ab0e-1a14c727e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:27.811100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d939f-6819-46a4-ab0e-1a14c727e813 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:27.812210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d939f-6819-46a4-ab0e-1a14c727e813 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:27.812776 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d939f-6819-46a4-ab0e-1a14c727e813 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:27.813716 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d939f-6819-46a4-ab0e-1a14c727e813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:27.814201 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d939f-6819-46a4-ab0e-1a14c727e813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:27.815786 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d939f-6819-46a4-ab0e-1a14c727e813 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:27.835813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 248/487] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:27.837429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d939f-6819-46a4-ab0e-1a14c727e813 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:27.876877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 240/488] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:07:27.876877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc19aa9-866f-49b0-ba70-4c6c5ec617c8 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:27.876877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc19aa9-866f-49b0-ba70-4c6c5ec617c8 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:27.876877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc19aa9-866f-49b0-ba70-4c6c5ec617c8 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:27.876877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 241/489] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:27.881624 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9015cf-6469-4a5c-9dff-6392286fc3fd None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:27.881967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9015cf-6469-4a5c-9dff-6392286fc3fd None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:27.882255 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9015cf-6469-4a5c-9dff-6392286fc3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:27.900992 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-be9015cf-6469-4a5c-9dff-6392286fc3fd None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:27.902204 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-be9015cf-6469-4a5c-9dff-6392286fc3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:27.975214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d939f-6819-46a4-ab0e-1a14c727e813 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:28.000253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 242/490] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:28.002064 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9015cf-6469-4a5c-9dff-6392286fc3fd None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:28.018800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:28.034729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:28.051254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af45a1-ff3d-4725-bd25-20adbf549b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:28.052076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af45a1-ff3d-4725-bd25-20adbf549b1a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:28.052372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af45a1-ff3d-4725-bd25-20adbf549b1a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:28.052651 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af45a1-ff3d-4725-bd25-20adbf549b1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:28.060932 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af45a1-ff3d-4725-bd25-20adbf549b1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:28.062594 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af45a1-ff3d-4725-bd25-20adbf549b1a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-39af45a1-ff3d-4725-bd25-20adbf549b1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:28.086669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:28.088022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 249/491] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:07:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:28.094067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af45a1-ff3d-4725-bd25-20adbf549b1a None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:28.146786 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9015cf-6469-4a5c-9dff-6392286fc3fd None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:28.196292 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b777de4-8f21-4f47-adbc-657c099e845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:28.197933 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b777de4-8f21-4f47-adbc-657c099e845d None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:28.198423 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b777de4-8f21-4f47-adbc-657c099e845d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:28.198897 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b777de4-8f21-4f47-adbc-657c099e845d None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:28.214853 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b777de4-8f21-4f47-adbc-657c099e845d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:28.216614 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af45a1-ff3d-4725-bd25-20adbf549b1a None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-1b777de4-8f21-4f47-adbc-657c099e845d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:28.225338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:28.226539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 243/492] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:28.229473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b777de4-8f21-4f47-adbc-657c099e845d None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:28.290053 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cb4e5-bb75-455d-a0df-9f2399339227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:28.291117 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cb4e5-bb75-455d-a0df-9f2399339227 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:28.291389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cb4e5-bb75-455d-a0df-9f2399339227 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:28.291764 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534cb4e5-bb75-455d-a0df-9f2399339227 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:28.304648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534cb4e5-bb75-455d-a0df-9f2399339227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-534cb4e5-bb75-455d-a0df-9f2399339227 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:28.313329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:28.314545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:28.315554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:28.320291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 250/493] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:28.321536 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cb4e5-bb75-455d-a0df-9f2399339227 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:28.325259 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:28.369987 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b777de4-8f21-4f47-adbc-657c099e845d None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:28.407803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:28.421670 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e25c02-4960-4852-8276-586338cba559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:28.422950 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e25c02-4960-4852-8276-586338cba559 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:28.423541 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e25c02-4960-4852-8276-586338cba559 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:28.424748 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e25c02-4960-4852-8276-586338cba559 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:28.426105 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e25c02-4960-4852-8276-586338cba559 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:28.426957 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e25c02-4960-4852-8276-586338cba559 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:28.429768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e25c02-4960-4852-8276-586338cba559 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:28.466676 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534cb4e5-bb75-455d-a0df-9f2399339227 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:28.485367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 244/494] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:07:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:28.486420 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e25c02-4960-4852-8276-586338cba559 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:28.516379 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394735f2-a7b0-433f-b2b9-248377f11602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:28.517095 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394735f2-a7b0-433f-b2b9-248377f11602 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:28.517386 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394735f2-a7b0-433f-b2b9-248377f11602 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:28.517907 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394735f2-a7b0-433f-b2b9-248377f11602 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:28.518425 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394735f2-a7b0-433f-b2b9-248377f11602 None admin] RBAC: Adding query filter params (name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:28.518953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394735f2-a7b0-433f-b2b9-248377f11602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:28.520151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394735f2-a7b0-433f-b2b9-248377f11602 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:28.531952 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 251/495] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:28.533276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394735f2-a7b0-433f-b2b9-248377f11602 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:28.610234 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e25c02-4960-4852-8276-586338cba559 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:28.644801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:28.651535 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af42710-e819-4c51-8d3f-dbf3bcf2871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:28.652209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af42710-e819-4c51-8d3f-dbf3bcf2871e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:28.652617 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af42710-e819-4c51-8d3f-dbf3bcf2871e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:28.652816 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af42710-e819-4c51-8d3f-dbf3bcf2871e None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:28.653625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af42710-e819-4c51-8d3f-dbf3bcf2871e None admin] RBAC: Adding query filter params (name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:28.654064 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af42710-e819-4c51-8d3f-dbf3bcf2871e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:28.655406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af42710-e819-4c51-8d3f-dbf3bcf2871e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:28.657636 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394735f2-a7b0-433f-b2b9-248377f11602 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:28.667110 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 245/496] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:28.668273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c361-2bb7-4c07-b834-e54d8c72b064 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:28.668598 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c361-2bb7-4c07-b834-e54d8c72b064 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:28.668922 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7c361-2bb7-4c07-b834-e54d8c72b064 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:28.670253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 246/497] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:28.672171 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7c361-2bb7-4c07-b834-e54d8c72b064 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:28.675546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:28.699602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524137f3-5913-489d-909c-c7571ec01390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:28.700256 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524137f3-5913-489d-909c-c7571ec01390 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:28.700675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524137f3-5913-489d-909c-c7571ec01390 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:28.700956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524137f3-5913-489d-909c-c7571ec01390 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:28.781657 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524137f3-5913-489d-909c-c7571ec01390 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:28.783204 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524137f3-5913-489d-909c-c7571ec01390 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:28.790069 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7c361-2bb7-4c07-b834-e54d8c72b064 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:28.830030 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114149f4-56c9-4d95-b349-81755114c02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:28.830851 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114149f4-56c9-4d95-b349-81755114c02d None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:28.831142 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114149f4-56c9-4d95-b349-81755114c02d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:28.831422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114149f4-56c9-4d95-b349-81755114c02d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:28.843200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114149f4-56c9-4d95-b349-81755114c02d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:28.844802 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114149f4-56c9-4d95-b349-81755114c02d None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-114149f4-56c9-4d95-b349-81755114c02d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:28.863288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:28.864482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:28.865532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:28.865532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:28.865532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:28.865532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:28.865532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 247/498] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:28.866006 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc7b0c-3123-4aa3-9faa-143c787940ff None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:28.866297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc7b0c-3123-4aa3-9faa-143c787940ff None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:28.866576 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc7b0c-3123-4aa3-9faa-143c787940ff None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:28.867948 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 248/499] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:28.868752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc7b0c-3123-4aa3-9faa-143c787940ff None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:28.898788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 252/500] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:28.900238 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c6918-2775-4371-972a-9b0b6d29ea5f None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:28.900576 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c6918-2775-4371-972a-9b0b6d29ea5f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:28.900844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c6918-2775-4371-972a-9b0b6d29ea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:28.919228 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-a24c6918-2775-4371-972a-9b0b6d29ea5f None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:28.920363 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-a24c6918-2775-4371-972a-9b0b6d29ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:28.920992 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:28.928493 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:28.978044 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc7b0c-3123-4aa3-9faa-143c787940ff None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:28.990726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 253/501] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:28.993280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c6918-2775-4371-972a-9b0b6d29ea5f None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:29.018127 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0c15e-e971-467b-acca-a009a5521d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:29.018932 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c15e-e971-467b-acca-a009a5521d50 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:29.019214 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c15e-e971-467b-acca-a009a5521d50 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:29.019489 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0c15e-e971-467b-acca-a009a5521d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:29.031069 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0c15e-e971-467b-acca-a009a5521d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:29.032583 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0c15e-e971-467b-acca-a009a5521d50 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-91c0c15e-e971-467b-acca-a009a5521d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:29.050883 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:29.052269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:29.053534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:29.053534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:29.053534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:29.053534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:29.053534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 249/502] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:29.053920 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702cb80-abda-44ad-af8f-19566ffcec6f None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:29.054017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702cb80-abda-44ad-af8f-19566ffcec6f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:29.054242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702cb80-abda-44ad-af8f-19566ffcec6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:29.072862 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-2702cb80-abda-44ad-af8f-19566ffcec6f None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:29.074103 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-2702cb80-abda-44ad-af8f-19566ffcec6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:29.103735 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c6918-2775-4371-972a-9b0b6d29ea5f None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:29.139974 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f78227-ef1e-4928-8251-b02d51d6c650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:29.140710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f78227-ef1e-4928-8251-b02d51d6c650 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:29.140999 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f78227-ef1e-4928-8251-b02d51d6c650 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:29.141279 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f78227-ef1e-4928-8251-b02d51d6c650 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:29.142047 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f78227-ef1e-4928-8251-b02d51d6c650 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:29.142352 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f78227-ef1e-4928-8251-b02d51d6c650 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:29.143748 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f78227-ef1e-4928-8251-b02d51d6c650 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:29.144502 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 250/503] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:29.145208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702cb80-abda-44ad-af8f-19566ffcec6f None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:29.155722 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 254/504] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:29.156652 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f78227-ef1e-4928-8251-b02d51d6c650 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:29.159496 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:29.191071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:29.263858 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f78227-ef1e-4928-8251-b02d51d6c650 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:29.264455 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702cb80-abda-44ad-af8f-19566ffcec6f None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:29.299161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee93982-81f3-44aa-a862-7a98e5e8e5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:29.299448 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daef569-fc3e-4d77-90ed-b1b59c1599de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:29.299957 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee93982-81f3-44aa-a862-7a98e5e8e5ed None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:29.300111 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daef569-fc3e-4d77-90ed-b1b59c1599de None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:29.300224 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee93982-81f3-44aa-a862-7a98e5e8e5ed None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:29.300413 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daef569-fc3e-4d77-90ed-b1b59c1599de None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:29.300529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee93982-81f3-44aa-a862-7a98e5e8e5ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:29.300690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daef569-fc3e-4d77-90ed-b1b59c1599de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:29.311327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daef569-fc3e-4d77-90ed-b1b59c1599de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:29.311633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee93982-81f3-44aa-a862-7a98e5e8e5ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:29.313425 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daef569-fc3e-4d77-90ed-b1b59c1599de None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-9ee93982-81f3-44aa-a862-7a98e5e8e5ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:29.319833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:29.321050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 255/505] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:29.322098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee93982-81f3-44aa-a862-7a98e5e8e5ed None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-6daef569-fc3e-4d77-90ed-b1b59c1599de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:29.337878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:29.339001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:29.340083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:29.340083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:29.340083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:29.340083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:29.340083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 251/506] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:29.340456 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daef569-fc3e-4d77-90ed-b1b59c1599de None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:29.393239 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:29.423496 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:29.433490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee93982-81f3-44aa-a862-7a98e5e8e5ed None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:29.446471 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daef569-fc3e-4d77-90ed-b1b59c1599de None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:29.473252 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461d49d-87a6-4fe2-aa7c-41a58d5bd936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:29.474348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461d49d-87a6-4fe2-aa7c-41a58d5bd936 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:29.474844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461d49d-87a6-4fe2-aa7c-41a58d5bd936 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:29.475420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461d49d-87a6-4fe2-aa7c-41a58d5bd936 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:29.480867 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461d49d-87a6-4fe2-aa7c-41a58d5bd936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:29.481193 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461d49d-87a6-4fe2-aa7c-41a58d5bd936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:29.482628 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461d49d-87a6-4fe2-aa7c-41a58d5bd936 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:29.485759 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30683445-7b80-427b-8abf-73008ced8b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:29.486573 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30683445-7b80-427b-8abf-73008ced8b5e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:29.486859 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30683445-7b80-427b-8abf-73008ced8b5e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:29.487145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30683445-7b80-427b-8abf-73008ced8b5e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:29.495074 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30683445-7b80-427b-8abf-73008ced8b5e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:29.496619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30683445-7b80-427b-8abf-73008ced8b5e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:29.506062 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 256/507] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:29.506949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461d49d-87a6-4fe2-aa7c-41a58d5bd936 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-30683445-7b80-427b-8abf-73008ced8b5e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:29.515546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:29.517009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:29.518124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:29.518124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:29.518124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:29.518124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:29.518124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:29.518124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:29.518124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:29.518124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:29.518124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:07:29.518124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:29.520683 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 252/508] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:29.521402 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30683445-7b80-427b-8abf-73008ced8b5e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:29.631844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461d49d-87a6-4fe2-aa7c-41a58d5bd936 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:29.634476 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30683445-7b80-427b-8abf-73008ced8b5e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:29.656409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:29.671370 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde3a4b-d0e1-422c-b541-646b11418ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:29.675794 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde3a4b-d0e1-422c-b541-646b11418ac0 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:29.675794 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde3a4b-d0e1-422c-b541-646b11418ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:29.675794 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde3a4b-d0e1-422c-b541-646b11418ac0 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:29.676105 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c521b-aa23-4b34-bfc8-b4938661a43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:29.676223 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde3a4b-d0e1-422c-b541-646b11418ac0 None admin] RBAC: Adding query filter params (name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:29.676316 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c521b-aa23-4b34-bfc8-b4938661a43b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:29.676447 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde3a4b-d0e1-422c-b541-646b11418ac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:29.676550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c521b-aa23-4b34-bfc8-b4938661a43b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:29.676550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c521b-aa23-4b34-bfc8-b4938661a43b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:29.676724 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde3a4b-d0e1-422c-b541-646b11418ac0 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:29.681120 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:29.684461 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 253/509] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:29.685371 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde3a4b-d0e1-422c-b541-646b11418ac0 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:29.686000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c521b-aa23-4b34-bfc8-b4938661a43b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:29.688086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c521b-aa23-4b34-bfc8-b4938661a43b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-aa2c521b-aa23-4b34-bfc8-b4938661a43b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:29.708400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:29.709577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:29.710638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:29.710638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:29.710638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:29.710638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:29.710638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 257/510] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:29.710638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c521b-aa23-4b34-bfc8-b4938661a43b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:29.798535 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde3a4b-d0e1-422c-b541-646b11418ac0 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:29.821885 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c521b-aa23-4b34-bfc8-b4938661a43b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:29.834763 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b355484-178c-43a4-99fe-e77580e42b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:29.835430 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b355484-178c-43a4-99fe-e77580e42b18 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:29.835721 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b355484-178c-43a4-99fe-e77580e42b18 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:29.835995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b355484-178c-43a4-99fe-e77580e42b18 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:29.836748 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b355484-178c-43a4-99fe-e77580e42b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:29.837058 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b355484-178c-43a4-99fe-e77580e42b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:29.838441 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b355484-178c-43a4-99fe-e77580e42b18 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:29.852772 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 254/511] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:29.853976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b355484-178c-43a4-99fe-e77580e42b18 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:29.868470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5be199-d12b-4729-aac2-6839d2df63b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:29.869248 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5be199-d12b-4729-aac2-6839d2df63b0 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:29.869551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5be199-d12b-4729-aac2-6839d2df63b0 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:29.869827 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5be199-d12b-4729-aac2-6839d2df63b0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:29.878111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5be199-d12b-4729-aac2-6839d2df63b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:29.879587 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5be199-d12b-4729-aac2-6839d2df63b0 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-ad5be199-d12b-4729-aac2-6839d2df63b0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:29.894510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:29.895930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 258/512] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:29.897280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5be199-d12b-4729-aac2-6839d2df63b0 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:29.929240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:29.946449 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:29.972539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b355484-178c-43a4-99fe-e77580e42b18 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:30.011978 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa94cc5-5db3-49b2-9c1f-382376379187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:30.012956 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa94cc5-5db3-49b2-9c1f-382376379187 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:30.013243 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa94cc5-5db3-49b2-9c1f-382376379187 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:30.013525 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa94cc5-5db3-49b2-9c1f-382376379187 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:30.014382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa94cc5-5db3-49b2-9c1f-382376379187 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:30.014717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa94cc5-5db3-49b2-9c1f-382376379187 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:30.016596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa94cc5-5db3-49b2-9c1f-382376379187 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:30.021524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5be199-d12b-4729-aac2-6839d2df63b0 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:30.034976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 255/513] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:07:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:30.036265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa94cc5-5db3-49b2-9c1f-382376379187 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:30.066246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49e2c6-7d3b-49bd-a5b8-90fe3c8e8d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:30.067065 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49e2c6-7d3b-49bd-a5b8-90fe3c8e8d5e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:30.067370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49e2c6-7d3b-49bd-a5b8-90fe3c8e8d5e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:30.067713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49e2c6-7d3b-49bd-a5b8-90fe3c8e8d5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:30.087168 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49e2c6-7d3b-49bd-a5b8-90fe3c8e8d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:30.088759 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49e2c6-7d3b-49bd-a5b8-90fe3c8e8d5e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-8a49e2c6-7d3b-49bd-a5b8-90fe3c8e8d5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:30.107459 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:30.115346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:30.116855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:30.116855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:30.116855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:30.116855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:30.116855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 259/514] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:30.116855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49e2c6-7d3b-49bd-a5b8-90fe3c8e8d5e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:30.151117 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa94cc5-5db3-49b2-9c1f-382376379187 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:30.170219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:30.187193 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b3d30-1b33-42a7-9dc3-8142b555551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:30.187866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b3d30-1b33-42a7-9dc3-8142b555551c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:30.188145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b3d30-1b33-42a7-9dc3-8142b555551c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:30.188455 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b3d30-1b33-42a7-9dc3-8142b555551c None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:30.189178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b3d30-1b33-42a7-9dc3-8142b555551c None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:30.189486 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b3d30-1b33-42a7-9dc3-8142b555551c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:30.190936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b3d30-1b33-42a7-9dc3-8142b555551c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:30.197800 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:30.202665 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 256/515] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:30.203522 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b3d30-1b33-42a7-9dc3-8142b555551c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:30.215346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49e2c6-7d3b-49bd-a5b8-90fe3c8e8d5e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:30.253746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95b412-f05f-4103-833b-12456ace0d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:30.254464 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95b412-f05f-4103-833b-12456ace0d57 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:30.254751 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95b412-f05f-4103-833b-12456ace0d57 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:30.255033 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95b412-f05f-4103-833b-12456ace0d57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:30.262457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95b412-f05f-4103-833b-12456ace0d57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:30.263934 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95b412-f05f-4103-833b-12456ace0d57 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-6a95b412-f05f-4103-833b-12456ace0d57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:30.278806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:30.280032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:30.281848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:30.281848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:30.281848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:30.281848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:30.281848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:30.281848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:30.281848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:30.281848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:30.281848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 260/516] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:30.281848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95b412-f05f-4103-833b-12456ace0d57 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:30.327718 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b3d30-1b33-42a7-9dc3-8142b555551c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:30.363142 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809382cd-1061-44d6-8c5f-ea58d68b6e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:30.363816 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809382cd-1061-44d6-8c5f-ea58d68b6e34 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:30.364089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809382cd-1061-44d6-8c5f-ea58d68b6e34 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:30.364385 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809382cd-1061-44d6-8c5f-ea58d68b6e34 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:30.365229 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809382cd-1061-44d6-8c5f-ea58d68b6e34 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:30.365536 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809382cd-1061-44d6-8c5f-ea58d68b6e34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:30.367156 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809382cd-1061-44d6-8c5f-ea58d68b6e34 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:30.385666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 257/517] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:07:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:30.386208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809382cd-1061-44d6-8c5f-ea58d68b6e34 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:30.386430 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95b412-f05f-4103-833b-12456ace0d57 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:30.423399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:30.426618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31fe5f7-1c2c-44e7-a3e6-3eca5b122f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:30.427347 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31fe5f7-1c2c-44e7-a3e6-3eca5b122f35 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:30.427627 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31fe5f7-1c2c-44e7-a3e6-3eca5b122f35 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:30.427980 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31fe5f7-1c2c-44e7-a3e6-3eca5b122f35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:30.444307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:30.491650 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809382cd-1061-44d6-8c5f-ea58d68b6e34 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:30.498338 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31fe5f7-1c2c-44e7-a3e6-3eca5b122f35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:30.500532 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31fe5f7-1c2c-44e7-a3e6-3eca5b122f35 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:30.527620 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b44f09-9ad6-40c5-9778-385bdaa2dfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:30.528209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b44f09-9ad6-40c5-9778-385bdaa2dfcd None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:30.528530 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b44f09-9ad6-40c5-9778-385bdaa2dfcd None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:30.528807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b44f09-9ad6-40c5-9778-385bdaa2dfcd None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:30.529542 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b44f09-9ad6-40c5-9778-385bdaa2dfcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:30.529909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b44f09-9ad6-40c5-9778-385bdaa2dfcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:30.531326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b44f09-9ad6-40c5-9778-385bdaa2dfcd None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:30.543204 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 258/518] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:30.544033 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b44f09-9ad6-40c5-9778-385bdaa2dfcd None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:30.605807 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 261/519] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:30.606694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31fe5f7-1c2c-44e7-a3e6-3eca5b122f35 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:30.644641 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:30.653979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b44f09-9ad6-40c5-9778-385bdaa2dfcd None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:30.660560 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:30.690342 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403a00b-0004-4cf2-ada2-3f7bfcb32536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:30.691031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403a00b-0004-4cf2-ada2-3f7bfcb32536 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:30.691320 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403a00b-0004-4cf2-ada2-3f7bfcb32536 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:30.691603 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1403a00b-0004-4cf2-ada2-3f7bfcb32536 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:30.692319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403a00b-0004-4cf2-ada2-3f7bfcb32536 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:30.693797 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1403a00b-0004-4cf2-ada2-3f7bfcb32536 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:30.711510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31fe5f7-1c2c-44e7-a3e6-3eca5b122f35 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:30.745412 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597033b-60fc-40ac-ae4d-42adef68a95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:30.746069 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597033b-60fc-40ac-ae4d-42adef68a95d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:30.746369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597033b-60fc-40ac-ae4d-42adef68a95d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:30.746631 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597033b-60fc-40ac-ae4d-42adef68a95d None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:30.747406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597033b-60fc-40ac-ae4d-42adef68a95d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:30.747718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597033b-60fc-40ac-ae4d-42adef68a95d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:30.749363 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597033b-60fc-40ac-ae4d-42adef68a95d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:30.767524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 262/520] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:30.768466 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597033b-60fc-40ac-ae4d-42adef68a95d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:30.777158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 259/521] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:07:30 2022] POST /identity/v3/users => generated 247 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:07:30.778226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403a00b-0004-4cf2-ada2-3f7bfcb32536 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:30.889340 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597033b-60fc-40ac-ae4d-42adef68a95d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:30.899706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:30.921526 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1403a00b-0004-4cf2-ada2-3f7bfcb32536 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:30.932422 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c4dd6-e724-44e1-9a2c-86e964311c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:30.933480 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c4dd6-e724-44e1-9a2c-86e964311c58 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:30.933777 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c4dd6-e724-44e1-9a2c-86e964311c58 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:30.934057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c4dd6-e724-44e1-9a2c-86e964311c58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:30.967628 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6458d55-fd97-44a4-b401-10aceaee998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:30.968599 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6458d55-fd97-44a4-b401-10aceaee998e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:30.968957 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6458d55-fd97-44a4-b401-10aceaee998e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:30.969313 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6458d55-fd97-44a4-b401-10aceaee998e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:30.980398 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6458d55-fd97-44a4-b401-10aceaee998e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:30.982052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6458d55-fd97-44a4-b401-10aceaee998e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-e6458d55-fd97-44a4-b401-10aceaee998e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:30.997752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:30.999527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:31.002051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:31.002051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:31.002051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:31.002051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:31.002051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:31.002051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:31.002051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:31.002051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:31.002051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:07:31.002051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:31.004306 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 260/522] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:31.005164 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6458d55-fd97-44a4-b401-10aceaee998e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:31.023559 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:31.028417 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c4dd6-e724-44e1-9a2c-86e964311c58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:31.030119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c4dd6-e724-44e1-9a2c-86e964311c58 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:31.145847 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6458d55-fd97-44a4-b401-10aceaee998e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:31.186224 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 263/523] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:31.186224 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c4dd6-e724-44e1-9a2c-86e964311c58 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:31.189751 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcdf31-c41d-4111-8c21-847f58af507f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:31.190596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcdf31-c41d-4111-8c21-847f58af507f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:31.190839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcdf31-c41d-4111-8c21-847f58af507f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:31.191115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcdf31-c41d-4111-8c21-847f58af507f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:31.194155 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:31.204426 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcdf31-c41d-4111-8c21-847f58af507f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:31.205948 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcdf31-c41d-4111-8c21-847f58af507f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-f3fcdf31-c41d-4111-8c21-847f58af507f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:31.227680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:31.228864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:31.229887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:31.229887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:31.229887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:31.229887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:31.229887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 261/524] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:31.230218 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcdf31-c41d-4111-8c21-847f58af507f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:31.304070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:31.315099 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c4dd6-e724-44e1-9a2c-86e964311c58 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:31.353255 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcdf31-c41d-4111-8c21-847f58af507f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:31.353506 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dac27-60c2-4052-a59c-edc65aeb9ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:31.354155 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dac27-60c2-4052-a59c-edc65aeb9ab0 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:31.354446 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dac27-60c2-4052-a59c-edc65aeb9ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:31.354734 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148dac27-60c2-4052-a59c-edc65aeb9ab0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:31.355576 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148dac27-60c2-4052-a59c-edc65aeb9ab0 None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=f6d44c613f524fe1804d6c9b65fe97ca) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:31.355904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148dac27-60c2-4052-a59c-edc65aeb9ab0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:31.357458 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148dac27-60c2-4052-a59c-edc65aeb9ab0 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:31.366759 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 264/525] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:31 2022] GET /identity/v3/role_assignments?user.id=f6d44c613f524fe1804d6c9b65fe97ca&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:31.388370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dac27-60c2-4052-a59c-edc65aeb9ab0 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:31.405302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8af554-bb96-4a7c-9ac6-b3db27d56e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:31.406622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8af554-bb96-4a7c-9ac6-b3db27d56e5d None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:31.406980 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8af554-bb96-4a7c-9ac6-b3db27d56e5d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:31.407374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8af554-bb96-4a7c-9ac6-b3db27d56e5d None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:31.408383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8af554-bb96-4a7c-9ac6-b3db27d56e5d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:31.408777 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8af554-bb96-4a7c-9ac6-b3db27d56e5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:31.410514 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8af554-bb96-4a7c-9ac6-b3db27d56e5d None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:31.434438 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 262/526] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 20:07:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:31.435553 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8af554-bb96-4a7c-9ac6-b3db27d56e5d None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:31.508569 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:31.549446 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148dac27-60c2-4052-a59c-edc65aeb9ab0 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:31.559606 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8af554-bb96-4a7c-9ac6-b3db27d56e5d None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:31.587412 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:31.594986 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567a8d5-059c-4474-8e21-10982fd1d6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:31.595990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567a8d5-059c-4474-8e21-10982fd1d6ad None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:31.596288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567a8d5-059c-4474-8e21-10982fd1d6ad None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:31.596610 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567a8d5-059c-4474-8e21-10982fd1d6ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:31.602525 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f036e84-c2ec-4bbb-b949-69c336907c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:31.603261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f036e84-c2ec-4bbb-b949-69c336907c6c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:31.603540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f036e84-c2ec-4bbb-b949-69c336907c6c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:31.603931 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f036e84-c2ec-4bbb-b949-69c336907c6c None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:31.605921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f036e84-c2ec-4bbb-b949-69c336907c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:31.606305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f036e84-c2ec-4bbb-b949-69c336907c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:31.607819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f036e84-c2ec-4bbb-b949-69c336907c6c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:31.612155 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567a8d5-059c-4474-8e21-10982fd1d6ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:31.613792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567a8d5-059c-4474-8e21-10982fd1d6ad None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:31.628418 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 263/527] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:31.630990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f036e84-c2ec-4bbb-b949-69c336907c6c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-0567a8d5-059c-4474-8e21-10982fd1d6ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:31.644731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:31.645851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:31.646845 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:31.646845 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:31.646845 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:31.646845 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:31.649654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 265/528] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:31.650709 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567a8d5-059c-4474-8e21-10982fd1d6ad None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:31.768706 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f036e84-c2ec-4bbb-b949-69c336907c6c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:31.778478 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567a8d5-059c-4474-8e21-10982fd1d6ad None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:31.810058 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:31.813802 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fedb8c-0751-43a0-abf9-0a9e17b1fd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:31.814553 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fedb8c-0751-43a0-abf9-0a9e17b1fd2e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:31.814916 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fedb8c-0751-43a0-abf9-0a9e17b1fd2e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:31.815316 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fedb8c-0751-43a0-abf9-0a9e17b1fd2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:31.826471 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c26107-d0f6-43bd-83fe-0e8d4ec2f35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:31.828238 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c26107-d0f6-43bd-83fe-0e8d4ec2f35e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:31.828824 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c26107-d0f6-43bd-83fe-0e8d4ec2f35e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:31.829692 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c26107-d0f6-43bd-83fe-0e8d4ec2f35e None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:31.841278 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c26107-d0f6-43bd-83fe-0e8d4ec2f35e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:31.843518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c26107-d0f6-43bd-83fe-0e8d4ec2f35e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-45c26107-d0f6-43bd-83fe-0e8d4ec2f35e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:31.859357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:31.860437 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:31.861609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:31.861609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:31.861609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:31.861609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:31.861609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:31.861609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:31.861609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:31.861609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:31.865319 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 266/529] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:31.866295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c26107-d0f6-43bd-83fe-0e8d4ec2f35e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:31.942509 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:31.988290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fedb8c-0751-43a0-abf9-0a9e17b1fd2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:31.990141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fedb8c-0751-43a0-abf9-0a9e17b1fd2e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:32.003453 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c26107-d0f6-43bd-83fe-0e8d4ec2f35e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:32.046947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c45801-c1cf-4577-b287-326f7eaa3e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:32.047790 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c45801-c1cf-4577-b287-326f7eaa3e13 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:32.047975 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c45801-c1cf-4577-b287-326f7eaa3e13 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:32.048165 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c45801-c1cf-4577-b287-326f7eaa3e13 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:32.049043 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c45801-c1cf-4577-b287-326f7eaa3e13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:32.049418 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c45801-c1cf-4577-b287-326f7eaa3e13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:32.050953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c45801-c1cf-4577-b287-326f7eaa3e13 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:32.069741 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 267/530] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:32.069741 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c45801-c1cf-4577-b287-326f7eaa3e13 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:32.137303 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:32.191469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 264/531] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:32.192713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fedb8c-0751-43a0-abf9-0a9e17b1fd2e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:32.258737 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c45801-c1cf-4577-b287-326f7eaa3e13 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:32.298622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:32.310607 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc3b18-3bf6-460b-a4e7-49e8de65e0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:32.311439 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc3b18-3bf6-460b-a4e7-49e8de65e0ae None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:32.312303 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc3b18-3bf6-460b-a4e7-49e8de65e0ae None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:32.312610 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc3b18-3bf6-460b-a4e7-49e8de65e0ae None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:32.313490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc3b18-3bf6-460b-a4e7-49e8de65e0ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:32.313962 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc3b18-3bf6-460b-a4e7-49e8de65e0ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:32.316398 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc3b18-3bf6-460b-a4e7-49e8de65e0ae None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:32.321231 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fedb8c-0751-43a0-abf9-0a9e17b1fd2e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:32.336634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 268/532] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:32.338705 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc3b18-3bf6-460b-a4e7-49e8de65e0ae None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:32.386788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a5f8f-87d0-47e1-9b45-55116a82b9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:32.388800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a5f8f-87d0-47e1-9b45-55116a82b9b6 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:32.389797 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a5f8f-87d0-47e1-9b45-55116a82b9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:32.390630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a5f8f-87d0-47e1-9b45-55116a82b9b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:32.404760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a5f8f-87d0-47e1-9b45-55116a82b9b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:32.408485 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a5f8f-87d0-47e1-9b45-55116a82b9b6 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-467a5f8f-87d0-47e1-9b45-55116a82b9b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:32.426600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:32.427697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:32.428755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:32.428755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:32.428755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:32.428755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:32.428755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:32.428755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:32.428755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:32.428755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:32.436319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 265/533] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:32.437636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a5f8f-87d0-47e1-9b45-55116a82b9b6 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:32.468913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc3b18-3bf6-460b-a4e7-49e8de65e0ae None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:32.471803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:32.522185 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30eff7-7ac6-4559-9757-a098f5f0deba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:32.523000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30eff7-7ac6-4559-9757-a098f5f0deba None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:32.523289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30eff7-7ac6-4559-9757-a098f5f0deba None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:32.523570 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30eff7-7ac6-4559-9757-a098f5f0deba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:32.537550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30eff7-7ac6-4559-9757-a098f5f0deba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:32.539461 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30eff7-7ac6-4559-9757-a098f5f0deba None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-5b30eff7-7ac6-4559-9757-a098f5f0deba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:32.566654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:32.567791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:32.568849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:32.568849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:32.568849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:32.568849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:32.572775 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 269/534] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:32.573761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30eff7-7ac6-4559-9757-a098f5f0deba None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:32.589621 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a5f8f-87d0-47e1-9b45-55116a82b9b6 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:32.634863 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05b2ce-d557-43b0-bf8b-a548602b08ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:32.635545 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b2ce-d557-43b0-bf8b-a548602b08ed None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:32.635834 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b2ce-d557-43b0-bf8b-a548602b08ed None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:32.636134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b2ce-d557-43b0-bf8b-a548602b08ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:32.637575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05b2ce-d557-43b0-bf8b-a548602b08ed None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=4cebf4a3afdc4124aef25bed824c1dec) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:32.637898 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05b2ce-d557-43b0-bf8b-a548602b08ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:32.639704 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05b2ce-d557-43b0-bf8b-a548602b08ed None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:32.648562 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 266/535] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:32 2022] GET /identity/v3/role_assignments?user.id=4cebf4a3afdc4124aef25bed824c1dec&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:32.650045 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05b2ce-d557-43b0-bf8b-a548602b08ed None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:32.663879 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:32.767059 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30eff7-7ac6-4559-9757-a098f5f0deba None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:32.848035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e815a-b106-4d68-9109-4dbdf24af183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:32.849187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e815a-b106-4d68-9109-4dbdf24af183 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:32.849700 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e815a-b106-4d68-9109-4dbdf24af183 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:32.850190 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e815a-b106-4d68-9109-4dbdf24af183 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:32.851732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e815a-b106-4d68-9109-4dbdf24af183 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:32.852296 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e815a-b106-4d68-9109-4dbdf24af183 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:32.855147 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e815a-b106-4d68-9109-4dbdf24af183 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:32.886978 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05b2ce-d557-43b0-bf8b-a548602b08ed None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:32.896115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 270/536] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:32.897904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e815a-b106-4d68-9109-4dbdf24af183 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:32.920672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:32.938873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93e0b2-6f8d-4b80-98fd-25ec267897f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:32.940285 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93e0b2-6f8d-4b80-98fd-25ec267897f3 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:32.941062 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93e0b2-6f8d-4b80-98fd-25ec267897f3 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:32.941558 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93e0b2-6f8d-4b80-98fd-25ec267897f3 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:32.942512 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93e0b2-6f8d-4b80-98fd-25ec267897f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:32.943038 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93e0b2-6f8d-4b80-98fd-25ec267897f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:32.945784 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93e0b2-6f8d-4b80-98fd-25ec267897f3 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:32.963667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 267/537] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:32.972166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93e0b2-6f8d-4b80-98fd-25ec267897f3 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:33.080243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e815a-b106-4d68-9109-4dbdf24af183 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:33.103073 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:33.137534 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013bfc7-1f8a-4d5c-93ed-e0bab689740e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:33.138475 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013bfc7-1f8a-4d5c-93ed-e0bab689740e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:33.138917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013bfc7-1f8a-4d5c-93ed-e0bab689740e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:33.139377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013bfc7-1f8a-4d5c-93ed-e0bab689740e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:33.140943 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013bfc7-1f8a-4d5c-93ed-e0bab689740e None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=c8710592e5a84bb4b885b050e23549dc) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:33.141665 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013bfc7-1f8a-4d5c-93ed-e0bab689740e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:33.144675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013bfc7-1f8a-4d5c-93ed-e0bab689740e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:33.156785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 271/538] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:32 2022] GET /identity/v3/role_assignments?user.id=c8710592e5a84bb4b885b050e23549dc&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:33.220223 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93e0b2-6f8d-4b80-98fd-25ec267897f3 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:33.286607 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dae5a5-f36e-47df-b442-096a75ef43d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:33.288032 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dae5a5-f36e-47df-b442-096a75ef43d2 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:33.288585 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dae5a5-f36e-47df-b442-096a75ef43d2 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:33.289102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dae5a5-f36e-47df-b442-096a75ef43d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:33.302845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dae5a5-f36e-47df-b442-096a75ef43d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:33.305919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dae5a5-f36e-47df-b442-096a75ef43d2 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-f7dae5a5-f36e-47df-b442-096a75ef43d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:33.335807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:33.336979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:33.338220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:33.338220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:33.338220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:33.338220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:33.338220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:33.338220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:33.338220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:33.338220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:33.338220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 268/539] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:33.344035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013bfc7-1f8a-4d5c-93ed-e0bab689740e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:33.387902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef8f42-e24b-41f6-b444-1b0f17068a5f None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:33.388121 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef8f42-e24b-41f6-b444-1b0f17068a5f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:33.388392 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef8f42-e24b-41f6-b444-1b0f17068a5f None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:33.390332 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 269/540] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:33.396399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74dc9b-66d2-4bc6-8403-0457716f9d28 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:33.396881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74dc9b-66d2-4bc6-8403-0457716f9d28 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:33.397364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74dc9b-66d2-4bc6-8403-0457716f9d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:33.433490 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-1b74dc9b-66d2-4bc6-8403-0457716f9d28 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:33.434740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-1b74dc9b-66d2-4bc6-8403-0457716f9d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:33.492939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:33.520321 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013bfc7-1f8a-4d5c-93ed-e0bab689740e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:33.562596 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39072c2-e882-473a-997f-7313024d873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:33.563427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39072c2-e882-473a-997f-7313024d873b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:33.563734 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39072c2-e882-473a-997f-7313024d873b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:33.564026 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39072c2-e882-473a-997f-7313024d873b None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:33.565088 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39072c2-e882-473a-997f-7313024d873b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:33.565405 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39072c2-e882-473a-997f-7313024d873b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:33.567092 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39072c2-e882-473a-997f-7313024d873b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:33.585595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 272/541] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:33.591284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39072c2-e882-473a-997f-7313024d873b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:33.603024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 270/542] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:07:33.627044 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74dc9b-66d2-4bc6-8403-0457716f9d28 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:33.728468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39072c2-e882-473a-997f-7313024d873b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:33.776365 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b44a6-babe-409b-952e-bf683fe64fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:33.776365 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b44a6-babe-409b-952e-bf683fe64fd0 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:33.776365 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b44a6-babe-409b-952e-bf683fe64fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:33.776365 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b44a6-babe-409b-952e-bf683fe64fd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:33.776838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b44a6-babe-409b-952e-bf683fe64fd0 None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=d7917b3c79bd42cea17ec8eba5a265ed) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:33.776838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b44a6-babe-409b-952e-bf683fe64fd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:33.776838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b44a6-babe-409b-952e-bf683fe64fd0 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:33.779040 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:33.790799 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 273/543] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:33 2022] GET /identity/v3/role_assignments?user.id=d7917b3c79bd42cea17ec8eba5a265ed&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:33.867208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74dc9b-66d2-4bc6-8403-0457716f9d28 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:33.922039 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8247be-5240-4f60-adda-3265311a79d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:33.922887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8247be-5240-4f60-adda-3265311a79d1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:33.923173 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8247be-5240-4f60-adda-3265311a79d1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:33.923435 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8247be-5240-4f60-adda-3265311a79d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:33.938601 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8247be-5240-4f60-adda-3265311a79d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-3d8247be-5240-4f60-adda-3265311a79d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:33.947355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:33.948548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:33.950145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 271/544] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:33.955387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b44a6-babe-409b-952e-bf683fe64fd0 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:34.000317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:34.011384 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435b681-ffcd-4c20-aa87-5581f7490808 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:34.011732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435b681-ffcd-4c20-aa87-5581f7490808 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:34.017572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435b681-ffcd-4c20-aa87-5581f7490808 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:34.017572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 272/545] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:34.019930 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4284a5-f67b-46f4-873d-08e68c1283b3 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:34.020219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4284a5-f67b-46f4-873d-08e68c1283b3 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:34.020540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4284a5-f67b-46f4-873d-08e68c1283b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:34.047122 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-3f4284a5-f67b-46f4-873d-08e68c1283b3 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:34.048512 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-3f4284a5-f67b-46f4-873d-08e68c1283b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:34.108186 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b44a6-babe-409b-952e-bf683fe64fd0 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:34.167362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e308e3-fadc-4311-83b5-de39a706d362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:34.168054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e308e3-fadc-4311-83b5-de39a706d362 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:34.168357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e308e3-fadc-4311-83b5-de39a706d362 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:34.168685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e308e3-fadc-4311-83b5-de39a706d362 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:34.169502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e308e3-fadc-4311-83b5-de39a706d362 None admin] RBAC: Adding query filter params (name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:34.175056 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e308e3-fadc-4311-83b5-de39a706d362 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:34.176573 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e308e3-fadc-4311-83b5-de39a706d362 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:34.195695 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 274/546] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:34.202360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e308e3-fadc-4311-83b5-de39a706d362 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:34.212495 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 273/547] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:07:34.250448 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4284a5-f67b-46f4-873d-08e68c1283b3 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:34.304758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:34.344207 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e308e3-fadc-4311-83b5-de39a706d362 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:34.383889 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deff185-a779-4390-a888-567c24a0b49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:34.384815 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deff185-a779-4390-a888-567c24a0b49b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:34.385107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deff185-a779-4390-a888-567c24a0b49b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:34.385389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deff185-a779-4390-a888-567c24a0b49b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:34.390727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:34.400046 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deff185-a779-4390-a888-567c24a0b49b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:34.403847 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deff185-a779-4390-a888-567c24a0b49b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-5deff185-a779-4390-a888-567c24a0b49b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:34.422552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:34.423821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:34.425149 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:34.425149 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:34.425149 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:34.425149 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:34.428085 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 275/548] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:34.433556 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deff185-a779-4390-a888-567c24a0b49b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:34.525548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4284a5-f67b-46f4-873d-08e68c1283b3 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:34.549025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deff185-a779-4390-a888-567c24a0b49b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:34.589111 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c016463-b990-4213-a108-d932872d2607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:34.589981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c016463-b990-4213-a108-d932872d2607 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:34.590544 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c016463-b990-4213-a108-d932872d2607 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:34.590544 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c016463-b990-4213-a108-d932872d2607 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:34.596740 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:34.599655 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc82f7-3d70-454c-89bf-35013cab315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:34.600556 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc82f7-3d70-454c-89bf-35013cab315a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:34.601029 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc82f7-3d70-454c-89bf-35013cab315a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:34.601490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc82f7-3d70-454c-89bf-35013cab315a None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:34.602378 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc82f7-3d70-454c-89bf-35013cab315a None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:34.602861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc82f7-3d70-454c-89bf-35013cab315a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:34.604536 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc82f7-3d70-454c-89bf-35013cab315a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:34.612398 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c016463-b990-4213-a108-d932872d2607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:34.619600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 276/549] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:34.624826 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc82f7-3d70-454c-89bf-35013cab315a None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-3c016463-b990-4213-a108-d932872d2607 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:34.626147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:34.627280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:34.628210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:34.631321 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 274/550] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:34.637299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c016463-b990-4213-a108-d932872d2607 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:34.801041 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c016463-b990-4213-a108-d932872d2607 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:34.806783 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc82f7-3d70-454c-89bf-35013cab315a None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:34.832705 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:34.852799 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3520-5420-48d5-870f-b39c7c49cf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:34.854002 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3520-5420-48d5-870f-b39c7c49cf8b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:34.854132 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f4e5d-6e3d-4323-b7c5-4a301913fe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:34.854661 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3520-5420-48d5-870f-b39c7c49cf8b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:34.854789 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f4e5d-6e3d-4323-b7c5-4a301913fe74 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:34.854966 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f4e5d-6e3d-4323-b7c5-4a301913fe74 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:34.855235 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f4e5d-6e3d-4323-b7c5-4a301913fe74 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:34.855805 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e3520-5420-48d5-870f-b39c7c49cf8b None admin] PATH_INFO: `/v3/users/swift` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:34.856035 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f4e5d-6e3d-4323-b7c5-4a301913fe74 None admin] RBAC: Adding query filter params (name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:34.856402 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f4e5d-6e3d-4323-b7c5-4a301913fe74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:34.857814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f4e5d-6e3d-4323-b7c5-4a301913fe74 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:34.865393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e3520-5420-48d5-870f-b39c7c49cf8b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:34.867304 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e3520-5420-48d5-870f-b39c7c49cf8b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:34.880369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 275/551] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:34.882858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f4e5d-6e3d-4323-b7c5-4a301913fe74 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-203e3520-5420-48d5-870f-b39c7c49cf8b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:34.890720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:34.892161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:34.901485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:34.901485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:34.901485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:34.901485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:34.901485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:34.901485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:34.901485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:34.901485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:34.901485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:07:34.901485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:34.904014 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 277/552] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:07:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:34.910102 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3520-5420-48d5-870f-b39c7c49cf8b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:34.988084 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:35.023636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f4e5d-6e3d-4323-b7c5-4a301913fe74 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:35.088479 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e3520-5420-48d5-870f-b39c7c49cf8b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:35.116806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f69f-e5d9-43ff-ac28-1cb48b5d60f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:35.118848 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f69f-e5d9-43ff-ac28-1cb48b5d60f3 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.119635 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f69f-e5d9-43ff-ac28-1cb48b5d60f3 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.122512 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47f69f-e5d9-43ff-ac28-1cb48b5d60f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.143562 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47f69f-e5d9-43ff-ac28-1cb48b5d60f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:35.146269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47f69f-e5d9-43ff-ac28-1cb48b5d60f3 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:35.161233 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7cf888-2c22-407c-99e8-5b24d3c29afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:35.162559 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7cf888-2c22-407c-99e8-5b24d3c29afc None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.163030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7cf888-2c22-407c-99e8-5b24d3c29afc None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.163515 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7cf888-2c22-407c-99e8-5b24d3c29afc None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.164569 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7cf888-2c22-407c-99e8-5b24d3c29afc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:35.165067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7cf888-2c22-407c-99e8-5b24d3c29afc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:35.166716 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7cf888-2c22-407c-99e8-5b24d3c29afc None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-2a47f69f-e5d9-43ff-ac28-1cb48b5d60f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:35.190578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:35.191674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:35.192710 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:35.192710 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:35.192710 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:35.192710 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:35.199641 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 276/553] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:35.200867 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f69f-e5d9-43ff-ac28-1cb48b5d60f3 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:35.228751 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:35.256363 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 278/554] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:07:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:35.261755 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7cf888-2c22-407c-99e8-5b24d3c29afc None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:35.330363 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47f69f-e5d9-43ff-ac28-1cb48b5d60f3 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:35.397097 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85032931-dd9f-4394-a254-d5c510139387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:35.397679 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85032931-dd9f-4394-a254-d5c510139387 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.397963 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85032931-dd9f-4394-a254-d5c510139387 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.398237 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85032931-dd9f-4394-a254-d5c510139387 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.398969 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85032931-dd9f-4394-a254-d5c510139387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:35.399274 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85032931-dd9f-4394-a254-d5c510139387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:35.400678 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85032931-dd9f-4394-a254-d5c510139387 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:35.432133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 277/555] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:35.436352 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85032931-dd9f-4394-a254-d5c510139387 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:35.452947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:35.465860 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7cf888-2c22-407c-99e8-5b24d3c29afc None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:35.509863 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3556ff-0c3c-4a96-b8c8-db961055cf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:35.510674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3556ff-0c3c-4a96-b8c8-db961055cf5f None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.510968 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3556ff-0c3c-4a96-b8c8-db961055cf5f None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.511240 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3556ff-0c3c-4a96-b8c8-db961055cf5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.628709 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:35.663815 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3556ff-0c3c-4a96-b8c8-db961055cf5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:35.665237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3556ff-0c3c-4a96-b8c8-db961055cf5f None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:35.665575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85032931-dd9f-4394-a254-d5c510139387 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:35.704186 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38809869-6132-442e-9723-f64a4d474f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:35.705402 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38809869-6132-442e-9723-f64a4d474f2b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.705906 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38809869-6132-442e-9723-f64a4d474f2b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.706401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38809869-6132-442e-9723-f64a4d474f2b None admin] PATH_INFO: `/v3/users/nova` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.714554 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38809869-6132-442e-9723-f64a4d474f2b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:35.716134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38809869-6132-442e-9723-f64a4d474f2b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-38809869-6132-442e-9723-f64a4d474f2b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:35.734167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:35.735414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:35.736497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:35.736497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:35.736497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:35.736497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:35.736497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:35.736497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:35.736497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:35.736497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:35.736497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:07:35.736497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:35.736497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 278/556] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:07:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:35.737971 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b228a5-15ec-483e-bad3-694ee3538e9e None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.738336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b228a5-15ec-483e-bad3-694ee3538e9e None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.738685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b228a5-15ec-483e-bad3-694ee3538e9e None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.740495 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 279/557] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:35.746163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b228a5-15ec-483e-bad3-694ee3538e9e None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:35.796674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:35.806065 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 279/558] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:07:35.811724 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580de8d-7024-4b49-a098-9bddfde9e9ad None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.812018 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580de8d-7024-4b49-a098-9bddfde9e9ad None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.812403 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580de8d-7024-4b49-a098-9bddfde9e9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.829915 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-b580de8d-7024-4b49-a098-9bddfde9e9ad None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:35.831117 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-b580de8d-7024-4b49-a098-9bddfde9e9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:35.899113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b228a5-15ec-483e-bad3-694ee3538e9e None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:35.934623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 280/559] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:35.936051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839561b-408f-4071-9798-087f04d9269a None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.936370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839561b-408f-4071-9798-087f04d9269a None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.936985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839561b-408f-4071-9798-087f04d9269a None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.940514 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 281/560] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:35.940514 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839561b-408f-4071-9798-087f04d9269a None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:35.956046 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e27f2b-ee87-46e1-b2ba-91bcaae43a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:35.957123 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e27f2b-ee87-46e1-b2ba-91bcaae43a27 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.957541 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e27f2b-ee87-46e1-b2ba-91bcaae43a27 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.957927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e27f2b-ee87-46e1-b2ba-91bcaae43a27 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:35.959106 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e27f2b-ee87-46e1-b2ba-91bcaae43a27 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:35.959605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e27f2b-ee87-46e1-b2ba-91bcaae43a27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:35.961117 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e27f2b-ee87-46e1-b2ba-91bcaae43a27 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:35.972794 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:35.983681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 280/561] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:07:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:35.986234 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ece99-70f3-4164-8a9d-afa7b0e1b921 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:35.986767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ece99-70f3-4164-8a9d-afa7b0e1b921 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:35.987489 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ece99-70f3-4164-8a9d-afa7b0e1b921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.006252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-9e1ece99-70f3-4164-8a9d-afa7b0e1b921 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:36.008323 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-9e1ece99-70f3-4164-8a9d-afa7b0e1b921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:36.070833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839561b-408f-4071-9798-087f04d9269a None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:36.101925 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:36.106943 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 281/562] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:36.109898 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ece99-70f3-4164-8a9d-afa7b0e1b921 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:36.120194 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39e9cb-cc9f-4a92-879b-3b9ad4626436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:36.121061 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39e9cb-cc9f-4a92-879b-3b9ad4626436 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.121347 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39e9cb-cc9f-4a92-879b-3b9ad4626436 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.121626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39e9cb-cc9f-4a92-879b-3b9ad4626436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.135689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39e9cb-cc9f-4a92-879b-3b9ad4626436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:36.137263 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39e9cb-cc9f-4a92-879b-3b9ad4626436 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-df39e9cb-cc9f-4a92-879b-3b9ad4626436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:36.159044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:36.160133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:36.161219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:36.161219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:36.161219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:36.161219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:36.167370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 282/563] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:36.167370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39e9cb-cc9f-4a92-879b-3b9ad4626436 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:36.263251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ece99-70f3-4164-8a9d-afa7b0e1b921 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:36.288589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39e9cb-cc9f-4a92-879b-3b9ad4626436 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:36.294381 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:36.303135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3b537-eebb-4e8e-a886-8e70d76d976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:36.311962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3b537-eebb-4e8e-a886-8e70d76d976e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.311962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3b537-eebb-4e8e-a886-8e70d76d976e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.311962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3b537-eebb-4e8e-a886-8e70d76d976e None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.316358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3b537-eebb-4e8e-a886-8e70d76d976e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-c7e3b537-eebb-4e8e-a886-8e70d76d976e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:36.324141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:36.326157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 282/564] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:36.328667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3b537-eebb-4e8e-a886-8e70d76d976e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:36.344537 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff9937-6851-4f98-bb80-232a64e64eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:36.345347 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff9937-6851-4f98-bb80-232a64e64eb1 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.345622 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff9937-6851-4f98-bb80-232a64e64eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.345897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff9937-6851-4f98-bb80-232a64e64eb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.368717 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff9937-6851-4f98-bb80-232a64e64eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:36.371507 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff9937-6851-4f98-bb80-232a64e64eb1 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-54ff9937-6851-4f98-bb80-232a64e64eb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:36.393187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:36.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:36.395337 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:36.395337 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:36.395337 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:36.395337 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:36.395666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 283/565] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:36.396732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff9937-6851-4f98-bb80-232a64e64eb1 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:36.428733 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:36.460946 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3b537-eebb-4e8e-a886-8e70d76d976e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:36.495973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b1a91-34a3-4d76-bd47-40ee9f46748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:36.496820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b1a91-34a3-4d76-bd47-40ee9f46748f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.497107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b1a91-34a3-4d76-bd47-40ee9f46748f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.497387 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b1a91-34a3-4d76-bd47-40ee9f46748f None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.506364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff9937-6851-4f98-bb80-232a64e64eb1 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:36.508794 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b1a91-34a3-4d76-bd47-40ee9f46748f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-072b1a91-34a3-4d76-bd47-40ee9f46748f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:36.517280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:36.518501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 283/566] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:36.519470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b1a91-34a3-4d76-bd47-40ee9f46748f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:36.548213 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0831f5-fb9f-48f3-bab4-8bdfd12aa458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:36.548993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0831f5-fb9f-48f3-bab4-8bdfd12aa458 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.549276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0831f5-fb9f-48f3-bab4-8bdfd12aa458 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.549550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0831f5-fb9f-48f3-bab4-8bdfd12aa458 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.550275 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0831f5-fb9f-48f3-bab4-8bdfd12aa458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:36.550585 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0831f5-fb9f-48f3-bab4-8bdfd12aa458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:36.551976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0831f5-fb9f-48f3-bab4-8bdfd12aa458 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:36.565336 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 284/567] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:36.566766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2cd9-92bb-43eb-9f9c-927f8b21c50e None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.567060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2cd9-92bb-43eb-9f9c-927f8b21c50e None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.567333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe2cd9-92bb-43eb-9f9c-927f8b21c50e None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.568768 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 285/568] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:36.569495 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe2cd9-92bb-43eb-9f9c-927f8b21c50e None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:36.575860 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:36.644030 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b1a91-34a3-4d76-bd47-40ee9f46748f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:36.677095 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:36.685258 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b98ff4-537f-489e-8aea-564990d38085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:36.689250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b98ff4-537f-489e-8aea-564990d38085 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.689250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b98ff4-537f-489e-8aea-564990d38085 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.689250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b98ff4-537f-489e-8aea-564990d38085 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.689250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b98ff4-537f-489e-8aea-564990d38085 None admin] RBAC: Adding query filter params (name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:36.689250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b98ff4-537f-489e-8aea-564990d38085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:36.689250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b98ff4-537f-489e-8aea-564990d38085 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:36.690516 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe2cd9-92bb-43eb-9f9c-927f8b21c50e None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:36.698220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 284/569] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:36.699089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b98ff4-537f-489e-8aea-564990d38085 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:36.730391 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabcb41-df6c-4179-b360-4390079b098d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:36.731095 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabcb41-df6c-4179-b360-4390079b098d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.731388 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabcb41-df6c-4179-b360-4390079b098d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.731657 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabcb41-df6c-4179-b360-4390079b098d None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.732422 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabcb41-df6c-4179-b360-4390079b098d None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:36.732733 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabcb41-df6c-4179-b360-4390079b098d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:36.734135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabcb41-df6c-4179-b360-4390079b098d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:36.745693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 286/570] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:36.746664 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabcb41-df6c-4179-b360-4390079b098d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:36.813550 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b98ff4-537f-489e-8aea-564990d38085 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:36.824744 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:36.852786 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f3778-002e-4b89-9954-afe1a48b1c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:36.853722 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f3778-002e-4b89-9954-afe1a48b1c94 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.853722 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f3778-002e-4b89-9954-afe1a48b1c94 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.853999 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f3778-002e-4b89-9954-afe1a48b1c94 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.854752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f3778-002e-4b89-9954-afe1a48b1c94 None admin] RBAC: Adding query filter params (name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:36.855150 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f3778-002e-4b89-9954-afe1a48b1c94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:36.856626 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f3778-002e-4b89-9954-afe1a48b1c94 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:36.858189 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabcb41-df6c-4179-b360-4390079b098d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:36.866195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 285/571] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:36.867477 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c9fcb-6b3b-436e-b913-8230ffca763d None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.867767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c9fcb-6b3b-436e-b913-8230ffca763d None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.868050 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c9fcb-6b3b-436e-b913-8230ffca763d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.888686 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-997c9fcb-6b3b-436e-b913-8230ffca763d None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:36.889931 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-997c9fcb-6b3b-436e-b913-8230ffca763d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:36.911331 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf8d6c-0fee-4c92-9ddb-558b91232055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:36.912142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf8d6c-0fee-4c92-9ddb-558b91232055 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:36.912472 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf8d6c-0fee-4c92-9ddb-558b91232055 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:36.912737 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf8d6c-0fee-4c92-9ddb-558b91232055 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:36.920171 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf8d6c-0fee-4c92-9ddb-558b91232055 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:36.922536 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf8d6c-0fee-4c92-9ddb-558b91232055 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-1cbf8d6c-0fee-4c92-9ddb-558b91232055 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:36.937093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:36.938351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:36.939518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:36.939518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:36.939518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:36.939518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:36.939518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:36.939518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:36.939518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:36.939518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:36.939518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 287/572] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:36.939518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf8d6c-0fee-4c92-9ddb-558b91232055 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:36.951320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:36.969299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 286/573] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:36.970168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c9fcb-6b3b-436e-b913-8230ffca763d None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:37.057095 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf8d6c-0fee-4c92-9ddb-558b91232055 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:37.082870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c9fcb-6b3b-436e-b913-8230ffca763d None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:37.084492 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:37.097678 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cafe9a-c293-4b66-bfcd-a7b454494876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:37.098120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cafe9a-c293-4b66-bfcd-a7b454494876 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:37.098515 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cafe9a-c293-4b66-bfcd-a7b454494876 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:37.098762 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cafe9a-c293-4b66-bfcd-a7b454494876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:37.110727 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cafe9a-c293-4b66-bfcd-a7b454494876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:37.112431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cafe9a-c293-4b66-bfcd-a7b454494876 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:37.123005 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2ca87-156e-4a10-928c-4b98f46850c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:37.123818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2ca87-156e-4a10-928c-4b98f46850c1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:37.124134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2ca87-156e-4a10-928c-4b98f46850c1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:37.124484 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2ca87-156e-4a10-928c-4b98f46850c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-a1cafe9a-c293-4b66-bfcd-a7b454494876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:37.130523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:37.131699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:37.133077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:37.133077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:37.133077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:37.133077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:37.133378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2ca87-156e-4a10-928c-4b98f46850c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:37.133378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2ca87-156e-4a10-928c-4b98f46850c1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:37.133857 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 288/574] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:37.134760 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cafe9a-c293-4b66-bfcd-a7b454494876 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-70e2ca87-156e-4a10-928c-4b98f46850c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:37.148557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:37.149944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:37.151153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:37.151153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:37.151153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:37.151153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:37.151153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:37.151153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:37.151153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:37.151153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:37.151153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 287/575] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:37.151153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2ca87-156e-4a10-928c-4b98f46850c1 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:37.218659 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:37.262915 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2ca87-156e-4a10-928c-4b98f46850c1 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:37.266455 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cafe9a-c293-4b66-bfcd-a7b454494876 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:37.299007 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efb1fb-6220-4cbd-a570-9542c2ad03a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:37.299672 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efb1fb-6220-4cbd-a570-9542c2ad03a1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:37.299956 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efb1fb-6220-4cbd-a570-9542c2ad03a1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:37.300230 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efb1fb-6220-4cbd-a570-9542c2ad03a1 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:37.301051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efb1fb-6220-4cbd-a570-9542c2ad03a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:37.301365 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efb1fb-6220-4cbd-a570-9542c2ad03a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:37.302797 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efb1fb-6220-4cbd-a570-9542c2ad03a1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:37.303725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89f614-1aad-49ed-a501-893971c2d69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:37.304586 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89f614-1aad-49ed-a501-893971c2d69c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:37.304873 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89f614-1aad-49ed-a501-893971c2d69c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:37.305153 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89f614-1aad-49ed-a501-893971c2d69c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:37.316281 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89f614-1aad-49ed-a501-893971c2d69c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:37.317787 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b89f614-1aad-49ed-a501-893971c2d69c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:37.322346 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 288/576] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:37.323207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efb1fb-6220-4cbd-a570-9542c2ad03a1 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-8b89f614-1aad-49ed-a501-893971c2d69c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:37.339016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:37.340094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:37.341450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:37.341450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:37.341450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:37.341450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:37.341450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 289/577] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:37.341450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89f614-1aad-49ed-a501-893971c2d69c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:37.355066 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:37.434452 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efb1fb-6220-4cbd-a570-9542c2ad03a1 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:37.456139 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89f614-1aad-49ed-a501-893971c2d69c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:37.470452 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:37.470858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ee572-9a97-4a8e-9861-e3c64e88adec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:37.471443 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ee572-9a97-4a8e-9861-e3c64e88adec None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:37.471733 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ee572-9a97-4a8e-9861-e3c64e88adec None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:37.472015 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ee572-9a97-4a8e-9861-e3c64e88adec None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:37.482816 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ee572-9a97-4a8e-9861-e3c64e88adec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-e68ee572-9a97-4a8e-9861-e3c64e88adec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:37.490977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:37.492161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:37.493720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 289/578] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:37.494605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0845ef-2251-4e23-bfd8-f728b478974a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:37.494714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ee572-9a97-4a8e-9861-e3c64e88adec None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:37.494807 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0845ef-2251-4e23-bfd8-f728b478974a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:37.494807 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0845ef-2251-4e23-bfd8-f728b478974a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:37.494807 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0845ef-2251-4e23-bfd8-f728b478974a None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:37.495036 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0845ef-2251-4e23-bfd8-f728b478974a None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:37.495217 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0845ef-2251-4e23-bfd8-f728b478974a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:37.496803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0845ef-2251-4e23-bfd8-f728b478974a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:37.513096 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 290/579] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:37.513964 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0845ef-2251-4e23-bfd8-f728b478974a None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:37.593728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:37.611775 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ee572-9a97-4a8e-9861-e3c64e88adec None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:37.619634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0845ef-2251-4e23-bfd8-f728b478974a None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:37.647651 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b53ac-14d2-46c6-9b02-41941960866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:37.648362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b53ac-14d2-46c6-9b02-41941960866a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:37.648642 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b53ac-14d2-46c6-9b02-41941960866a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:37.648922 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b53ac-14d2-46c6-9b02-41941960866a None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:37.649718 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b53ac-14d2-46c6-9b02-41941960866a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:37.650028 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b53ac-14d2-46c6-9b02-41941960866a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:37.651475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b53ac-14d2-46c6-9b02-41941960866a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:37.658471 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55493c-83f0-45ab-8eb7-e4bc8e81891d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:37.659130 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55493c-83f0-45ab-8eb7-e4bc8e81891d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:37.659423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55493c-83f0-45ab-8eb7-e4bc8e81891d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:37.659770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55493c-83f0-45ab-8eb7-e4bc8e81891d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:37.660642 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55493c-83f0-45ab-8eb7-e4bc8e81891d None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=7777c807131c4d9886c6aeff5f756711) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:37.660960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55493c-83f0-45ab-8eb7-e4bc8e81891d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:37.662405 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55493c-83f0-45ab-8eb7-e4bc8e81891d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:37.664741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 290/580] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:37.665498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b53ac-14d2-46c6-9b02-41941960866a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:37.672882 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 291/581] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:37 2022] GET /identity/v3/role_assignments?user.id=7777c807131c4d9886c6aeff5f756711&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:37.673966 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55493c-83f0-45ab-8eb7-e4bc8e81891d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:37.725729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:37.803023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b53ac-14d2-46c6-9b02-41941960866a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:37.810378 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55493c-83f0-45ab-8eb7-e4bc8e81891d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:37.848480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ee2e7-e6b8-4f29-a287-9ded1ac22956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:37.849071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ee2e7-e6b8-4f29-a287-9ded1ac22956 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:37.849357 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ee2e7-e6b8-4f29-a287-9ded1ac22956 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:37.849636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ee2e7-e6b8-4f29-a287-9ded1ac22956 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:37.850476 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ee2e7-e6b8-4f29-a287-9ded1ac22956 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:37.850788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ee2e7-e6b8-4f29-a287-9ded1ac22956 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:37.852230 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ee2e7-e6b8-4f29-a287-9ded1ac22956 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:37.863165 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e75b3-be82-42c4-a1c9-6acbc8e7f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:37.863863 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e75b3-be82-42c4-a1c9-6acbc8e7f7ef None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:37.864128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e75b3-be82-42c4-a1c9-6acbc8e7f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:37.864445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e75b3-be82-42c4-a1c9-6acbc8e7f7ef None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:37.865212 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e75b3-be82-42c4-a1c9-6acbc8e7f7ef None admin] RBAC: Adding query filter params (name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:37.865526 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e75b3-be82-42c4-a1c9-6acbc8e7f7ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:37.866126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 291/582] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:37.866828 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ee2e7-e6b8-4f29-a287-9ded1ac22956 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:37.869908 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:37.871578 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e75b3-be82-42c4-a1c9-6acbc8e7f7ef None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:37.882220 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 292/583] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:37.885853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e75b3-be82-42c4-a1c9-6acbc8e7f7ef None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:38.005570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ee2e7-e6b8-4f29-a287-9ded1ac22956 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:38.010368 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:38.015837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e75b3-be82-42c4-a1c9-6acbc8e7f7ef None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:38.048696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf36f4-8c79-4b9d-b521-3355290149c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:38.049511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf36f4-8c79-4b9d-b521-3355290149c5 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:38.049797 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf36f4-8c79-4b9d-b521-3355290149c5 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:38.050194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf36f4-8c79-4b9d-b521-3355290149c5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:38.058668 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf36f4-8c79-4b9d-b521-3355290149c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:38.059274 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd094ee-d250-408f-8a1d-77b2e377a5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:38.060102 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd094ee-d250-408f-8a1d-77b2e377a5a5 None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:38.060267 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf36f4-8c79-4b9d-b521-3355290149c5 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:38.060500 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd094ee-d250-408f-8a1d-77b2e377a5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:38.060710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd094ee-d250-408f-8a1d-77b2e377a5a5 None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/f6d44c613f524fe1804d6c9b65fe97ca/roles/afaf4273358148d3b0b9cac8dbeeba97` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-fadf36f4-8c79-4b9d-b521-3355290149c5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:38.078540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:38.079858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:38.081031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:38.081031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:38.081031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:38.081031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:38.081031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:38.081031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:38.081031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:38.081031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:38.081031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:07:38.081031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:38.084081 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd094ee-d250-408f-8a1d-77b2e377a5a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=f6d44c613f524fe1804d6c9b65fe97ca, role_id=afaf4273358148d3b0b9cac8dbeeba97)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:38.084698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 292/584] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:38.085596 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd094ee-d250-408f-8a1d-77b2e377a5a5 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:38.085714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf36f4-8c79-4b9d-b521-3355290149c5 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:38.157871 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:38.161882 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 293/585] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:07:37 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/f6d44c613f524fe1804d6c9b65fe97ca/roles/afaf4273358148d3b0b9cac8dbeeba97 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:07:38.162498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd094ee-d250-408f-8a1d-77b2e377a5a5 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:38.231402 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf36f4-8c79-4b9d-b521-3355290149c5 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:38.278695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f464a9d-c847-48d2-8807-046cadb5aba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:38.280208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f464a9d-c847-48d2-8807-046cadb5aba9 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:38.280955 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f464a9d-c847-48d2-8807-046cadb5aba9 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:38.281488 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f464a9d-c847-48d2-8807-046cadb5aba9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:38.291489 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f464a9d-c847-48d2-8807-046cadb5aba9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:38.291866 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd094ee-d250-408f-8a1d-77b2e377a5a5 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:38.294213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f464a9d-c847-48d2-8807-046cadb5aba9 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-3f464a9d-c847-48d2-8807-046cadb5aba9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:38.314245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:38.315940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:38.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:38.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:38.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:38.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:38.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:38.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:38.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:38.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:38.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:07:38.317107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:38.320462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 293/586] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:38.321917 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f464a9d-c847-48d2-8807-046cadb5aba9 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:38.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e01508-2f04-4943-b873-a9cab3fd8a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:38.339337 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e01508-2f04-4943-b873-a9cab3fd8a27 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:38.339620 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e01508-2f04-4943-b873-a9cab3fd8a27 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:38.339897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e01508-2f04-4943-b873-a9cab3fd8a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:38.353378 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e01508-2f04-4943-b873-a9cab3fd8a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:38.355300 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e01508-2f04-4943-b873-a9cab3fd8a27 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:38.366222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-52e01508-2f04-4943-b873-a9cab3fd8a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:38.377972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:38.381049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:38.382107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:38.382107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:38.382107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:38.382107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:38.382107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 294/587] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:38.382107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e01508-2f04-4943-b873-a9cab3fd8a27 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:38.480321 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f464a9d-c847-48d2-8807-046cadb5aba9 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:38.511040 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:38.515077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e01508-2f04-4943-b873-a9cab3fd8a27 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:38.527132 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ddcdc-a011-4c71-bc7c-5d53f704201f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:38.528488 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ddcdc-a011-4c71-bc7c-5d53f704201f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:38.528975 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ddcdc-a011-4c71-bc7c-5d53f704201f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:38.529417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ddcdc-a011-4c71-bc7c-5d53f704201f None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:38.531682 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ddcdc-a011-4c71-bc7c-5d53f704201f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:38.532220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ddcdc-a011-4c71-bc7c-5d53f704201f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:38.534443 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ddcdc-a011-4c71-bc7c-5d53f704201f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:38.558639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce1616-5538-4811-b64c-032373e47a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:38.559033 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 294/588] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:07:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:38.559826 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1616-5538-4811-b64c-032373e47a5a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:38.560114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ddcdc-a011-4c71-bc7c-5d53f704201f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:38.560483 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1616-5538-4811-b64c-032373e47a5a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:38.560949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce1616-5538-4811-b64c-032373e47a5a None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:38.562155 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce1616-5538-4811-b64c-032373e47a5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:38.562644 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce1616-5538-4811-b64c-032373e47a5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:38.565758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce1616-5538-4811-b64c-032373e47a5a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:38.595827 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 295/589] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:07:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:38.595827 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce1616-5538-4811-b64c-032373e47a5a None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:38.664837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:38.706844 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ddcdc-a011-4c71-bc7c-5d53f704201f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:38.729998 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce1616-5538-4811-b64c-032373e47a5a None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:38.755768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d176a-a166-46ba-b470-9d88849bdefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:38.756609 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d176a-a166-46ba-b470-9d88849bdefe None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:38.756968 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d176a-a166-46ba-b470-9d88849bdefe None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:38.757370 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d176a-a166-46ba-b470-9d88849bdefe None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:38.758556 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d176a-a166-46ba-b470-9d88849bdefe None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:38.758941 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d176a-a166-46ba-b470-9d88849bdefe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:38.760556 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d176a-a166-46ba-b470-9d88849bdefe None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:38.778874 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 295/590] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:38.778874 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d176a-a166-46ba-b470-9d88849bdefe None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:38.784798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748b293-3d58-4f7d-a07c-caa978c6880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:38.785895 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748b293-3d58-4f7d-a07c-caa978c6880d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:38.786386 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748b293-3d58-4f7d-a07c-caa978c6880d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:38.786871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748b293-3d58-4f7d-a07c-caa978c6880d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:38.805048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748b293-3d58-4f7d-a07c-caa978c6880d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:38.807493 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748b293-3d58-4f7d-a07c-caa978c6880d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:38.831669 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-c748b293-3d58-4f7d-a07c-caa978c6880d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:38.832260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:38.833345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:38.834357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:38.834357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:38.834357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:38.834357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:38.834608 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 296/591] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:38.835968 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748b293-3d58-4f7d-a07c-caa978c6880d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:38.913279 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d176a-a166-46ba-b470-9d88849bdefe None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:38.952680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bc41b-2906-4a59-927d-7ea3cf40d3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:38.953460 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bc41b-2906-4a59-927d-7ea3cf40d3f7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:38.953743 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bc41b-2906-4a59-927d-7ea3cf40d3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:38.954019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bc41b-2906-4a59-927d-7ea3cf40d3f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:38.966401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bc41b-2906-4a59-927d-7ea3cf40d3f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:38.967129 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748b293-3d58-4f7d-a07c-caa978c6880d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:38.967903 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bc41b-2906-4a59-927d-7ea3cf40d3f7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:38.986711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-5a3bc41b-2906-4a59-927d-7ea3cf40d3f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:38.987921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:38.988995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:38.990070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:38.990070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:38.990070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:38.990070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:38.996952 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 296/592] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:38.998160 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bc41b-2906-4a59-927d-7ea3cf40d3f7 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.016035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37790d66-bd2b-41b7-a9f4-8d920e199517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.016805 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37790d66-bd2b-41b7-a9f4-8d920e199517 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.017093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37790d66-bd2b-41b7-a9f4-8d920e199517 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.017372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37790d66-bd2b-41b7-a9f4-8d920e199517 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.025666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37790d66-bd2b-41b7-a9f4-8d920e199517 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.027454 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37790d66-bd2b-41b7-a9f4-8d920e199517 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-37790d66-bd2b-41b7-a9f4-8d920e199517 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:39.047080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:39.048214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:39.049360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:39.049360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:39.049360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:39.049360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:39.049360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:39.049360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:39.049360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:39.049360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:39.049360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:07:39.049360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:39.055699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 297/593] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:39.055699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37790d66-bd2b-41b7-a9f4-8d920e199517 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.131994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:39.144350 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bc41b-2906-4a59-927d-7ea3cf40d3f7 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.181518 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25643883-fea0-4116-aa64-07639399be3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.182362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643883-fea0-4116-aa64-07639399be3b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.182750 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643883-fea0-4116-aa64-07639399be3b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.183133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25643883-fea0-4116-aa64-07639399be3b None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.184047 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25643883-fea0-4116-aa64-07639399be3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:39.184410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25643883-fea0-4116-aa64-07639399be3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.185899 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25643883-fea0-4116-aa64-07639399be3b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.190777 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37790d66-bd2b-41b7-a9f4-8d920e199517 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.203934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 297/594] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:39.205142 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25643883-fea0-4116-aa64-07639399be3b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.230164 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015a3c8-118b-441e-ac16-9a8759a03e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.231296 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015a3c8-118b-441e-ac16-9a8759a03e61 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.231780 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015a3c8-118b-441e-ac16-9a8759a03e61 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.232311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015a3c8-118b-441e-ac16-9a8759a03e61 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.233852 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015a3c8-118b-441e-ac16-9a8759a03e61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:39.234383 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015a3c8-118b-441e-ac16-9a8759a03e61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.236181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015a3c8-118b-441e-ac16-9a8759a03e61 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.249401 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 298/595] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:39.250546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015a3c8-118b-441e-ac16-9a8759a03e61 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.285885 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:39.319021 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25643883-fea0-4116-aa64-07639399be3b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.356474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916b68-687b-46d2-9c4a-ce5006c6ac70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.357278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916b68-687b-46d2-9c4a-ce5006c6ac70 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.357667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916b68-687b-46d2-9c4a-ce5006c6ac70 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.357979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12916b68-687b-46d2-9c4a-ce5006c6ac70 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.360110 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12916b68-687b-46d2-9c4a-ce5006c6ac70 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:39.360256 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12916b68-687b-46d2-9c4a-ce5006c6ac70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.361765 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12916b68-687b-46d2-9c4a-ce5006c6ac70 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.370410 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015a3c8-118b-441e-ac16-9a8759a03e61 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.387629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 298/596] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:39.388735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916b68-687b-46d2-9c4a-ce5006c6ac70 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.402780 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:39.414228 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee351e-c274-4de8-b36d-21d165f2e12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.418400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee351e-c274-4de8-b36d-21d165f2e12f None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.418400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee351e-c274-4de8-b36d-21d165f2e12f None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.418400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee351e-c274-4de8-b36d-21d165f2e12f None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.424972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee351e-c274-4de8-b36d-21d165f2e12f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.426700 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee351e-c274-4de8-b36d-21d165f2e12f None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-f9ee351e-c274-4de8-b36d-21d165f2e12f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:39.442492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:39.443600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:39.445221 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:39.445221 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:39.445221 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:39.445221 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:39.445221 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:39.445221 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:39.445221 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:39.445221 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:39.448022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 299/597] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:39.449201 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee351e-c274-4de8-b36d-21d165f2e12f None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.505191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12916b68-687b-46d2-9c4a-ce5006c6ac70 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.542911 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d101ef-8a83-4420-9520-563683ce02cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.543700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d101ef-8a83-4420-9520-563683ce02cd None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.543991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d101ef-8a83-4420-9520-563683ce02cd None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.544383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d101ef-8a83-4420-9520-563683ce02cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.552104 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d101ef-8a83-4420-9520-563683ce02cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.553814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d101ef-8a83-4420-9520-563683ce02cd None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.558934 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-76d101ef-8a83-4420-9520-563683ce02cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:39.568909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:39.578307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:39.579374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:39.579374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:39.579374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:39.579374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:39.579374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:39.579374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:39.579374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:39.579374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:39.579374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 299/598] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:39.579938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee351e-c274-4de8-b36d-21d165f2e12f None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.580023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d101ef-8a83-4420-9520-563683ce02cd None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.611372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2a760-7445-42ca-bedb-77c28515cd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.612191 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2a760-7445-42ca-bedb-77c28515cd16 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.612512 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2a760-7445-42ca-bedb-77c28515cd16 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.612796 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2a760-7445-42ca-bedb-77c28515cd16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.624237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2a760-7445-42ca-bedb-77c28515cd16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.625782 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2a760-7445-42ca-bedb-77c28515cd16 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-62a2a760-7445-42ca-bedb-77c28515cd16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:39.646054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:39.647128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:39.648146 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:39.648146 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:39.648146 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:39.648146 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:39.648146 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 300/599] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:39.649079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2a760-7445-42ca-bedb-77c28515cd16 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.657203 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:39.690117 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d101ef-8a83-4420-9520-563683ce02cd None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.728987 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec575f-2461-4617-9f7b-5a816179301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.729712 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec575f-2461-4617-9f7b-5a816179301e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.729961 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec575f-2461-4617-9f7b-5a816179301e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.730239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec575f-2461-4617-9f7b-5a816179301e None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.731059 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec575f-2461-4617-9f7b-5a816179301e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:39.731374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec575f-2461-4617-9f7b-5a816179301e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.732893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec575f-2461-4617-9f7b-5a816179301e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.751504 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 300/600] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:39.752563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec575f-2461-4617-9f7b-5a816179301e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.767998 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2a760-7445-42ca-bedb-77c28515cd16 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.807987 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879db7c0-9bf6-45f6-84c6-50398505f441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.808969 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879db7c0-9bf6-45f6-84c6-50398505f441 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.809420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879db7c0-9bf6-45f6-84c6-50398505f441 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.810103 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879db7c0-9bf6-45f6-84c6-50398505f441 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.811120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879db7c0-9bf6-45f6-84c6-50398505f441 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:39.811644 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879db7c0-9bf6-45f6-84c6-50398505f441 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.813457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879db7c0-9bf6-45f6-84c6-50398505f441 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.829366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 301/601] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:39.830715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879db7c0-9bf6-45f6-84c6-50398505f441 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.834202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:39.871917 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec575f-2461-4617-9f7b-5a816179301e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.904150 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:39.910603 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71b9c5-4da0-4e93-bbfd-a8846193872c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.911277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71b9c5-4da0-4e93-bbfd-a8846193872c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.911561 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71b9c5-4da0-4e93-bbfd-a8846193872c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.911839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71b9c5-4da0-4e93-bbfd-a8846193872c None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:39.912601 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71b9c5-4da0-4e93-bbfd-a8846193872c None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:39.912973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71b9c5-4da0-4e93-bbfd-a8846193872c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:39.920109 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71b9c5-4da0-4e93-bbfd-a8846193872c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:39.931864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 301/602] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:39.932875 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71b9c5-4da0-4e93-bbfd-a8846193872c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:39.956280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879db7c0-9bf6-45f6-84c6-50398505f441 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:39.995128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d148a-39e5-412b-afd5-f810a74a1542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:39.996485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d148a-39e5-412b-afd5-f810a74a1542 None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:39.996954 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d148a-39e5-412b-afd5-f810a74a1542 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:39.997398 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d148a-39e5-412b-afd5-f810a74a1542 None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/4cebf4a3afdc4124aef25bed824c1dec/roles/afaf4273358148d3b0b9cac8dbeeba97` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.017808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d148a-39e5-412b-afd5-f810a74a1542 None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=4cebf4a3afdc4124aef25bed824c1dec, role_id=afaf4273358148d3b0b9cac8dbeeba97)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:40.019672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d148a-39e5-412b-afd5-f810a74a1542 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:40.042782 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71b9c5-4da0-4e93-bbfd-a8846193872c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:40.080673 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fad12-5c58-4f03-ab76-fb513717da9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:40.081248 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fad12-5c58-4f03-ab76-fb513717da9e None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.081529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fad12-5c58-4f03-ab76-fb513717da9e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.081805 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fad12-5c58-4f03-ab76-fb513717da9e None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/c8710592e5a84bb4b885b050e23549dc/roles/afaf4273358148d3b0b9cac8dbeeba97` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.097739 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fad12-5c58-4f03-ab76-fb513717da9e None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=c8710592e5a84bb4b885b050e23549dc, role_id=afaf4273358148d3b0b9cac8dbeeba97)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:40.099275 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5fad12-5c58-4f03-ab76-fb513717da9e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:40.106587 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 302/603] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:07:39 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/4cebf4a3afdc4124aef25bed824c1dec/roles/afaf4273358148d3b0b9cac8dbeeba97 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:07:40.107547 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d148a-39e5-412b-afd5-f810a74a1542 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:40.176920 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:40.188488 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 302/604] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:07:39 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/c8710592e5a84bb4b885b050e23549dc/roles/afaf4273358148d3b0b9cac8dbeeba97 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:07:40.191358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c4c8c-d841-4051-99de-ac9aafedc76d None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.191957 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c4c8c-d841-4051-99de-ac9aafedc76d None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.192530 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c4c8c-d841-4051-99de-ac9aafedc76d None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.195110 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 303/605] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:40.201518 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d158507-7cda-4ca5-b2a7-8acc113c5625 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.201870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d158507-7cda-4ca5-b2a7-8acc113c5625 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.202085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d158507-7cda-4ca5-b2a7-8acc113c5625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.229044 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-1d158507-7cda-4ca5-b2a7-8acc113c5625 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:40.230229 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-1d158507-7cda-4ca5-b2a7-8acc113c5625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:40.269180 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d148a-39e5-412b-afd5-f810a74a1542 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:40.277297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:40.321963 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 304/606] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:40.326493 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da1ba0-5633-4d38-baf9-477a00ed7b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:40.328865 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da1ba0-5633-4d38-baf9-477a00ed7b1b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.329442 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da1ba0-5633-4d38-baf9-477a00ed7b1b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.330003 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da1ba0-5633-4d38-baf9-477a00ed7b1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.338913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da1ba0-5633-4d38-baf9-477a00ed7b1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:40.340166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d158507-7cda-4ca5-b2a7-8acc113c5625 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:40.341150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da1ba0-5633-4d38-baf9-477a00ed7b1b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-08da1ba0-5633-4d38-baf9-477a00ed7b1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:40.364387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:40.365486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:40.366530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:40.366530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:40.366530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:40.366530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:40.366530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:40.366530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:40.366530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:40.366530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:40.370630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 303/607] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:40.377478 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da1ba0-5633-4d38-baf9-477a00ed7b1b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:40.481340 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d158507-7cda-4ca5-b2a7-8acc113c5625 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:40.535969 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90e209-993f-4e6b-9103-d3b36b86cf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:40.537101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90e209-993f-4e6b-9103-d3b36b86cf90 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.537438 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90e209-993f-4e6b-9103-d3b36b86cf90 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.537728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90e209-993f-4e6b-9103-d3b36b86cf90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.545178 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da1ba0-5633-4d38-baf9-477a00ed7b1b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:40.550464 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90e209-993f-4e6b-9103-d3b36b86cf90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-9d90e209-993f-4e6b-9103-d3b36b86cf90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:40.558283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:40.559449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:40.560474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 305/608] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:40.564522 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:40.569352 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90e209-993f-4e6b-9103-d3b36b86cf90 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:40.593727 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b05366-a746-4a8c-85d9-20088b33c015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:40.595072 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b05366-a746-4a8c-85d9-20088b33c015 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.595581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b05366-a746-4a8c-85d9-20088b33c015 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.596104 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b05366-a746-4a8c-85d9-20088b33c015 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.597239 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b05366-a746-4a8c-85d9-20088b33c015 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:40.597771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b05366-a746-4a8c-85d9-20088b33c015 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:40.599721 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b05366-a746-4a8c-85d9-20088b33c015 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:40.617371 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 304/609] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:40.619268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:40.623158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19da99-b501-4c43-8543-7a74e2d46fbb None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.623449 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19da99-b501-4c43-8543-7a74e2d46fbb None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.623732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19da99-b501-4c43-8543-7a74e2d46fbb None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.629866 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 305/610] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:40.630758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19da99-b501-4c43-8543-7a74e2d46fbb None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:40.722120 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90e209-993f-4e6b-9103-d3b36b86cf90 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:40.745956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19da99-b501-4c43-8543-7a74e2d46fbb None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:40.769314 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f35ea-9827-4afd-86e2-9998432f9c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:40.770938 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f35ea-9827-4afd-86e2-9998432f9c6f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.771441 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f35ea-9827-4afd-86e2-9998432f9c6f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.772177 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f35ea-9827-4afd-86e2-9998432f9c6f None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.773150 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f35ea-9827-4afd-86e2-9998432f9c6f None admin] RBAC: Adding query filter params (name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:40.773667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f35ea-9827-4afd-86e2-9998432f9c6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:40.776243 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f35ea-9827-4afd-86e2-9998432f9c6f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:40.787775 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc48fb-dfb1-404b-b7bb-a2ca87e0263d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:40.788831 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc48fb-dfb1-404b-b7bb-a2ca87e0263d None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.789237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc48fb-dfb1-404b-b7bb-a2ca87e0263d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.794283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc48fb-dfb1-404b-b7bb-a2ca87e0263d None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/d7917b3c79bd42cea17ec8eba5a265ed/roles/afaf4273358148d3b0b9cac8dbeeba97` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.797986 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 306/611] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:40.798338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f55b-9885-4786-9e8e-544e9abd61e1 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:40.798820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f55b-9885-4786-9e8e-544e9abd61e1 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:40.799284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1f55b-9885-4786-9e8e-544e9abd61e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:40.819195 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc48fb-dfb1-404b-b7bb-a2ca87e0263d None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=d7917b3c79bd42cea17ec8eba5a265ed, role_id=afaf4273358148d3b0b9cac8dbeeba97)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:40.819794 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-e2a1f55b-9885-4786-9e8e-544e9abd61e1 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:40.821213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-e2a1f55b-9885-4786-9e8e-544e9abd61e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:40.822254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc48fb-dfb1-404b-b7bb-a2ca87e0263d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:40.924645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 306/612] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:07:40 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/d7917b3c79bd42cea17ec8eba5a265ed/roles/afaf4273358148d3b0b9cac8dbeeba97 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:07:40.926398 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc48fb-dfb1-404b-b7bb-a2ca87e0263d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:40.938712 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 307/613] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:40.964069 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:41.022007 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f55b-9885-4786-9e8e-544e9abd61e1 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:41.048825 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc48fb-dfb1-404b-b7bb-a2ca87e0263d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:41.071921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:41.089797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2c5b4-39f0-482c-a77f-4065b370527d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:41.091790 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c5b4-39f0-482c-a77f-4065b370527d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:41.092267 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c5b4-39f0-482c-a77f-4065b370527d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:41.092818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c5b4-39f0-482c-a77f-4065b370527d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:41.105837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2c5b4-39f0-482c-a77f-4065b370527d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:41.107759 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d2c5b4-39f0-482c-a77f-4065b370527d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-95d2c5b4-39f0-482c-a77f-4065b370527d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:41.128837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:41.129938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:41.130953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:41.130953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:41.130953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:41.130953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:41.133531 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 307/614] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:41.139477 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2c5b4-39f0-482c-a77f-4065b370527d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:41.241021 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:41.254011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2c5b4-39f0-482c-a77f-4065b370527d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:41.292429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de27428-c4f2-487e-92d6-e034aef084c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:41.293528 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de27428-c4f2-487e-92d6-e034aef084c9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:41.294073 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de27428-c4f2-487e-92d6-e034aef084c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:41.294572 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de27428-c4f2-487e-92d6-e034aef084c9 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:41.295544 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de27428-c4f2-487e-92d6-e034aef084c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:41.296907 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de27428-c4f2-487e-92d6-e034aef084c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:41.298545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de27428-c4f2-487e-92d6-e034aef084c9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:41.312919 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 308/615] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:41.318874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de27428-c4f2-487e-92d6-e034aef084c9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:41.338404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1f55b-9885-4786-9e8e-544e9abd61e1 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:41.417377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68f5fe-4c15-4400-a352-2ac472084802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:41.418698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68f5fe-4c15-4400-a352-2ac472084802 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:41.419188 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68f5fe-4c15-4400-a352-2ac472084802 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:41.419673 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba68f5fe-4c15-4400-a352-2ac472084802 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:41.434844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de27428-c4f2-487e-92d6-e034aef084c9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:41.441446 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba68f5fe-4c15-4400-a352-2ac472084802 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-ba68f5fe-4c15-4400-a352-2ac472084802 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:41.458594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:41.459752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:41.460738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:41.468601 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 308/616] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:41.471023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68f5fe-4c15-4400-a352-2ac472084802 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:41.477536 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c230a-49bc-4c99-9c45-32be671dcd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:41.478690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c230a-49bc-4c99-9c45-32be671dcd5e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:41.479206 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c230a-49bc-4c99-9c45-32be671dcd5e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:41.479706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c230a-49bc-4c99-9c45-32be671dcd5e None admin] PATH_INFO: `/v3/users/swift` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:41.488949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c230a-49bc-4c99-9c45-32be671dcd5e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:41.490905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c230a-49bc-4c99-9c45-32be671dcd5e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-4c4c230a-49bc-4c99-9c45-32be671dcd5e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:41.506593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:41.507714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:41.508817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:41.508817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:41.508817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:41.508817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:41.508817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:41.508817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:41.508817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:41.508817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:41.508817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:07:41.508817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:41.511533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 309/617] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:07:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:41.516213 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:41.517599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c230a-49bc-4c99-9c45-32be671dcd5e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:41.634472 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c230a-49bc-4c99-9c45-32be671dcd5e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:41.674646 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a0a6c-bfa0-4cb2-ab83-5ce700110095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:41.675831 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a0a6c-bfa0-4cb2-ab83-5ce700110095 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:41.676364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a0a6c-bfa0-4cb2-ab83-5ce700110095 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:41.676898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a0a6c-bfa0-4cb2-ab83-5ce700110095 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:41.677998 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a0a6c-bfa0-4cb2-ab83-5ce700110095 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:41.678554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a0a6c-bfa0-4cb2-ab83-5ce700110095 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:41.680485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a0a6c-bfa0-4cb2-ab83-5ce700110095 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:41.701356 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 310/618] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:07:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:41.707366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a0a6c-bfa0-4cb2-ab83-5ce700110095 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:41.744106 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:41.783123 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba68f5fe-4c15-4400-a352-2ac472084802 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:41.803547 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:41.853397 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2a362-d0d5-4213-a53d-4bdf79d24fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:41.854099 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2a362-d0d5-4213-a53d-4bdf79d24fa5 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:41.854437 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2a362-d0d5-4213-a53d-4bdf79d24fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:41.854716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2a362-d0d5-4213-a53d-4bdf79d24fa5 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:41.855657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2a362-d0d5-4213-a53d-4bdf79d24fa5 None admin] RBAC: Adding query filter params (name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:41.855848 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2a362-d0d5-4213-a53d-4bdf79d24fa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:41.857235 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2a362-d0d5-4213-a53d-4bdf79d24fa5 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:41.869042 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a0a6c-bfa0-4cb2-ab83-5ce700110095 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:41.872359 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 309/619] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:41.879992 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2a362-d0d5-4213-a53d-4bdf79d24fa5 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:41.910871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462889c7-9046-4aa8-b0ca-6dfdad4f6290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:41.912074 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462889c7-9046-4aa8-b0ca-6dfdad4f6290 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:41.912608 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462889c7-9046-4aa8-b0ca-6dfdad4f6290 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:41.913147 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462889c7-9046-4aa8-b0ca-6dfdad4f6290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:41.925748 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462889c7-9046-4aa8-b0ca-6dfdad4f6290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:41.927556 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462889c7-9046-4aa8-b0ca-6dfdad4f6290 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-462889c7-9046-4aa8-b0ca-6dfdad4f6290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:41.947689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:41.948788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:41.949808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:41.949808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:41.949808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:41.949808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:41.952153 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 311/620] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:41.957940 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462889c7-9046-4aa8-b0ca-6dfdad4f6290 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:42.070599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462889c7-9046-4aa8-b0ca-6dfdad4f6290 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:42.073894 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:42.111934 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72e84f-6435-4b54-a666-8c1e4513a1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:42.114113 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72e84f-6435-4b54-a666-8c1e4513a1b8 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:42.114642 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72e84f-6435-4b54-a666-8c1e4513a1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:42.115149 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72e84f-6435-4b54-a666-8c1e4513a1b8 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:42.116101 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72e84f-6435-4b54-a666-8c1e4513a1b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:42.116749 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72e84f-6435-4b54-a666-8c1e4513a1b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:42.118411 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72e84f-6435-4b54-a666-8c1e4513a1b8 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:42.131773 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 312/621] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:42.137572 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72e84f-6435-4b54-a666-8c1e4513a1b8 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:42.200451 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2a362-d0d5-4213-a53d-4bdf79d24fa5 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:42.252160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72e84f-6435-4b54-a666-8c1e4513a1b8 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:42.291409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be3a07-8dcf-4779-b1f0-7478860d6115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:42.292654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be3a07-8dcf-4779-b1f0-7478860d6115 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:42.293206 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be3a07-8dcf-4779-b1f0-7478860d6115 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:42.293711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be3a07-8dcf-4779-b1f0-7478860d6115 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:42.302035 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52276b35-35c9-41fa-9abb-9a8bb4316b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:42.302857 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52276b35-35c9-41fa-9abb-9a8bb4316b1f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:42.303144 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52276b35-35c9-41fa-9abb-9a8bb4316b1f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:42.303385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be3a07-8dcf-4779-b1f0-7478860d6115 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:42.303498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52276b35-35c9-41fa-9abb-9a8bb4316b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:42.305251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be3a07-8dcf-4779-b1f0-7478860d6115 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-e2be3a07-8dcf-4779-b1f0-7478860d6115 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:42.322533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:42.323591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:42.324649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:42.324649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:42.324649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:42.324649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:42.324649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:42.324649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:42.324649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:42.324649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:42.327295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 313/622] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:42.333224 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be3a07-8dcf-4779-b1f0-7478860d6115 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:42.333539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52276b35-35c9-41fa-9abb-9a8bb4316b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:42.335167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52276b35-35c9-41fa-9abb-9a8bb4316b1f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:42.345560 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:42.363336 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-52276b35-35c9-41fa-9abb-9a8bb4316b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:42.379291 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:42.380409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:42.381448 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:42.381448 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:42.381448 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:42.381448 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:42.381448 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 310/623] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 501 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:42.388561 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52276b35-35c9-41fa-9abb-9a8bb4316b1f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:42.448167 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be3a07-8dcf-4779-b1f0-7478860d6115 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:42.486928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38ef80-cfd4-45c6-a102-a725242cd243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:42.487957 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38ef80-cfd4-45c6-a102-a725242cd243 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:42.488530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38ef80-cfd4-45c6-a102-a725242cd243 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:42.489024 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38ef80-cfd4-45c6-a102-a725242cd243 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:42.490052 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38ef80-cfd4-45c6-a102-a725242cd243 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:42.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38ef80-cfd4-45c6-a102-a725242cd243 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:42.492370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38ef80-cfd4-45c6-a102-a725242cd243 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:42.507079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 314/624] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:42.513146 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38ef80-cfd4-45c6-a102-a725242cd243 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:42.606001 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:42.627105 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38ef80-cfd4-45c6-a102-a725242cd243 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:42.668935 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52276b35-35c9-41fa-9abb-9a8bb4316b1f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:42.693494 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62297a-d1b2-4c07-aa85-005953db61ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:42.694800 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62297a-d1b2-4c07-aa85-005953db61ca None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:42.695335 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62297a-d1b2-4c07-aa85-005953db61ca None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:42.695830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62297a-d1b2-4c07-aa85-005953db61ca None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/7777c807131c4d9886c6aeff5f756711/roles/afaf4273358148d3b0b9cac8dbeeba97` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:42.727515 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62297a-d1b2-4c07-aa85-005953db61ca None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=7777c807131c4d9886c6aeff5f756711, role_id=afaf4273358148d3b0b9cac8dbeeba97)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:42.730160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b62297a-d1b2-4c07-aa85-005953db61ca None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:42.737991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9dfe5c-10e5-46e8-876a-829d5628923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:42.739155 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9dfe5c-10e5-46e8-876a-829d5628923f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:42.740185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9dfe5c-10e5-46e8-876a-829d5628923f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:42.740702 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9dfe5c-10e5-46e8-876a-829d5628923f None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:42.741656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9dfe5c-10e5-46e8-876a-829d5628923f None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:42.742179 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9dfe5c-10e5-46e8-876a-829d5628923f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:42.743902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9dfe5c-10e5-46e8-876a-829d5628923f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:42.759812 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 311/625] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:42.768366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9dfe5c-10e5-46e8-876a-829d5628923f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:42.826493 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 315/626] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:07:42 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/7777c807131c4d9886c6aeff5f756711/roles/afaf4273358148d3b0b9cac8dbeeba97 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:07:42.872949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:42.944028 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9dfe5c-10e5-46e8-876a-829d5628923f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:43.008188 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfbe2a-0dc4-45ea-8b5f-f2d214a5e249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:43.009532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfbe2a-0dc4-45ea-8b5f-f2d214a5e249 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:43.010004 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfbe2a-0dc4-45ea-8b5f-f2d214a5e249 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:43.010440 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfbe2a-0dc4-45ea-8b5f-f2d214a5e249 None admin] PATH_INFO: `/v3/users/nova` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:43.024415 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfbe2a-0dc4-45ea-8b5f-f2d214a5e249 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:43.026900 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfbe2a-0dc4-45ea-8b5f-f2d214a5e249 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-0cdfbe2a-0dc4-45ea-8b5f-f2d214a5e249 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:43.054716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:43.056088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:43.057521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:43.057521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:43.057521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:43.057521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:43.057521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:43.057521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:43.057521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:43.057521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:43.057521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:07:43.057521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:43.057521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 312/627] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:07:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:43.063664 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62297a-d1b2-4c07-aa85-005953db61ca None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:43.218257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62297a-d1b2-4c07-aa85-005953db61ca None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:43.255614 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7580334-7a09-4642-8276-9053452f4009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:43.256733 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7580334-7a09-4642-8276-9053452f4009 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:43.257267 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7580334-7a09-4642-8276-9053452f4009 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:43.257768 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7580334-7a09-4642-8276-9053452f4009 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:43.258887 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7580334-7a09-4642-8276-9053452f4009 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:43.259424 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7580334-7a09-4642-8276-9053452f4009 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:43.261177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7580334-7a09-4642-8276-9053452f4009 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:43.263225 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:43.283181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 316/628] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:07:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:43.288829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfbe2a-0dc4-45ea-8b5f-f2d214a5e249 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:43.437672 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfbe2a-0dc4-45ea-8b5f-f2d214a5e249 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:43.468186 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:43.487326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8446d7-a2f1-42ef-80b0-4170ab4ab955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:43.487904 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8446d7-a2f1-42ef-80b0-4170ab4ab955 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:43.488185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8446d7-a2f1-42ef-80b0-4170ab4ab955 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:43.488490 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8446d7-a2f1-42ef-80b0-4170ab4ab955 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:43.585735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8446d7-a2f1-42ef-80b0-4170ab4ab955 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:43.587819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8446d7-a2f1-42ef-80b0-4170ab4ab955 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:43.611031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041b99d-5fc2-4f7f-b7f9-32d3a2e7a3be None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:43.611597 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041b99d-5fc2-4f7f-b7f9-32d3a2e7a3be None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:43.611597 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041b99d-5fc2-4f7f-b7f9-32d3a2e7a3be None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:43.617600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 317/629] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:43.625973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5732b-b007-4239-a26c-d826f7851d89 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:43.626309 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5732b-b007-4239-a26c-d826f7851d89 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:43.626538 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5732b-b007-4239-a26c-d826f7851d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:43.661561 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-31d5732b-b007-4239-a26c-d826f7851d89 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:43.662799 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-31d5732b-b007-4239-a26c-d826f7851d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:43.708672 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 313/630] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:43.714954 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8446d7-a2f1-42ef-80b0-4170ab4ab955 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:43.752823 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:43.794933 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 318/631] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:43.834784 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5732b-b007-4239-a26c-d826f7851d89 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:43.869770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8446d7-a2f1-42ef-80b0-4170ab4ab955 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:43.922957 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e00298-9a31-4610-8084-472c082bc91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:43.924991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e00298-9a31-4610-8084-472c082bc91a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:43.925495 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e00298-9a31-4610-8084-472c082bc91a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:43.926126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e00298-9a31-4610-8084-472c082bc91a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:43.941256 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e00298-9a31-4610-8084-472c082bc91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:43.943140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e00298-9a31-4610-8084-472c082bc91a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-06e00298-9a31-4610-8084-472c082bc91a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:43.968713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:43.969864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:43.970886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:43.970886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:43.970886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:43.970886 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:43.973202 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 314/632] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:43.973990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5732b-b007-4239-a26c-d826f7851d89 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:43.974943 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727933dd-8fdd-4d89-baf8-55115c266ed7 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:43.975403 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727933dd-8fdd-4d89-baf8-55115c266ed7 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:43.975836 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727933dd-8fdd-4d89-baf8-55115c266ed7 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:43.984098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 315/633] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:43.985296 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727933dd-8fdd-4d89-baf8-55115c266ed7 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.001020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:44.016476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd67852-69a2-4ddd-abfc-8099dbda7d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.016476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd67852-69a2-4ddd-abfc-8099dbda7d09 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.016476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd67852-69a2-4ddd-abfc-8099dbda7d09 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.016476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd67852-69a2-4ddd-abfc-8099dbda7d09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.033515 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd67852-69a2-4ddd-abfc-8099dbda7d09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-0dd67852-69a2-4ddd-abfc-8099dbda7d09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:44.042812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:44.043956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:44.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:44.052527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 319/634] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:44.054316 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bbdbb-b2b5-40ba-8941-1640960e95a0 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.054837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bbdbb-b2b5-40ba-8941-1640960e95a0 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.055365 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070bbdbb-b2b5-40ba-8941-1640960e95a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.074801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-070bbdbb-b2b5-40ba-8941-1640960e95a0 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:44.076545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-070bbdbb-b2b5-40ba-8941-1640960e95a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:44.083521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:44.141266 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727933dd-8fdd-4d89-baf8-55115c266ed7 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.176623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 320/635] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:44.177885 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bbdbb-b2b5-40ba-8941-1640960e95a0 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.192375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364900-ee7e-4c98-9522-9f908641af2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.192375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364900-ee7e-4c98-9522-9f908641af2f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.192375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364900-ee7e-4c98-9522-9f908641af2f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.192375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364900-ee7e-4c98-9522-9f908641af2f None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.192818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42364900-ee7e-4c98-9522-9f908641af2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:44.192818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42364900-ee7e-4c98-9522-9f908641af2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.193844 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42364900-ee7e-4c98-9522-9f908641af2f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.209300 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 316/636] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:44.214749 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364900-ee7e-4c98-9522-9f908641af2f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.313161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070bbdbb-b2b5-40ba-8941-1640960e95a0 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.330127 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:44.345813 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364900-ee7e-4c98-9522-9f908641af2f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.355597 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093394f-6cca-4538-93ce-c0228d0adec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.356069 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093394f-6cca-4538-93ce-c0228d0adec1 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.356425 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093394f-6cca-4538-93ce-c0228d0adec1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.356685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093394f-6cca-4538-93ce-c0228d0adec1 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.357481 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093394f-6cca-4538-93ce-c0228d0adec1 None admin] RBAC: Adding query filter params (name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:44.357792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093394f-6cca-4538-93ce-c0228d0adec1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.359813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093394f-6cca-4538-93ce-c0228d0adec1 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.369930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 321/637] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:44.370738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093394f-6cca-4538-93ce-c0228d0adec1 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.386421 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:44.396665 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773eb42b-1d20-4eee-86df-719f46840199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.402297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eb42b-1d20-4eee-86df-719f46840199 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.402297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eb42b-1d20-4eee-86df-719f46840199 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.402297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eb42b-1d20-4eee-86df-719f46840199 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.407474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773eb42b-1d20-4eee-86df-719f46840199 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.409145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773eb42b-1d20-4eee-86df-719f46840199 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-773eb42b-1d20-4eee-86df-719f46840199 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:44.427199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:44.428511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:44.429776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:44.429776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:44.429776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:44.429776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:44.429776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:44.429776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:44.429776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:44.429776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:44.431758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 317/638] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:44.433399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e57d04-976f-4c9e-935a-fdd88edac1f9 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.433877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e57d04-976f-4c9e-935a-fdd88edac1f9 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.434308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e57d04-976f-4c9e-935a-fdd88edac1f9 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.435806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 318/639] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:44.436799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e57d04-976f-4c9e-935a-fdd88edac1f9 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.511890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093394f-6cca-4538-93ce-c0228d0adec1 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.553797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ad2d4-da1c-4b9c-9538-3d14bc673d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.554607 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ad2d4-da1c-4b9c-9538-3d14bc673d89 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.555451 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ad2d4-da1c-4b9c-9538-3d14bc673d89 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.555789 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ad2d4-da1c-4b9c-9538-3d14bc673d89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.567650 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ad2d4-da1c-4b9c-9538-3d14bc673d89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:44.569583 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e57d04-976f-4c9e-935a-fdd88edac1f9 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-ce5ad2d4-da1c-4b9c-9538-3d14bc673d89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:44.575315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:44.576615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:44.578924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:44.582005 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 322/640] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:44.582914 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ad2d4-da1c-4b9c-9538-3d14bc673d89 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.613109 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307db19-405f-44a3-bc2f-8ab6bb147be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.614421 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307db19-405f-44a3-bc2f-8ab6bb147be0 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.615250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307db19-405f-44a3-bc2f-8ab6bb147be0 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.615989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307db19-405f-44a3-bc2f-8ab6bb147be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.617876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:44.629289 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307db19-405f-44a3-bc2f-8ab6bb147be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.630820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307db19-405f-44a3-bc2f-8ab6bb147be0 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-d307db19-405f-44a3-bc2f-8ab6bb147be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:44.653324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:44.654414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:44.660810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:44.660810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:44.660810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:44.660810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:44.660810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 319/641] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:44.660810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafb7e7-f12b-4b71-82b6-c385477d74ea None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.660810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafb7e7-f12b-4b71-82b6-c385477d74ea None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.660810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fafb7e7-f12b-4b71-82b6-c385477d74ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.682904 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-2fafb7e7-f12b-4b71-82b6-c385477d74ea None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:44.684936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-2fafb7e7-f12b-4b71-82b6-c385477d74ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:44.691391 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ad2d4-da1c-4b9c-9538-3d14bc673d89 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.726383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:44.729356 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bba54c-0054-4b01-9168-786a3e8ee35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.730028 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bba54c-0054-4b01-9168-786a3e8ee35e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.730307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bba54c-0054-4b01-9168-786a3e8ee35e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.730613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bba54c-0054-4b01-9168-786a3e8ee35e None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.731368 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bba54c-0054-4b01-9168-786a3e8ee35e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:44.731687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bba54c-0054-4b01-9168-786a3e8ee35e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.733187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bba54c-0054-4b01-9168-786a3e8ee35e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.746813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 323/642] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:44.751697 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bba54c-0054-4b01-9168-786a3e8ee35e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.762723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 320/643] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:44.766233 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafb7e7-f12b-4b71-82b6-c385477d74ea None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.860311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:44.869031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bba54c-0054-4b01-9168-786a3e8ee35e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.877682 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fafb7e7-f12b-4b71-82b6-c385477d74ea None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:44.910818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533035e-012e-42ab-bb25-a4da5b3cb5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.911782 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533035e-012e-42ab-bb25-a4da5b3cb5c7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.912061 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533035e-012e-42ab-bb25-a4da5b3cb5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.912359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533035e-012e-42ab-bb25-a4da5b3cb5c7 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.913283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533035e-012e-42ab-bb25-a4da5b3cb5c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:44.913910 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533035e-012e-42ab-bb25-a4da5b3cb5c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.915592 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533035e-012e-42ab-bb25-a4da5b3cb5c7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.921185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af1217-020f-412b-b25b-2514e6c792c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:44.921185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af1217-020f-412b-b25b-2514e6c792c5 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:44.921185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af1217-020f-412b-b25b-2514e6c792c5 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:44.921185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af1217-020f-412b-b25b-2514e6c792c5 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:44.921863 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af1217-020f-412b-b25b-2514e6c792c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:44.922380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af1217-020f-412b-b25b-2514e6c792c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:44.923788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af1217-020f-412b-b25b-2514e6c792c5 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:44.931190 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 324/644] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:44.931190 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533035e-012e-42ab-bb25-a4da5b3cb5c7 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.935920 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 321/645] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:44.937364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af1217-020f-412b-b25b-2514e6c792c5 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:44.969276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:45.048874 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af1217-020f-412b-b25b-2514e6c792c5 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.053444 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533035e-012e-42ab-bb25-a4da5b3cb5c7 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.093617 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419247d-14ac-446c-ada7-61b7d5a4f55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.094416 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419247d-14ac-446c-ada7-61b7d5a4f55f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.094724 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419247d-14ac-446c-ada7-61b7d5a4f55f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.094962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419247d-14ac-446c-ada7-61b7d5a4f55f None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.095639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac112f5-065c-482c-8ff8-71ce669a8bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.096289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac112f5-065c-482c-8ff8-71ce669a8bfa None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.096619 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac112f5-065c-482c-8ff8-71ce669a8bfa None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.096894 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac112f5-065c-482c-8ff8-71ce669a8bfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.097735 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac112f5-065c-482c-8ff8-71ce669a8bfa None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=f6d44c613f524fe1804d6c9b65fe97ca) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:45.098050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac112f5-065c-482c-8ff8-71ce669a8bfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.099513 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac112f5-065c-482c-8ff8-71ce669a8bfa None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.106826 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 325/646] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/role_assignments?user.id=f6d44c613f524fe1804d6c9b65fe97ca&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:45.107812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac112f5-065c-482c-8ff8-71ce669a8bfa None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.122076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419247d-14ac-446c-ada7-61b7d5a4f55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-1419247d-14ac-446c-ada7-61b7d5a4f55f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:45.133152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:45.134324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:45.135302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:45.136036 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:45.139208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 322/647] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:45.141140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419247d-14ac-446c-ada7-61b7d5a4f55f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.228909 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac112f5-065c-482c-8ff8-71ce669a8bfa None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.252895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:45.268012 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419247d-14ac-446c-ada7-61b7d5a4f55f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.269662 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f36cd-2ead-40d3-b89a-f76cc83dfdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.270611 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f36cd-2ead-40d3-b89a-f76cc83dfdad None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.271004 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f36cd-2ead-40d3-b89a-f76cc83dfdad None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.271422 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f36cd-2ead-40d3-b89a-f76cc83dfdad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.284978 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f36cd-2ead-40d3-b89a-f76cc83dfdad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.286387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f36cd-2ead-40d3-b89a-f76cc83dfdad None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-d49f36cd-2ead-40d3-b89a-f76cc83dfdad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:45.307308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:45.308568 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:45.309607 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:45.309607 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:45.309607 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:45.309607 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:45.309607 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 326/648] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:45.310552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f36cd-2ead-40d3-b89a-f76cc83dfdad None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.319657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab70456-fe29-4bb2-be66-b8f7c63c767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.320822 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab70456-fe29-4bb2-be66-b8f7c63c767c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.321273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab70456-fe29-4bb2-be66-b8f7c63c767c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.321802 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab70456-fe29-4bb2-be66-b8f7c63c767c None admin] PATH_INFO: `/v3/users/glance` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.331972 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab70456-fe29-4bb2-be66-b8f7c63c767c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.334190 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab70456-fe29-4bb2-be66-b8f7c63c767c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-4ab70456-fe29-4bb2-be66-b8f7c63c767c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:45.351829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:45.353034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:45.354124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:45.354124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:45.354124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:45.354124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:45.354124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:45.354124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:45.354124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:45.354124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:45.354124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:07:45.354124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:45.354766 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 323/649] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:45.358953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab70456-fe29-4bb2-be66-b8f7c63c767c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.407730 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:45.450681 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f36cd-2ead-40d3-b89a-f76cc83dfdad None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.494846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2836e8-ce56-4158-b384-274b58d05c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.495526 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2836e8-ce56-4158-b384-274b58d05c60 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.495816 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2836e8-ce56-4158-b384-274b58d05c60 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.496095 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2836e8-ce56-4158-b384-274b58d05c60 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.496955 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2836e8-ce56-4158-b384-274b58d05c60 None admin] RBAC: Adding query filter params (name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:45.497887 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2836e8-ce56-4158-b384-274b58d05c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.498684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2836e8-ce56-4158-b384-274b58d05c60 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.502063 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab70456-fe29-4bb2-be66-b8f7c63c767c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.509728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 327/650] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:45.510602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2836e8-ce56-4158-b384-274b58d05c60 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.542725 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1a2e8-73f5-42be-94e8-8640fa1827bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.543407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1a2e8-73f5-42be-94e8-8640fa1827bc None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.543705 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1a2e8-73f5-42be-94e8-8640fa1827bc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.543977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1a2e8-73f5-42be-94e8-8640fa1827bc None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.544753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1a2e8-73f5-42be-94e8-8640fa1827bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:45.545065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1a2e8-73f5-42be-94e8-8640fa1827bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.546793 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1a2e8-73f5-42be-94e8-8640fa1827bc None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.559200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 324/651] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:45.560057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1a2e8-73f5-42be-94e8-8640fa1827bc None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.576247 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:45.641030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2836e8-ce56-4158-b384-274b58d05c60 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.675387 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1a2e8-73f5-42be-94e8-8640fa1827bc None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.678674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956e8d9-6a3f-4e1b-9c52-c8d52bbc5541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.679450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e8d9-6a3f-4e1b-9c52-c8d52bbc5541 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.679739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e8d9-6a3f-4e1b-9c52-c8d52bbc5541 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.680019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956e8d9-6a3f-4e1b-9c52-c8d52bbc5541 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.680898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956e8d9-6a3f-4e1b-9c52-c8d52bbc5541 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:45.681210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956e8d9-6a3f-4e1b-9c52-c8d52bbc5541 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.682670 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956e8d9-6a3f-4e1b-9c52-c8d52bbc5541 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.685044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:45.705855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 328/652] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:07:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:45.707295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfff7d-686e-499f-9394-3456e0cef650 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.707588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfff7d-686e-499f-9394-3456e0cef650 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.707868 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfff7d-686e-499f-9394-3456e0cef650 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.712380 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 329/653] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:45.712380 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfff7d-686e-499f-9394-3456e0cef650 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.719453 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07716916-2b44-4a86-80b8-422290e187fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.720258 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07716916-2b44-4a86-80b8-422290e187fd None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.720569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07716916-2b44-4a86-80b8-422290e187fd None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.720846 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07716916-2b44-4a86-80b8-422290e187fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.731939 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07716916-2b44-4a86-80b8-422290e187fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.733543 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07716916-2b44-4a86-80b8-422290e187fd None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-07716916-2b44-4a86-80b8-422290e187fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:45.752137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:45.753375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:45.754410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:45.754410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:45.754410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:45.754410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:45.758223 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 325/654] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:45.759114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07716916-2b44-4a86-80b8-422290e187fd None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:45.835207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:45.882255 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07716916-2b44-4a86-80b8-422290e187fd None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.932078 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf5955-d9a6-4a3a-8bd1-df0206b57a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.932687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf5955-d9a6-4a3a-8bd1-df0206b57a4b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.932964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf5955-d9a6-4a3a-8bd1-df0206b57a4b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.933236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf5955-d9a6-4a3a-8bd1-df0206b57a4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.947836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfff7d-686e-499f-9394-3456e0cef650 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:45.985723 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c5faa-0254-4b2f-a307-926c41c36d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:45.986523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c5faa-0254-4b2f-a307-926c41c36d47 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:45.986807 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c5faa-0254-4b2f-a307-926c41c36d47 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:45.987099 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c5faa-0254-4b2f-a307-926c41c36d47 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:45.995001 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c5faa-0254-4b2f-a307-926c41c36d47 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:45.996495 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c5faa-0254-4b2f-a307-926c41c36d47 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:46.009743 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf5955-d9a6-4a3a-8bd1-df0206b57a4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:46.011213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf5955-d9a6-4a3a-8bd1-df0206b57a4b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-9c8c5faa-0254-4b2f-a307-926c41c36d47 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:46.013055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:46.014119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:46.015246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:46.015246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:46.015246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:46.015246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:46.015246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:46.015246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:46.015246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:46.015246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:46.015246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:07:46.015246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:46.015246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 330/655] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:46.015908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b886a-35f0-42a7-9acd-a344eb0edc1d None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:46.015908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b886a-35f0-42a7-9acd-a344eb0edc1d None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:46.016430 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b886a-35f0-42a7-9acd-a344eb0edc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:46.033953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-590b886a-35f0-42a7-9acd-a344eb0edc1d None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:46.035194 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-590b886a-35f0-42a7-9acd-a344eb0edc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:46.090838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:46.093142 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:46.118974 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 331/656] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:46.118974 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b886a-35f0-42a7-9acd-a344eb0edc1d None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:46.145263 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 326/657] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:46.146278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf5955-d9a6-4a3a-8bd1-df0206b57a4b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:46.236242 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b886a-35f0-42a7-9acd-a344eb0edc1d None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:46.263624 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf5955-d9a6-4a3a-8bd1-df0206b57a4b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:46.273355 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01457b11-9dab-4824-9f8f-fb7f1d2dffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:46.274001 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01457b11-9dab-4824-9f8f-fb7f1d2dffa9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:46.274289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01457b11-9dab-4824-9f8f-fb7f1d2dffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:46.274656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01457b11-9dab-4824-9f8f-fb7f1d2dffa9 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:46.275391 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01457b11-9dab-4824-9f8f-fb7f1d2dffa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:46.275706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01457b11-9dab-4824-9f8f-fb7f1d2dffa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:46.277134 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01457b11-9dab-4824-9f8f-fb7f1d2dffa9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:46.289726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 332/658] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:46.290711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01457b11-9dab-4824-9f8f-fb7f1d2dffa9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:46.308325 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c85ea-c20a-40e5-b1c1-21d378f15925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:46.309062 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c85ea-c20a-40e5-b1c1-21d378f15925 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:46.309414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c85ea-c20a-40e5-b1c1-21d378f15925 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:46.309635 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c85ea-c20a-40e5-b1c1-21d378f15925 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:46.310481 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c85ea-c20a-40e5-b1c1-21d378f15925 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:46.310842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c85ea-c20a-40e5-b1c1-21d378f15925 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:46.312908 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c85ea-c20a-40e5-b1c1-21d378f15925 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:46.337696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 327/659] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:07:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:46.337696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c85ea-c20a-40e5-b1c1-21d378f15925 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:46.342187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:46.361350 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:46.402661 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01457b11-9dab-4824-9f8f-fb7f1d2dffa9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:46.444796 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3348a7-cef1-4403-87bb-c597c069fabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:46.445600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3348a7-cef1-4403-87bb-c597c069fabf None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:46.445882 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3348a7-cef1-4403-87bb-c597c069fabf None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:46.446157 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3348a7-cef1-4403-87bb-c597c069fabf None admin] PATH_INFO: `/v3/roles/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:46.453321 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c85ea-c20a-40e5-b1c1-21d378f15925 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:46.459823 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3348a7-cef1-4403-87bb-c597c069fabf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-3f3348a7-cef1-4403-87bb-c597c069fabf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:46.468719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:46.469916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:46.470908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:46.475063 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 333/660] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:46.476614 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3348a7-cef1-4403-87bb-c597c069fabf None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:46.499736 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e69ba-0ae8-449c-bf88-3f1bc8bc49a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:46.500962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e69ba-0ae8-449c-bf88-3f1bc8bc49a0 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:46.501263 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e69ba-0ae8-449c-bf88-3f1bc8bc49a0 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:46.501546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e69ba-0ae8-449c-bf88-3f1bc8bc49a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:46.512501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e69ba-0ae8-449c-bf88-3f1bc8bc49a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:46.513996 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e69ba-0ae8-449c-bf88-3f1bc8bc49a0 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-484e69ba-0ae8-449c-bf88-3f1bc8bc49a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:46.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:46.536727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:46.537958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:46.537958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:46.537958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:46.537958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:46.537958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 328/661] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:46.537958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e69ba-0ae8-449c-bf88-3f1bc8bc49a0 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:46.594880 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3348a7-cef1-4403-87bb-c597c069fabf None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:46.596610 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:46.614989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:46.633700 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49780b36-a357-44f8-ad44-f9dfdbed7943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:46.634517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49780b36-a357-44f8-ad44-f9dfdbed7943 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:46.634804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49780b36-a357-44f8-ad44-f9dfdbed7943 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:46.635082 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49780b36-a357-44f8-ad44-f9dfdbed7943 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:46.644090 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49780b36-a357-44f8-ad44-f9dfdbed7943 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:46.645564 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49780b36-a357-44f8-ad44-f9dfdbed7943 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:46.653848 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e69ba-0ae8-449c-bf88-3f1bc8bc49a0 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-49780b36-a357-44f8-ad44-f9dfdbed7943 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:46.661142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:46.662473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 334/662] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:46.663595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49780b36-a357-44f8-ad44-f9dfdbed7943 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:46.706507 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a15ba-37b7-4085-a4cb-7037a36e72c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:46.706852 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a15ba-37b7-4085-a4cb-7037a36e72c9 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:46.707239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a15ba-37b7-4085-a4cb-7037a36e72c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:46.707729 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025a15ba-37b7-4085-a4cb-7037a36e72c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:46.784645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49780b36-a357-44f8-ad44-f9dfdbed7943 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:46.784910 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a15ba-37b7-4085-a4cb-7037a36e72c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:46.786229 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025a15ba-37b7-4085-a4cb-7037a36e72c9 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:46.823646 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be92c03-ef1b-4286-b9ab-7f8b8324e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:46.824316 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be92c03-ef1b-4286-b9ab-7f8b8324e8f6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:46.824633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be92c03-ef1b-4286-b9ab-7f8b8324e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:46.824910 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be92c03-ef1b-4286-b9ab-7f8b8324e8f6 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:46.825670 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be92c03-ef1b-4286-b9ab-7f8b8324e8f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:46.825986 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be92c03-ef1b-4286-b9ab-7f8b8324e8f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:46.827958 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be92c03-ef1b-4286-b9ab-7f8b8324e8f6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:46.836448 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 335/663] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:46.837289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be92c03-ef1b-4286-b9ab-7f8b8324e8f6 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:46.868548 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:46.871636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:46.909806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 329/664] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:46.909806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a15ba-37b7-4085-a4cb-7037a36e72c9 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:46.963100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be92c03-ef1b-4286-b9ab-7f8b8324e8f6 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:47.005470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbf7c9-3a60-4c21-9b34-5c8b03190840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:47.006150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbf7c9-3a60-4c21-9b34-5c8b03190840 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:47.006434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbf7c9-3a60-4c21-9b34-5c8b03190840 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:47.006721 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbf7c9-3a60-4c21-9b34-5c8b03190840 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:47.007480 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbf7c9-3a60-4c21-9b34-5c8b03190840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:47.007794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbf7c9-3a60-4c21-9b34-5c8b03190840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:47.009252 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbf7c9-3a60-4c21-9b34-5c8b03190840 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:47.023703 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 336/665] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:47.024309 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbf7c9-3a60-4c21-9b34-5c8b03190840 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:47.043192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025a15ba-37b7-4085-a4cb-7037a36e72c9 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:47.089134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ceda2-f2be-4730-8fca-9f2c35f35d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:47.089820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ceda2-f2be-4730-8fca-9f2c35f35d7a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:47.090241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ceda2-f2be-4730-8fca-9f2c35f35d7a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:47.090417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ceda2-f2be-4730-8fca-9f2c35f35d7a None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:47.091222 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ceda2-f2be-4730-8fca-9f2c35f35d7a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:47.091530 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ceda2-f2be-4730-8fca-9f2c35f35d7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:47.093040 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ceda2-f2be-4730-8fca-9f2c35f35d7a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:47.120322 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 330/666] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 20:07:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:47.121689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ceda2-f2be-4730-8fca-9f2c35f35d7a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:47.133565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:47.151750 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbf7c9-3a60-4c21-9b34-5c8b03190840 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:47.160406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:47.196971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda805d-22b3-4faf-8073-be76d717dc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:47.198138 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda805d-22b3-4faf-8073-be76d717dc3d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:47.198606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda805d-22b3-4faf-8073-be76d717dc3d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:47.199086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda805d-22b3-4faf-8073-be76d717dc3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:47.214769 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda805d-22b3-4faf-8073-be76d717dc3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:47.216530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda805d-22b3-4faf-8073-be76d717dc3d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-6bda805d-22b3-4faf-8073-be76d717dc3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:47.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:47.239329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:47.240542 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:47.240542 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:47.240542 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:47.240542 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:47.242214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 337/667] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:47.243218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda805d-22b3-4faf-8073-be76d717dc3d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:47.251092 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ceda2-f2be-4730-8fca-9f2c35f35d7a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:47.297236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084d66d-1d6a-4879-9cc7-a09f55681d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:47.297962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084d66d-1d6a-4879-9cc7-a09f55681d9f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:47.298505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084d66d-1d6a-4879-9cc7-a09f55681d9f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:47.298786 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e084d66d-1d6a-4879-9cc7-a09f55681d9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:47.314320 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084d66d-1d6a-4879-9cc7-a09f55681d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:47.316232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e084d66d-1d6a-4879-9cc7-a09f55681d9f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-e084d66d-1d6a-4879-9cc7-a09f55681d9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:47.342697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:47.343825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:47.350057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:47.350057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:47.350057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:47.350057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:47.350057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 331/668] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:47.350057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084d66d-1d6a-4879-9cc7-a09f55681d9f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:47.361432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda805d-22b3-4faf-8073-be76d717dc3d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:47.400975 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585e94a-f703-4631-bd2a-924178075bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:47.401779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585e94a-f703-4631-bd2a-924178075bcd None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:47.402069 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585e94a-f703-4631-bd2a-924178075bcd None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:47.402351 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585e94a-f703-4631-bd2a-924178075bcd None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:47.406375 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:47.411117 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585e94a-f703-4631-bd2a-924178075bcd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:47.412688 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585e94a-f703-4631-bd2a-924178075bcd None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-5585e94a-f703-4631-bd2a-924178075bcd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:47.428554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:47.429778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:47.431325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:47.431325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:47.431325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:47.431325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:47.431325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:47.431325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:47.431325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:47.431325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:47.431325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 338/669] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:47.431325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585e94a-f703-4631-bd2a-924178075bcd None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:47.449352 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:47.478034 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e084d66d-1d6a-4879-9cc7-a09f55681d9f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:47.539450 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a22f4-ddce-424d-ac19-12ad3aa286fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:47.540518 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a22f4-ddce-424d-ac19-12ad3aa286fa None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:47.541054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a22f4-ddce-424d-ac19-12ad3aa286fa None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:47.541507 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9a22f4-ddce-424d-ac19-12ad3aa286fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:47.556201 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585e94a-f703-4631-bd2a-924178075bcd None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:47.611582 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49af8d8-9ddc-4211-8da0-91e52dfe2a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:47.612607 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49af8d8-9ddc-4211-8da0-91e52dfe2a55 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:47.613014 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49af8d8-9ddc-4211-8da0-91e52dfe2a55 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:47.613419 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49af8d8-9ddc-4211-8da0-91e52dfe2a55 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:47.614649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49af8d8-9ddc-4211-8da0-91e52dfe2a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:47.615166 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49af8d8-9ddc-4211-8da0-91e52dfe2a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:47.617583 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49af8d8-9ddc-4211-8da0-91e52dfe2a55 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:47.628725 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a22f4-ddce-424d-ac19-12ad3aa286fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:47.630823 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9a22f4-ddce-424d-ac19-12ad3aa286fa None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:47.638912 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 339/670] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:47.640276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49af8d8-9ddc-4211-8da0-91e52dfe2a55 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:47.713780 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:47.722051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:47.742001 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 332/671] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:47.742877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a22f4-ddce-424d-ac19-12ad3aa286fa None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:47.767571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49af8d8-9ddc-4211-8da0-91e52dfe2a55 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:47.805088 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce91d4-56d6-407e-8d57-2efd93c6710f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:47.805759 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce91d4-56d6-407e-8d57-2efd93c6710f None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:47.806037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce91d4-56d6-407e-8d57-2efd93c6710f None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:47.806370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce91d4-56d6-407e-8d57-2efd93c6710f None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:47.807034 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce91d4-56d6-407e-8d57-2efd93c6710f None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:47.807341 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce91d4-56d6-407e-8d57-2efd93c6710f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:47.808770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce91d4-56d6-407e-8d57-2efd93c6710f None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:47.821142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 340/672] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:47.822471 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8370d7-a3fd-4122-a0c4-ca100ce1ca0a None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:47.822758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8370d7-a3fd-4122-a0c4-ca100ce1ca0a None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:47.823032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8370d7-a3fd-4122-a0c4-ca100ce1ca0a None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:47.824270 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 341/673] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:47.824992 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8370d7-a3fd-4122-a0c4-ca100ce1ca0a None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:47.851605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9a22f4-ddce-424d-ac19-12ad3aa286fa None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:47.887538 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7527a-560f-401d-805c-c10d4f2eade2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:47.888201 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7527a-560f-401d-805c-c10d4f2eade2 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:47.888523 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7527a-560f-401d-805c-c10d4f2eade2 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:47.888797 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7527a-560f-401d-805c-c10d4f2eade2 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:47.890116 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7527a-560f-401d-805c-c10d4f2eade2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:47.890449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7527a-560f-401d-805c-c10d4f2eade2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:47.892018 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7527a-560f-401d-805c-c10d4f2eade2 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:47.903958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 333/674] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:47.904837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7527a-560f-401d-805c-c10d4f2eade2 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:47.930347 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8370d7-a3fd-4122-a0c4-ca100ce1ca0a None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:47.946646 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:47.949133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:47.965196 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd41ca2-3e72-4e4b-a157-56b8f6fb0a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:47.965986 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd41ca2-3e72-4e4b-a157-56b8f6fb0a64 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:47.966272 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd41ca2-3e72-4e4b-a157-56b8f6fb0a64 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:47.966548 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd41ca2-3e72-4e4b-a157-56b8f6fb0a64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:47.974928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd41ca2-3e72-4e4b-a157-56b8f6fb0a64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:47.976474 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd41ca2-3e72-4e4b-a157-56b8f6fb0a64 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-cdd41ca2-3e72-4e4b-a157-56b8f6fb0a64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:47.991235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:47.992715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 342/675] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:07:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:47.993770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd41ca2-3e72-4e4b-a157-56b8f6fb0a64 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.014178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7527a-560f-401d-805c-c10d4f2eade2 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.050843 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8f166-7a4a-454d-8d67-ac5bf3e6c889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.051641 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8f166-7a4a-454d-8d67-ac5bf3e6c889 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.051979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8f166-7a4a-454d-8d67-ac5bf3e6c889 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.052310 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8f166-7a4a-454d-8d67-ac5bf3e6c889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.064628 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8f166-7a4a-454d-8d67-ac5bf3e6c889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:48.066042 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8f166-7a4a-454d-8d67-ac5bf3e6c889 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-6ec8f166-7a4a-454d-8d67-ac5bf3e6c889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:48.085574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:48.086801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:48.087839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:48.087839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:48.087839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:48.087839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:48.087839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 334/676] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:48.088169 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67594a-1f8a-4b80-aaae-d7185a785e3f None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.088254 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67594a-1f8a-4b80-aaae-d7185a785e3f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.088566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67594a-1f8a-4b80-aaae-d7185a785e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.106163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-8f67594a-1f8a-4b80-aaae-d7185a785e3f None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:48.107457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-8f67594a-1f8a-4b80-aaae-d7185a785e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:48.109679 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd41ca2-3e72-4e4b-a157-56b8f6fb0a64 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.145661 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741714d-f2fa-4a80-87c5-0bcabd5b8e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.146445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741714d-f2fa-4a80-87c5-0bcabd5b8e7b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.146731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741714d-f2fa-4a80-87c5-0bcabd5b8e7b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.147007 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741714d-f2fa-4a80-87c5-0bcabd5b8e7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.154191 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741714d-f2fa-4a80-87c5-0bcabd5b8e7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:48.156358 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741714d-f2fa-4a80-87c5-0bcabd5b8e7b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-b741714d-f2fa-4a80-87c5-0bcabd5b8e7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:48.170990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:48.172260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:48.173415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:48.173415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:48.173415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:48.173415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:48.173415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:48.173415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:48.173415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:48.173415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:48.173415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 343/677] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:48.173415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741714d-f2fa-4a80-87c5-0bcabd5b8e7b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.181305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 335/678] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:48.182730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67594a-1f8a-4b80-aaae-d7185a785e3f None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.211719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:48.220876 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:48.279084 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741714d-f2fa-4a80-87c5-0bcabd5b8e7b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.297692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67594a-1f8a-4b80-aaae-d7185a785e3f None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.315533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3f9a0-2be4-4ea0-a649-1df7431060bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.316114 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3f9a0-2be4-4ea0-a649-1df7431060bf None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.316443 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3f9a0-2be4-4ea0-a649-1df7431060bf None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.316721 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee3f9a0-2be4-4ea0-a649-1df7431060bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.317556 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3f9a0-2be4-4ea0-a649-1df7431060bf None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=c8710592e5a84bb4b885b050e23549dc) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:48.317871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3f9a0-2be4-4ea0-a649-1df7431060bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:48.319320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee3f9a0-2be4-4ea0-a649-1df7431060bf None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:48.325718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 344/679] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/role_assignments?user.id=c8710592e5a84bb4b885b050e23549dc&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:48.326711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3f9a0-2be4-4ea0-a649-1df7431060bf None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.336192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d21165-ca53-482b-b42d-59c88a93dff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.336192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d21165-ca53-482b-b42d-59c88a93dff2 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.336192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d21165-ca53-482b-b42d-59c88a93dff2 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.336192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d21165-ca53-482b-b42d-59c88a93dff2 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.342942 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d21165-ca53-482b-b42d-59c88a93dff2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:48.342942 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d21165-ca53-482b-b42d-59c88a93dff2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:48.342942 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d21165-ca53-482b-b42d-59c88a93dff2 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:48.362717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 336/680] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:48.363550 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d21165-ca53-482b-b42d-59c88a93dff2 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.463943 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:48.465465 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:48.468726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee3f9a0-2be4-4ea0-a649-1df7431060bf None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.485941 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d21165-ca53-482b-b42d-59c88a93dff2 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.512997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620830b-ec57-404e-9c9e-c62620947f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.513674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620830b-ec57-404e-9c9e-c62620947f39 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.513957 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620830b-ec57-404e-9c9e-c62620947f39 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.514233 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620830b-ec57-404e-9c9e-c62620947f39 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.515521 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620830b-ec57-404e-9c9e-c62620947f39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:48.515831 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620830b-ec57-404e-9c9e-c62620947f39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:48.517267 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620830b-ec57-404e-9c9e-c62620947f39 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:48.527039 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173f9fd-3ff8-4e26-85c2-8a7654c75fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.527738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173f9fd-3ff8-4e26-85c2-8a7654c75fc9 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.528023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173f9fd-3ff8-4e26-85c2-8a7654c75fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.528305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173f9fd-3ff8-4e26-85c2-8a7654c75fc9 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.529131 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173f9fd-3ff8-4e26-85c2-8a7654c75fc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:48.529441 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173f9fd-3ff8-4e26-85c2-8a7654c75fc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:48.531782 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173f9fd-3ff8-4e26-85c2-8a7654c75fc9 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:48.534496 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 345/681] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:48.534496 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620830b-ec57-404e-9c9e-c62620947f39 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.557026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 337/682] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:48.557026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173f9fd-3ff8-4e26-85c2-8a7654c75fc9 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.657436 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620830b-ec57-404e-9c9e-c62620947f39 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.681424 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173f9fd-3ff8-4e26-85c2-8a7654c75fc9 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.696737 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52e1a2-9932-459f-9597-e3036ab45a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.697833 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e1a2-9932-459f-9597-e3036ab45a8c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.698153 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e1a2-9932-459f-9597-e3036ab45a8c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.698447 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e1a2-9932-459f-9597-e3036ab45a8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.710290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52e1a2-9932-459f-9597-e3036ab45a8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-ea52e1a2-9932-459f-9597-e3036ab45a8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:48.718725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:48.725445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 346/683] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:48.726404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52e1a2-9932-459f-9597-e3036ab45a8c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.728446 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c47b53-1fa5-4539-9d01-7b287d138951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.729317 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47b53-1fa5-4539-9d01-7b287d138951 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.729606 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47b53-1fa5-4539-9d01-7b287d138951 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.729894 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c47b53-1fa5-4539-9d01-7b287d138951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.744866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:48.749217 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:48.806571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c47b53-1fa5-4539-9d01-7b287d138951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:48.808042 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c47b53-1fa5-4539-9d01-7b287d138951 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:48.835769 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52e1a2-9932-459f-9597-e3036ab45a8c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:48.874399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a6380-dafb-45ba-99be-9e9731c30df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:48.875272 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6380-dafb-45ba-99be-9e9731c30df2 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:48.875562 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6380-dafb-45ba-99be-9e9731c30df2 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:48.875845 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a6380-dafb-45ba-99be-9e9731c30df2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:48.883442 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a6380-dafb-45ba-99be-9e9731c30df2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:48.884967 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a6380-dafb-45ba-99be-9e9731c30df2 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-b59a6380-dafb-45ba-99be-9e9731c30df2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:48.901739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:48.902876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:48.903901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:48.903901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:48.903901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:48.903901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:48.903901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:48.903901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:48.903901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:48.903901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:48.903901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 347/684] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:48.903901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a6380-dafb-45ba-99be-9e9731c30df2 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.928776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 338/685] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:48.930008 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c47b53-1fa5-4539-9d01-7b287d138951 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:48.976682 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:49.002424 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:49.025734 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a6380-dafb-45ba-99be-9e9731c30df2 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.036574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c47b53-1fa5-4539-9d01-7b287d138951 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.062982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5835b-cafc-4136-8967-d18082e3c358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.064000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5835b-cafc-4136-8967-d18082e3c358 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.064286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5835b-cafc-4136-8967-d18082e3c358 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.064592 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5835b-cafc-4136-8967-d18082e3c358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.065425 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5835b-cafc-4136-8967-d18082e3c358 None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=4cebf4a3afdc4124aef25bed824c1dec) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:49.065740 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5835b-cafc-4136-8967-d18082e3c358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.067226 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5835b-cafc-4136-8967-d18082e3c358 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.073772 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 348/686] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/role_assignments?user.id=4cebf4a3afdc4124aef25bed824c1dec&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:49.074598 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5835b-cafc-4136-8967-d18082e3c358 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.082231 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10593e-6bbf-444f-8e0d-ac14ace5764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.082939 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10593e-6bbf-444f-8e0d-ac14ace5764c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.083225 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10593e-6bbf-444f-8e0d-ac14ace5764c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.083510 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10593e-6bbf-444f-8e0d-ac14ace5764c None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.084292 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10593e-6bbf-444f-8e0d-ac14ace5764c None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:49.084633 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10593e-6bbf-444f-8e0d-ac14ace5764c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.086090 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10593e-6bbf-444f-8e0d-ac14ace5764c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.102926 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 339/687] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:49.104268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10593e-6bbf-444f-8e0d-ac14ace5764c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.206019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5835b-cafc-4136-8967-d18082e3c358 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.226022 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10593e-6bbf-444f-8e0d-ac14ace5764c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.240755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:49.247572 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa0f26-a9fb-40ba-a109-614488b67640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.248285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0f26-a9fb-40ba-a109-614488b67640 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.249603 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0f26-a9fb-40ba-a109-614488b67640 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.249603 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0f26-a9fb-40ba-a109-614488b67640 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.250417 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa0f26-a9fb-40ba-a109-614488b67640 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:49.250734 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa0f26-a9fb-40ba-a109-614488b67640 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.252241 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa0f26-a9fb-40ba-a109-614488b67640 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.271024 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 349/688] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:49.271024 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa0f26-a9fb-40ba-a109-614488b67640 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.272084 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:49.277792 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3852f-145d-4dec-aaca-cdffc20193e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.279628 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3852f-145d-4dec-aaca-cdffc20193e6 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.280502 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3852f-145d-4dec-aaca-cdffc20193e6 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.281029 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3852f-145d-4dec-aaca-cdffc20193e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.294853 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3852f-145d-4dec-aaca-cdffc20193e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.297408 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3852f-145d-4dec-aaca-cdffc20193e6 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-c1f3852f-145d-4dec-aaca-cdffc20193e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:49.317578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:49.318707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:49.319723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:49.319723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:49.319723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:49.319723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:49.323556 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 340/689] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:49.324398 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3852f-145d-4dec-aaca-cdffc20193e6 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.402939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa0f26-a9fb-40ba-a109-614488b67640 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.442431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8aae1e-dcf5-47bb-a4a4-afd6175dd083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.443230 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8aae1e-dcf5-47bb-a4a4-afd6175dd083 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.443514 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8aae1e-dcf5-47bb-a4a4-afd6175dd083 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.443826 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8aae1e-dcf5-47bb-a4a4-afd6175dd083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.455835 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8aae1e-dcf5-47bb-a4a4-afd6175dd083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.457505 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8aae1e-dcf5-47bb-a4a4-afd6175dd083 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.462600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3852f-145d-4dec-aaca-cdffc20193e6 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-4c8aae1e-dcf5-47bb-a4a4-afd6175dd083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:49.480490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:49.481795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:49.483567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:49.483567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:49.483567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:49.483567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:49.485970 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 350/690] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:49.487021 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8aae1e-dcf5-47bb-a4a4-afd6175dd083 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.507552 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9178909-3bfc-4029-9141-05ad169fe5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.508349 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178909-3bfc-4029-9141-05ad169fe5d7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.508809 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178909-3bfc-4029-9141-05ad169fe5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.509449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9178909-3bfc-4029-9141-05ad169fe5d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.510173 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9178909-3bfc-4029-9141-05ad169fe5d7 None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=d7917b3c79bd42cea17ec8eba5a265ed) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:49.510622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9178909-3bfc-4029-9141-05ad169fe5d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.512195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9178909-3bfc-4029-9141-05ad169fe5d7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.524155 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 341/691] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/role_assignments?user.id=d7917b3c79bd42cea17ec8eba5a265ed&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:49.525309 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9178909-3bfc-4029-9141-05ad169fe5d7 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.562601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:49.580763 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:49.647001 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8aae1e-dcf5-47bb-a4a4-afd6175dd083 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.717264 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea924ba6-d44c-4892-8f2f-4c59fd7db8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.719876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea924ba6-d44c-4892-8f2f-4c59fd7db8f6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.720852 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea924ba6-d44c-4892-8f2f-4c59fd7db8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.721291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea924ba6-d44c-4892-8f2f-4c59fd7db8f6 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.722225 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea924ba6-d44c-4892-8f2f-4c59fd7db8f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:49.722715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea924ba6-d44c-4892-8f2f-4c59fd7db8f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.724376 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea924ba6-d44c-4892-8f2f-4c59fd7db8f6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.740445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 351/692] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:49.745891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea924ba6-d44c-4892-8f2f-4c59fd7db8f6 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.806620 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9178909-3bfc-4029-9141-05ad169fe5d7 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.856465 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf8356-14bd-4974-8144-82aa5ac2e158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.857309 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf8356-14bd-4974-8144-82aa5ac2e158 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.857690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf8356-14bd-4974-8144-82aa5ac2e158 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.858092 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf8356-14bd-4974-8144-82aa5ac2e158 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.859396 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf8356-14bd-4974-8144-82aa5ac2e158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:49.859808 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf8356-14bd-4974-8144-82aa5ac2e158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.862130 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf8356-14bd-4974-8144-82aa5ac2e158 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.872662 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea924ba6-d44c-4892-8f2f-4c59fd7db8f6 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:49.880484 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 342/693] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:49.886061 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf8356-14bd-4974-8144-82aa5ac2e158 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.898261 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:49.919153 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e63ee-7a26-4d4f-bf33-073806337a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:49.920409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e63ee-7a26-4d4f-bf33-073806337a95 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:49.921888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e63ee-7a26-4d4f-bf33-073806337a95 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:49.922400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e63ee-7a26-4d4f-bf33-073806337a95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:49.930598 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e63ee-7a26-4d4f-bf33-073806337a95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:49.932525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e63ee-7a26-4d4f-bf33-073806337a95 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-af7e63ee-7a26-4d4f-bf33-073806337a95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:49.949977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:49.951115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:49.952742 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:49.952742 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:49.952742 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:49.952742 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:49.952742 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:49.952742 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:49.952742 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:49.952742 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:49.955446 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 352/694] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:49.961584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e63ee-7a26-4d4f-bf33-073806337a95 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:49.995797 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:50.029767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf8356-14bd-4974-8144-82aa5ac2e158 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:50.087152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8d4d9-00a9-41fa-838b-49b416516be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:50.087979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8d4d9-00a9-41fa-838b-49b416516be9 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:50.088264 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8d4d9-00a9-41fa-838b-49b416516be9 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:50.088569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8d4d9-00a9-41fa-838b-49b416516be9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:50.107897 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8d4d9-00a9-41fa-838b-49b416516be9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:50.109422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8d4d9-00a9-41fa-838b-49b416516be9 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-6be8d4d9-00a9-41fa-838b-49b416516be9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:50.136959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:50.138639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:50.141740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:50.141740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:50.141740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:50.141740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:50.141740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:50.141740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:50.141740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:50.141740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:50.141740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:07:50.141740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:50.141740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 343/695] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:50.145388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8d4d9-00a9-41fa-838b-49b416516be9 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:50.198415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e63ee-7a26-4d4f-bf33-073806337a95 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:50.267899 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8d4d9-00a9-41fa-838b-49b416516be9 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:50.308945 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320da174-8c01-4b8b-9039-d5d261f1653b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:50.309460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6c2c7-5bb6-4f28-a04a-71aa0eae151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:50.309678 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da174-8c01-4b8b-9039-d5d261f1653b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:50.309911 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da174-8c01-4b8b-9039-d5d261f1653b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:50.310190 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320da174-8c01-4b8b-9039-d5d261f1653b None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:50.311013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6c2c7-5bb6-4f28-a04a-71aa0eae151c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:50.311175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320da174-8c01-4b8b-9039-d5d261f1653b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:50.311480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320da174-8c01-4b8b-9039-d5d261f1653b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:50.311890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6c2c7-5bb6-4f28-a04a-71aa0eae151c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:50.312413 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed6c2c7-5bb6-4f28-a04a-71aa0eae151c None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:50.313037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320da174-8c01-4b8b-9039-d5d261f1653b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:50.313686 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6c2c7-5bb6-4f28-a04a-71aa0eae151c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:50.314228 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6c2c7-5bb6-4f28-a04a-71aa0eae151c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:50.315918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed6c2c7-5bb6-4f28-a04a-71aa0eae151c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:50.324864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:50.335126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 344/696] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:07:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:50.342214 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320da174-8c01-4b8b-9039-d5d261f1653b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:50.350926 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 353/697] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:50.374024 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6c2c7-5bb6-4f28-a04a-71aa0eae151c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:50.469896 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320da174-8c01-4b8b-9039-d5d261f1653b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:50.483640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:50.519782 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3bccd-0bd3-4cc3-ad03-d53ba46b297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:50.520362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3bccd-0bd3-4cc3-ad03-d53ba46b297a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:50.520654 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3bccd-0bd3-4cc3-ad03-d53ba46b297a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:50.520954 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3bccd-0bd3-4cc3-ad03-d53ba46b297a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:50.622434 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:50.629088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3bccd-0bd3-4cc3-ad03-d53ba46b297a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:50.630814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3bccd-0bd3-4cc3-ad03-d53ba46b297a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:50.636164 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed6c2c7-5bb6-4f28-a04a-71aa0eae151c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:50.720962 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e89f4d-3f20-413e-aac1-f016c17d2987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:50.721617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e89f4d-3f20-413e-aac1-f016c17d2987 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:50.721908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e89f4d-3f20-413e-aac1-f016c17d2987 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:50.722184 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e89f4d-3f20-413e-aac1-f016c17d2987 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:50.723034 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e89f4d-3f20-413e-aac1-f016c17d2987 None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=7777c807131c4d9886c6aeff5f756711) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:50.723348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e89f4d-3f20-413e-aac1-f016c17d2987 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:50.724873 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e89f4d-3f20-413e-aac1-f016c17d2987 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:50.732145 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 354/698] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:50 2022] GET /identity/v3/role_assignments?user.id=7777c807131c4d9886c6aeff5f756711&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 608 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:50.837687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 345/699] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:07:50.843318 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e89f4d-3f20-413e-aac1-f016c17d2987 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:50.970145 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e89f4d-3f20-413e-aac1-f016c17d2987 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:50.972691 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ab014-1faa-4ba4-8ec7-ded213ab9e1b None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:50.973240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ab014-1faa-4ba4-8ec7-ded213ab9e1b None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:50.973750 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ab014-1faa-4ba4-8ec7-ded213ab9e1b None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:50.975734 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 346/700] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:50.980156 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:50.982610 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5124803-1476-433d-8da3-85ae3eb62dd0 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:50.983652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5124803-1476-433d-8da3-85ae3eb62dd0 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:50.984143 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5124803-1476-433d-8da3-85ae3eb62dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:51.002947 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-f5124803-1476-433d-8da3-85ae3eb62dd0 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:51.004897 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-f5124803-1476-433d-8da3-85ae3eb62dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:51.041676 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c67f6-46cf-4207-b06f-342845706060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:51.041676 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c67f6-46cf-4207-b06f-342845706060 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:51.041676 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c67f6-46cf-4207-b06f-342845706060 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:51.041676 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776c67f6-46cf-4207-b06f-342845706060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:51.055969 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c67f6-46cf-4207-b06f-342845706060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:51.057499 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776c67f6-46cf-4207-b06f-342845706060 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-776c67f6-46cf-4207-b06f-342845706060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:51.082579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:51.083626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:51.084753 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:51.084753 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:51.084753 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:51.084753 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:51.086746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 355/701] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:51.092794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c67f6-46cf-4207-b06f-342845706060 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:51.112409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 347/702] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:51.142473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5124803-1476-433d-8da3-85ae3eb62dd0 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:51.196753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:51.208802 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776c67f6-46cf-4207-b06f-342845706060 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:51.246773 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a58570-0ab6-4b55-a225-c021c519ca13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:51.247400 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a58570-0ab6-4b55-a225-c021c519ca13 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:51.247682 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a58570-0ab6-4b55-a225-c021c519ca13 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:51.248034 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a58570-0ab6-4b55-a225-c021c519ca13 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:51.249504 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a58570-0ab6-4b55-a225-c021c519ca13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:51.249504 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a58570-0ab6-4b55-a225-c021c519ca13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:51.250930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a58570-0ab6-4b55-a225-c021c519ca13 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:51.258836 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5124803-1476-433d-8da3-85ae3eb62dd0 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:51.265711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 356/703] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:51.273475 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a58570-0ab6-4b55-a225-c021c519ca13 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:51.292102 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:51.303104 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62966116-c1bd-4ca5-9d08-9289177ce9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:51.304318 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62966116-c1bd-4ca5-9d08-9289177ce9fc None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:51.304866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62966116-c1bd-4ca5-9d08-9289177ce9fc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:51.305361 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62966116-c1bd-4ca5-9d08-9289177ce9fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:51.317532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62966116-c1bd-4ca5-9d08-9289177ce9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:51.319394 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62966116-c1bd-4ca5-9d08-9289177ce9fc None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-62966116-c1bd-4ca5-9d08-9289177ce9fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:51.339498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:51.340594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:51.341614 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:51.341614 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:51.341614 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:51.341614 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:51.344735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 348/704] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:51.350642 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62966116-c1bd-4ca5-9d08-9289177ce9fc None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:51.385876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a58570-0ab6-4b55-a225-c021c519ca13 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:51.424307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2536ca-3e2c-4bfd-86b9-198da84e3917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:51.425147 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2536ca-3e2c-4bfd-86b9-198da84e3917 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:51.425433 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2536ca-3e2c-4bfd-86b9-198da84e3917 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:51.425712 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2536ca-3e2c-4bfd-86b9-198da84e3917 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:51.434048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2536ca-3e2c-4bfd-86b9-198da84e3917 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:51.435521 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2536ca-3e2c-4bfd-86b9-198da84e3917 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:51.448717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-ed2536ca-3e2c-4bfd-86b9-198da84e3917 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:51.451510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:51.452557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:51.453571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:51.453571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:51.453571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:51.453571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:51.453571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:51.453571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:51.453571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:51.453571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:51.453571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 357/705] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:51.458514 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2536ca-3e2c-4bfd-86b9-198da84e3917 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:51.471280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62966116-c1bd-4ca5-9d08-9289177ce9fc None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:51.509492 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44fa25-b982-4b4f-b628-e55aaeec1a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:51.510548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44fa25-b982-4b4f-b628-e55aaeec1a0a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:51.511040 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44fa25-b982-4b4f-b628-e55aaeec1a0a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:51.511540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44fa25-b982-4b4f-b628-e55aaeec1a0a None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:51.512668 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44fa25-b982-4b4f-b628-e55aaeec1a0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:51.513220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44fa25-b982-4b4f-b628-e55aaeec1a0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:51.515379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44fa25-b982-4b4f-b628-e55aaeec1a0a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:51.528880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 349/706] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:51.536673 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44fa25-b982-4b4f-b628-e55aaeec1a0a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:51.550385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:51.596448 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2536ca-3e2c-4bfd-86b9-198da84e3917 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:51.629050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc756796-3940-4884-ae59-1a98ce8728b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:51.629712 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc756796-3940-4884-ae59-1a98ce8728b7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:51.629993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc756796-3940-4884-ae59-1a98ce8728b7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:51.630269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc756796-3940-4884-ae59-1a98ce8728b7 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:51.631050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc756796-3940-4884-ae59-1a98ce8728b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:51.631356 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc756796-3940-4884-ae59-1a98ce8728b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:51.633044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc756796-3940-4884-ae59-1a98ce8728b7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:51.647622 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 358/707] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:07:51.653900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc756796-3940-4884-ae59-1a98ce8728b7 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:51.688161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44fa25-b982-4b4f-b628-e55aaeec1a0a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:51.728571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18003f-a9ba-4c69-8361-71fea87543fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:51.729547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18003f-a9ba-4c69-8361-71fea87543fa None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:51.729986 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18003f-a9ba-4c69-8361-71fea87543fa None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:51.730427 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18003f-a9ba-4c69-8361-71fea87543fa None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:51.731304 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18003f-a9ba-4c69-8361-71fea87543fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:51.733573 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18003f-a9ba-4c69-8361-71fea87543fa None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:51.744760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:51.827987 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 350/708] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:07:51 2022] POST /identity/v3/users => generated 254 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:07:51.860493 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc756796-3940-4884-ae59-1a98ce8728b7 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:51.886638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:51.892811 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16dcb3-3de0-4f34-87c3-96e7e2b00b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:51.898677 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16dcb3-3de0-4f34-87c3-96e7e2b00b53 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:51.898677 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16dcb3-3de0-4f34-87c3-96e7e2b00b53 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:51.898677 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16dcb3-3de0-4f34-87c3-96e7e2b00b53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:51.898677 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16dcb3-3de0-4f34-87c3-96e7e2b00b53 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=f6d44c613f524fe1804d6c9b65fe97ca) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:51.898677 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16dcb3-3de0-4f34-87c3-96e7e2b00b53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:51.899110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16dcb3-3de0-4f34-87c3-96e7e2b00b53 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:51.904397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 359/709] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:51 2022] GET /identity/v3/role_assignments?user.id=f6d44c613f524fe1804d6c9b65fe97ca&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:51.924636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc235719-0ee9-4cfb-9acc-a49416fafe3c None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:51.925608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc235719-0ee9-4cfb-9acc-a49416fafe3c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:51.926085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc235719-0ee9-4cfb-9acc-a49416fafe3c None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:51.928273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 351/710] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:51.934003 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a01be-241d-46e9-a776-398bf6adc30e None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:51.934403 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a01be-241d-46e9-a776-398bf6adc30e None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:51.934585 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a01be-241d-46e9-a776-398bf6adc30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:51.958986 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-b21a01be-241d-46e9-a776-398bf6adc30e None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:51.960367 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-b21a01be-241d-46e9-a776-398bf6adc30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:52.069365 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 360/711] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:52.104052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc235719-0ee9-4cfb-9acc-a49416fafe3c None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:52.238320 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc235719-0ee9-4cfb-9acc-a49416fafe3c None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:52.285833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc42c26b-16de-4c27-b64a-c8fbfe43bafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:52.286673 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42c26b-16de-4c27-b64a-c8fbfe43bafc None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:52.286962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42c26b-16de-4c27-b64a-c8fbfe43bafc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:52.287242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc42c26b-16de-4c27-b64a-c8fbfe43bafc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:52.287911 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42c26b-16de-4c27-b64a-c8fbfe43bafc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:52.289364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc42c26b-16de-4c27-b64a-c8fbfe43bafc None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-cc42c26b-16de-4c27-b64a-c8fbfe43bafc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:52.312821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:52.314009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:52.314981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 352/712] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 20:07:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:52.320387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a01be-241d-46e9-a776-398bf6adc30e None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:52.458899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a01be-241d-46e9-a776-398bf6adc30e None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:52.492927 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:52.505988 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f648f97-962e-4c56-acca-945a5c18ec9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:52.506673 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f648f97-962e-4c56-acca-945a5c18ec9a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:52.506958 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f648f97-962e-4c56-acca-945a5c18ec9a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:52.507238 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f648f97-962e-4c56-acca-945a5c18ec9a None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:52.507965 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f648f97-962e-4c56-acca-945a5c18ec9a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:52.508278 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f648f97-962e-4c56-acca-945a5c18ec9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:52.509713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f648f97-962e-4c56-acca-945a5c18ec9a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:52.512179 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b9cd1-13cf-4467-b400-5d0f33465318 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:52.512506 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b9cd1-13cf-4467-b400-5d0f33465318 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:52.512793 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b9cd1-13cf-4467-b400-5d0f33465318 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:52.514524 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 353/713] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:52.515290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 361/714] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:07:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:52.520439 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b9cd1-13cf-4467-b400-5d0f33465318 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:52.523877 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a77af-3239-4731-b746-ea65cc9df105 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:52.524166 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a77af-3239-4731-b746-ea65cc9df105 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:52.524473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a77af-3239-4731-b746-ea65cc9df105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:52.549911 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-825a77af-3239-4731-b746-ea65cc9df105 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:52.551129 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-825a77af-3239-4731-b746-ea65cc9df105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:52.617854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:52.662862 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 362/715] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:07:52.690563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b9cd1-13cf-4467-b400-5d0f33465318 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:52.702825 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a77af-3239-4731-b746-ea65cc9df105 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:52.773153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00eba4d-b9ad-4734-8883-dd54b11c3609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:52.773747 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eba4d-b9ad-4734-8883-dd54b11c3609 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:52.774029 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eba4d-b9ad-4734-8883-dd54b11c3609 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:52.774305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eba4d-b9ad-4734-8883-dd54b11c3609 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:52.775027 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00eba4d-b9ad-4734-8883-dd54b11c3609 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:52.775335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00eba4d-b9ad-4734-8883-dd54b11c3609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:52.776874 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00eba4d-b9ad-4734-8883-dd54b11c3609 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:52.799425 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 354/716] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:07:52 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:52.839925 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a77af-3239-4731-b746-ea65cc9df105 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:52.884401 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:52.908612 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cbed4-961f-4998-9394-f799c4e71d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:52.909431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cbed4-961f-4998-9394-f799c4e71d16 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:52.909710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cbed4-961f-4998-9394-f799c4e71d16 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:52.909984 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cbed4-961f-4998-9394-f799c4e71d16 None admin] PATH_INFO: `/v3/services/network` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:52.910647 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cbed4-961f-4998-9394-f799c4e71d16 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:52.912049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cbed4-961f-4998-9394-f799c4e71d16 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-909cbed4-961f-4998-9394-f799c4e71d16 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:52.934295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:52.935468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:52.937287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 363/717] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:52 2022] GET /identity/v3/services/network => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:52.942583 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00eba4d-b9ad-4734-8883-dd54b11c3609 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:53.092854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00eba4d-b9ad-4734-8883-dd54b11c3609 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:53.132459 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:53.147552 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1df8f-d419-4698-8ca1-4da579ed8c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:53.148739 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1df8f-d419-4698-8ca1-4da579ed8c3c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:53.149267 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1df8f-d419-4698-8ca1-4da579ed8c3c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:53.149760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1df8f-d419-4698-8ca1-4da579ed8c3c None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:53.150685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1df8f-d419-4698-8ca1-4da579ed8c3c None admin] RBAC: Adding query filter params (name=network) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:53.151201 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1df8f-d419-4698-8ca1-4da579ed8c3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:53.152913 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1df8f-d419-4698-8ca1-4da579ed8c3c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:53.157669 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 355/718] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:07:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:53.163249 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cbed4-961f-4998-9394-f799c4e71d16 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:53.291407 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cbed4-961f-4998-9394-f799c4e71d16 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:53.342884 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa62f32-4cca-42f4-bc40-6b66189bf865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:53.343918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa62f32-4cca-42f4-bc40-6b66189bf865 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:53.344470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa62f32-4cca-42f4-bc40-6b66189bf865 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:53.344955 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa62f32-4cca-42f4-bc40-6b66189bf865 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:53.345917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa62f32-4cca-42f4-bc40-6b66189bf865 None admin] RBAC: Adding query filter params (type=network) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:53.346443 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa62f32-4cca-42f4-bc40-6b66189bf865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:53.348046 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa62f32-4cca-42f4-bc40-6b66189bf865 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:53.357099 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 364/719] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:07:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:54.276089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc32bfa-a3b9-406f-894b-54f1a4380960 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.276648 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc32bfa-a3b9-406f-894b-54f1a4380960 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.276765 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc32bfa-a3b9-406f-894b-54f1a4380960 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.278397 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 356/720] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:54.284618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f29a60-bfbe-4476-9926-44fc27996b59 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.285197 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f29a60-bfbe-4476-9926-44fc27996b59 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.285197 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f29a60-bfbe-4476-9926-44fc27996b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.306946 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-36f29a60-bfbe-4476-9926-44fc27996b59 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:54.308139 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-36f29a60-bfbe-4476-9926-44fc27996b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:54.355801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:54.381192 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 365/721] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:54.404971 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc32bfa-a3b9-406f-894b-54f1a4380960 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:54.525067 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc32bfa-a3b9-406f-894b-54f1a4380960 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:54.565093 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51adec16-d0fb-48c7-82fe-6dcbb204881f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:54.565914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adec16-d0fb-48c7-82fe-6dcbb204881f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.566196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adec16-d0fb-48c7-82fe-6dcbb204881f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.566473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adec16-d0fb-48c7-82fe-6dcbb204881f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.581757 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51adec16-d0fb-48c7-82fe-6dcbb204881f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-51adec16-d0fb-48c7-82fe-6dcbb204881f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:54.589700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:54.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:54.591940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 357/722] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:07:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:54.598107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f29a60-bfbe-4476-9926-44fc27996b59 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:54.679189 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bb6b4-e09b-4a9e-8da2-3b06ccc370cc None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.679499 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bb6b4-e09b-4a9e-8da2-3b06ccc370cc None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.679703 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bb6b4-e09b-4a9e-8da2-3b06ccc370cc None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.681872 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 358/723] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:07:54.687768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124962e-4744-4ae0-8615-1b6fe134cf3f None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.688375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124962e-4744-4ae0-8615-1b6fe134cf3f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.688881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124962e-4744-4ae0-8615-1b6fe134cf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.707580 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-5124962e-4744-4ae0-8615-1b6fe134cf3f None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:54.716364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-5124962e-4744-4ae0-8615-1b6fe134cf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:54.756539 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f29a60-bfbe-4476-9926-44fc27996b59 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:54.804730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:54.824137 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14a7f1-1844-4c10-8f97-95dc64b5db68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:54.825214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14a7f1-1844-4c10-8f97-95dc64b5db68 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.825739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14a7f1-1844-4c10-8f97-95dc64b5db68 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.826324 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14a7f1-1844-4c10-8f97-95dc64b5db68 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.827448 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14a7f1-1844-4c10-8f97-95dc64b5db68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:54.827967 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14a7f1-1844-4c10-8f97-95dc64b5db68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:54.836378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 359/724] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:07:54.836378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f733f7b-7d96-43be-8b79-cf77ad0a25a1 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.836378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f733f7b-7d96-43be-8b79-cf77ad0a25a1 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.836378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f733f7b-7d96-43be-8b79-cf77ad0a25a1 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.836378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 360/725] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:54.837583 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14a7f1-1844-4c10-8f97-95dc64b5db68 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:54.845977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129eb5-26dc-43de-b15e-0690ba17901d None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:54.846513 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129eb5-26dc-43de-b15e-0690ba17901d None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:54.846977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129eb5-26dc-43de-b15e-0690ba17901d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:54.851864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 366/726] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:54.858985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14a7f1-1844-4c10-8f97-95dc64b5db68 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:54.873646 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-45129eb5-26dc-43de-b15e-0690ba17901d None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:54.875767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-45129eb5-26dc-43de-b15e-0690ba17901d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:54.905761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:54.947273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 361/727] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:54.948634 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45129eb5-26dc-43de-b15e-0690ba17901d None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:54.990723 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14a7f1-1844-4c10-8f97-95dc64b5db68 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.030987 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150fe85-e4cf-45a0-a3e2-4691747172e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.031665 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150fe85-e4cf-45a0-a3e2-4691747172e6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.032031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150fe85-e4cf-45a0-a3e2-4691747172e6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.032407 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150fe85-e4cf-45a0-a3e2-4691747172e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.043593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150fe85-e4cf-45a0-a3e2-4691747172e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-1150fe85-e4cf-45a0-a3e2-4691747172e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:55.051798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:55.052904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:07:55.053853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:55.058181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 367/728] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:07:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:07:55.059123 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150fe85-e4cf-45a0-a3e2-4691747172e6 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.059532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45129eb5-26dc-43de-b15e-0690ba17901d None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.072772 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:55.099837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715c21e-ee74-44bb-9769-43585ebb0313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.101024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715c21e-ee74-44bb-9769-43585ebb0313 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.101521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715c21e-ee74-44bb-9769-43585ebb0313 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.102039 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715c21e-ee74-44bb-9769-43585ebb0313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.114136 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715c21e-ee74-44bb-9769-43585ebb0313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.115977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715c21e-ee74-44bb-9769-43585ebb0313 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-9715c21e-ee74-44bb-9769-43585ebb0313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:55.136048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:55.137140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:55.138163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:55.138163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:55.138163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:55.138163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:55.144850 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 362/729] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:55.146063 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715c21e-ee74-44bb-9769-43585ebb0313 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.167650 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:55.172836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150fe85-e4cf-45a0-a3e2-4691747172e6 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.210366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80545ff0-917c-4698-a64a-f4f59b0e5704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.211438 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80545ff0-917c-4698-a64a-f4f59b0e5704 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.211656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80545ff0-917c-4698-a64a-f4f59b0e5704 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.212041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80545ff0-917c-4698-a64a-f4f59b0e5704 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.224094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80545ff0-917c-4698-a64a-f4f59b0e5704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-80545ff0-917c-4698-a64a-f4f59b0e5704 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:07:55.232110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:55.233971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:07:55.234923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:55.239182 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 368/730] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:55.239182 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80545ff0-917c-4698-a64a-f4f59b0e5704 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.278178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715c21e-ee74-44bb-9769-43585ebb0313 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.324477 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102e7b4-1842-4d5f-b277-75cabb5cf90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.325239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102e7b4-1842-4d5f-b277-75cabb5cf90d None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.325685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102e7b4-1842-4d5f-b277-75cabb5cf90d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.325979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9102e7b4-1842-4d5f-b277-75cabb5cf90d None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.328105 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102e7b4-1842-4d5f-b277-75cabb5cf90d None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:55.328453 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102e7b4-1842-4d5f-b277-75cabb5cf90d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.329962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9102e7b4-1842-4d5f-b277-75cabb5cf90d None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.344513 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 363/731] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:55.345610 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102e7b4-1842-4d5f-b277-75cabb5cf90d None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.368617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80545ff0-917c-4698-a64a-f4f59b0e5704 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.378461 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:55.413560 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233bed0-715d-4532-92cf-155c6cf5e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.414247 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233bed0-715d-4532-92cf-155c6cf5e1ae None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.414545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233bed0-715d-4532-92cf-155c6cf5e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.414831 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233bed0-715d-4532-92cf-155c6cf5e1ae None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.415582 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233bed0-715d-4532-92cf-155c6cf5e1ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:55.415901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233bed0-715d-4532-92cf-155c6cf5e1ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.417460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233bed0-715d-4532-92cf-155c6cf5e1ae None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.432140 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 369/732] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:55.433079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233bed0-715d-4532-92cf-155c6cf5e1ae None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.458093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:55.477759 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9102e7b4-1842-4d5f-b277-75cabb5cf90d None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.524033 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdef1d-338b-4786-9cb8-99ccff5af85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.525351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdef1d-338b-4786-9cb8-99ccff5af85e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.525885 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdef1d-338b-4786-9cb8-99ccff5af85e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.526452 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdef1d-338b-4786-9cb8-99ccff5af85e None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.527437 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdef1d-338b-4786-9cb8-99ccff5af85e None admin] RBAC: Adding query filter params (name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:55.528095 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdef1d-338b-4786-9cb8-99ccff5af85e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.530655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdef1d-338b-4786-9cb8-99ccff5af85e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.542439 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 364/733] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:55.543726 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdef1d-338b-4786-9cb8-99ccff5af85e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.555193 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233bed0-715d-4532-92cf-155c6cf5e1ae None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.592325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fc312-57ae-49c3-ba20-b8d6b423c7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.593909 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fc312-57ae-49c3-ba20-b8d6b423c7b9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.594200 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fc312-57ae-49c3-ba20-b8d6b423c7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.594532 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fc312-57ae-49c3-ba20-b8d6b423c7b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.605858 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fc312-57ae-49c3-ba20-b8d6b423c7b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.607219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fc312-57ae-49c3-ba20-b8d6b423c7b9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-5e8fc312-57ae-49c3-ba20-b8d6b423c7b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:55.626617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:55.627836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:55.629552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:55.629552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:55.629552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:55.629552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:55.632445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 370/734] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:55.633344 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fc312-57ae-49c3-ba20-b8d6b423c7b9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.647237 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:55.663755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdef1d-338b-4786-9cb8-99ccff5af85e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.702523 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad0d93-58fd-4371-8e25-1a9be72241d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.703392 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad0d93-58fd-4371-8e25-1a9be72241d7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.703672 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad0d93-58fd-4371-8e25-1a9be72241d7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.703950 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad0d93-58fd-4371-8e25-1a9be72241d7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.710092 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:55.713414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad0d93-58fd-4371-8e25-1a9be72241d7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.715010 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad0d93-58fd-4371-8e25-1a9be72241d7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-73ad0d93-58fd-4371-8e25-1a9be72241d7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:55.729854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:55.730998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 365/735] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:55.732656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad0d93-58fd-4371-8e25-1a9be72241d7 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.747477 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fc312-57ae-49c3-ba20-b8d6b423c7b9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.787054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31450723-8783-42f4-9564-92b337a7cab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.787852 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31450723-8783-42f4-9564-92b337a7cab6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.788137 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31450723-8783-42f4-9564-92b337a7cab6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.788754 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31450723-8783-42f4-9564-92b337a7cab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.800054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31450723-8783-42f4-9564-92b337a7cab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.802566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31450723-8783-42f4-9564-92b337a7cab6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-31450723-8783-42f4-9564-92b337a7cab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:55.820855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:55.821944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:55.822966 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:55.822966 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:55.822966 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:55.822966 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:55.822966 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 371/736] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:55.824091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67ddbc-798a-40a9-ae3c-cf5b3661047e None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.824091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67ddbc-798a-40a9-ae3c-cf5b3661047e None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.824091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67ddbc-798a-40a9-ae3c-cf5b3661047e None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.830091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 372/737] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:55.830091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67ddbc-798a-40a9-ae3c-cf5b3661047e None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.849054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad0d93-58fd-4371-8e25-1a9be72241d7 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.884401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aa77a-64d5-47bb-96ce-458c4810f869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.885070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aa77a-64d5-47bb-96ce-458c4810f869 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.885357 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aa77a-64d5-47bb-96ce-458c4810f869 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.885635 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aa77a-64d5-47bb-96ce-458c4810f869 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.886370 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8aa77a-64d5-47bb-96ce-458c4810f869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:55.886683 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8aa77a-64d5-47bb-96ce-458c4810f869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.888059 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8aa77a-64d5-47bb-96ce-458c4810f869 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:55.901374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 366/738] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:55.902218 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aa77a-64d5-47bb-96ce-458c4810f869 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:55.908997 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:55.945438 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67ddbc-798a-40a9-ae3c-cf5b3661047e None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:55.973740 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:55.985916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21870a31-4552-41d9-b31c-b6596cf2381e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:55.985916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21870a31-4552-41d9-b31c-b6596cf2381e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:55.985916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21870a31-4552-41d9-b31c-b6596cf2381e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:55.986276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21870a31-4552-41d9-b31c-b6596cf2381e None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:55.986867 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21870a31-4552-41d9-b31c-b6596cf2381e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:55.987151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21870a31-4552-41d9-b31c-b6596cf2381e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:55.988863 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21870a31-4552-41d9-b31c-b6596cf2381e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:56.008324 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 373/739] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:56.010007 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f3cac-d1ca-440f-88d3-546000530be0 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:56.010224 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f3cac-d1ca-440f-88d3-546000530be0 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:56.010493 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f3cac-d1ca-440f-88d3-546000530be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:56.020494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aa77a-64d5-47bb-96ce-458c4810f869 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:56.029211 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-2c7f3cac-d1ca-440f-88d3-546000530be0 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:56.030446 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-2c7f3cac-d1ca-440f-88d3-546000530be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:56.057891 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e0559-199d-4866-9a00-be756de21826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:56.058571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0559-199d-4866-9a00-be756de21826 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:56.058856 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0559-199d-4866-9a00-be756de21826 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:56.059133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0559-199d-4866-9a00-be756de21826 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:56.059871 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e0559-199d-4866-9a00-be756de21826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:56.060177 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e0559-199d-4866-9a00-be756de21826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:56.062167 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e0559-199d-4866-9a00-be756de21826 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:56.074095 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 367/740] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:56.075622 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e0559-199d-4866-9a00-be756de21826 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:56.115066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 374/741] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:56.116015 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f3cac-d1ca-440f-88d3-546000530be0 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:56.162426 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:56.206888 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e0559-199d-4866-9a00-be756de21826 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:56.241595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f3cac-d1ca-440f-88d3-546000530be0 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:56.243038 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d8276-32df-4fc1-a8db-bf42d1cecc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:56.244058 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d8276-32df-4fc1-a8db-bf42d1cecc91 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:56.244430 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d8276-32df-4fc1-a8db-bf42d1cecc91 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:56.244753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d8276-32df-4fc1-a8db-bf42d1cecc91 None admin] PATH_INFO: `/v3/users/nova` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:56.253657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d8276-32df-4fc1-a8db-bf42d1cecc91 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:56.255117 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d8276-32df-4fc1-a8db-bf42d1cecc91 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:56.256983 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-b24d8276-32df-4fc1-a8db-bf42d1cecc91 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:56.269989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:56.277107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 368/742] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:07:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:56.278967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d8276-32df-4fc1-a8db-bf42d1cecc91 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:56.287564 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5afb10-e6d0-4daf-a447-a137021cb18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:56.288343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5afb10-e6d0-4daf-a447-a137021cb18b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:56.288714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5afb10-e6d0-4daf-a447-a137021cb18b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:56.289066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5afb10-e6d0-4daf-a447-a137021cb18b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:56.361264 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5afb10-e6d0-4daf-a447-a137021cb18b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:56.362744 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5afb10-e6d0-4daf-a447-a137021cb18b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:56.384985 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d8276-32df-4fc1-a8db-bf42d1cecc91 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:56.408148 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:56.419411 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684df6ea-0847-4983-ae60-9d11efaae364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:56.421197 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684df6ea-0847-4983-ae60-9d11efaae364 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:56.421197 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684df6ea-0847-4983-ae60-9d11efaae364 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:56.421452 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684df6ea-0847-4983-ae60-9d11efaae364 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:56.439215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684df6ea-0847-4983-ae60-9d11efaae364 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:56.440753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684df6ea-0847-4983-ae60-9d11efaae364 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-684df6ea-0847-4983-ae60-9d11efaae364 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:07:56.456141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:56.457299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 369/743] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:07:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:56.458378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684df6ea-0847-4983-ae60-9d11efaae364 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:56.481809 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 375/744] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:56.483876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5afb10-e6d0-4daf-a447-a137021cb18b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:56.498056 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:56.565520 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684df6ea-0847-4983-ae60-9d11efaae364 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:56.585187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5afb10-e6d0-4daf-a447-a137021cb18b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:56.598713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9f6fa-0f9b-4608-ad29-fba357570981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:56.599411 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9f6fa-0f9b-4608-ad29-fba357570981 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:56.599658 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9f6fa-0f9b-4608-ad29-fba357570981 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:56.599942 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa9f6fa-0f9b-4608-ad29-fba357570981 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:56.600701 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9f6fa-0f9b-4608-ad29-fba357570981 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:56.602067 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa9f6fa-0f9b-4608-ad29-fba357570981 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:56.621041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fbbe95-93de-467d-b56d-b9f19dba62d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:56.621706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbbe95-93de-467d-b56d-b9f19dba62d5 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:56.621985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbbe95-93de-467d-b56d-b9f19dba62d5 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:56.622263 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fbbe95-93de-467d-b56d-b9f19dba62d5 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:56.623085 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbbe95-93de-467d-b56d-b9f19dba62d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:56.623388 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbbe95-93de-467d-b56d-b9f19dba62d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:56.624852 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fbbe95-93de-467d-b56d-b9f19dba62d5 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:56.641523 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 376/745] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:07:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:56.642729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b297542-d1c6-491c-8980-d2617dac7207 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:56.643013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b297542-d1c6-491c-8980-d2617dac7207 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:56.643303 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b297542-d1c6-491c-8980-d2617dac7207 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:56.644574 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 377/746] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:07:56.649633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b297542-d1c6-491c-8980-d2617dac7207 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:56.703226 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 370/747] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:07:56 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:07:56.704376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9f6fa-0f9b-4608-ad29-fba357570981 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:56.724235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:56.759066 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:56.768625 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b297542-d1c6-491c-8980-d2617dac7207 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:56.809190 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fca7b2-f53f-4581-a764-af68e9231771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:56.809886 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fca7b2-f53f-4581-a764-af68e9231771 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:56.810371 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fca7b2-f53f-4581-a764-af68e9231771 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:56.810661 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fca7b2-f53f-4581-a764-af68e9231771 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:56.811531 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fca7b2-f53f-4581-a764-af68e9231771 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:56.811835 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fca7b2-f53f-4581-a764-af68e9231771 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:56.813347 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fca7b2-f53f-4581-a764-af68e9231771 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:56.828537 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa9f6fa-0f9b-4608-ad29-fba357570981 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:56.838294 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 378/748] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:07:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:56.838294 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fca7b2-f53f-4581-a764-af68e9231771 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:56.871113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421e905-6cbc-48b2-919c-5349cf766d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:56.872073 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421e905-6cbc-48b2-919c-5349cf766d06 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:56.872469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421e905-6cbc-48b2-919c-5349cf766d06 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:56.872858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421e905-6cbc-48b2-919c-5349cf766d06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:56.888192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421e905-6cbc-48b2-919c-5349cf766d06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:56.890566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421e905-6cbc-48b2-919c-5349cf766d06 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-5421e905-6cbc-48b2-919c-5349cf766d06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:56.909869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:56.911176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:56.912236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:56.912236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:56.912236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:56.912236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:56.912236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 371/749] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:56.912732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8daf7-2a2b-4a88-a8ee-5a465e5e090e None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:56.913058 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8daf7-2a2b-4a88-a8ee-5a465e5e090e None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:56.913342 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c8daf7-2a2b-4a88-a8ee-5a465e5e090e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:56.934632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-e6c8daf7-2a2b-4a88-a8ee-5a465e5e090e None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:56.935845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-e6c8daf7-2a2b-4a88-a8ee-5a465e5e090e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:56.962814 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fca7b2-f53f-4581-a764-af68e9231771 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:56.991551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:57.004847 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d03959-3417-46f9-a170-03c4c1f7f6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:57.008589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d03959-3417-46f9-a170-03c4c1f7f6d3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:57.008589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d03959-3417-46f9-a170-03c4c1f7f6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:57.008589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d03959-3417-46f9-a170-03c4c1f7f6d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:57.009336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 372/750] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:07:57.010149 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8daf7-2a2b-4a88-a8ee-5a465e5e090e None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:57.020634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d03959-3417-46f9-a170-03c4c1f7f6d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:57.022356 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d03959-3417-46f9-a170-03c4c1f7f6d3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-93d03959-3417-46f9-a170-03c4c1f7f6d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:57.042108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:57.043409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:57.046330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:57.046330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:57.046330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:57.046330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:57.046330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 379/751] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:57.046330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d03959-3417-46f9-a170-03c4c1f7f6d3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:57.051901 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:57.136467 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c8daf7-2a2b-4a88-a8ee-5a465e5e090e None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:57.158827 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d03959-3417-46f9-a170-03c4c1f7f6d3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:57.175282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6823d-f7fb-4b5d-bf84-eaa78e3c5352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:57.176113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6823d-f7fb-4b5d-bf84-eaa78e3c5352 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:57.176431 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6823d-f7fb-4b5d-bf84-eaa78e3c5352 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:57.176715 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea6823d-f7fb-4b5d-bf84-eaa78e3c5352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:57.196216 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dea5a-ee5b-4b63-9d48-17f4a97ab302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:57.197032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dea5a-ee5b-4b63-9d48-17f4a97ab302 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:57.197411 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dea5a-ee5b-4b63-9d48-17f4a97ab302 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:57.197777 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227dea5a-ee5b-4b63-9d48-17f4a97ab302 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:57.198512 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dea5a-ee5b-4b63-9d48-17f4a97ab302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:57.198827 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dea5a-ee5b-4b63-9d48-17f4a97ab302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:57.200221 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227dea5a-ee5b-4b63-9d48-17f4a97ab302 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:57.222245 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 380/752] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:57.222245 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dea5a-ee5b-4b63-9d48-17f4a97ab302 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:57.255492 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6823d-f7fb-4b5d-bf84-eaa78e3c5352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:57.256991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea6823d-f7fb-4b5d-bf84-eaa78e3c5352 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:57.262098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:57.285591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:57.332191 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227dea5a-ee5b-4b63-9d48-17f4a97ab302 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:57.363495 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 373/753] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:07:57 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:07:57.364801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6823d-f7fb-4b5d-bf84-eaa78e3c5352 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:57.374501 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1a579-ee8d-4353-847f-436d320ed43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:57.375173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a579-ee8d-4353-847f-436d320ed43b None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:57.375461 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a579-ee8d-4353-847f-436d320ed43b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:57.375742 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1a579-ee8d-4353-847f-436d320ed43b None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:57.376505 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1a579-ee8d-4353-847f-436d320ed43b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:57.377900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e1a579-ee8d-4353-847f-436d320ed43b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:57.452543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 381/754] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:07:57 2022] POST /identity/v3/services => generated 240 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:07:57.453554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1a579-ee8d-4353-847f-436d320ed43b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:57.488874 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea6823d-f7fb-4b5d-bf84-eaa78e3c5352 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:57.529779 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:57.563866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea170db2-d568-45b2-b19f-d62fb8a8d771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:57.565240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea170db2-d568-45b2-b19f-d62fb8a8d771 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:57.565728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea170db2-d568-45b2-b19f-d62fb8a8d771 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:57.566239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea170db2-d568-45b2-b19f-d62fb8a8d771 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:57.567779 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea170db2-d568-45b2-b19f-d62fb8a8d771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:57.568104 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea170db2-d568-45b2-b19f-d62fb8a8d771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:57.570155 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea170db2-d568-45b2-b19f-d62fb8a8d771 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:57.571634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:57.579199 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1a579-ee8d-4353-847f-436d320ed43b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:57.592009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 374/755] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:57.593867 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea170db2-d568-45b2-b19f-d62fb8a8d771 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:57.631031 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee480ec-a64b-49a7-8485-bb9aefd69278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:57.632042 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee480ec-a64b-49a7-8485-bb9aefd69278 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:57.632678 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee480ec-a64b-49a7-8485-bb9aefd69278 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:57.633042 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee480ec-a64b-49a7-8485-bb9aefd69278 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:57.644921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee480ec-a64b-49a7-8485-bb9aefd69278 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:57.646604 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee480ec-a64b-49a7-8485-bb9aefd69278 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-aee480ec-a64b-49a7-8485-bb9aefd69278 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:57.668779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:57.669888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:57.670953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:57.670953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:57.670953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:57.670953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:57.670953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:57.670953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:57.670953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:57.670953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:57.670953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 382/756] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:57.671661 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee480ec-a64b-49a7-8485-bb9aefd69278 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:57.729216 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea170db2-d568-45b2-b19f-d62fb8a8d771 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:57.769500 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcd392-4588-45fd-975a-66d88cf51607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:57.770436 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcd392-4588-45fd-975a-66d88cf51607 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:57.770647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcd392-4588-45fd-975a-66d88cf51607 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:57.770922 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcd392-4588-45fd-975a-66d88cf51607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:57.783074 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcd392-4588-45fd-975a-66d88cf51607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:57.784505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcd392-4588-45fd-975a-66d88cf51607 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-7cbcd392-4588-45fd-975a-66d88cf51607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:07:57.804026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:57.807305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:57.808422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:07:57.808422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:07:57.808422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:07:57.808422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:57.808422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 375/757] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:57.808422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcd392-4588-45fd-975a-66d88cf51607 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:57.810311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee480ec-a64b-49a7-8485-bb9aefd69278 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:57.847291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1575e-8509-42fb-ae30-afbafa554e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:57.850215 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1575e-8509-42fb-ae30-afbafa554e32 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:57.850215 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1575e-8509-42fb-ae30-afbafa554e32 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:57.850215 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1575e-8509-42fb-ae30-afbafa554e32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:57.854691 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:57.855946 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1575e-8509-42fb-ae30-afbafa554e32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:57.857666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1575e-8509-42fb-ae30-afbafa554e32 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-45e1575e-8509-42fb-ae30-afbafa554e32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:57.874150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:57.875546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:57.876846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:57.876846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:57.876846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:57.876846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:57.876846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:57.876846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:57.876846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:57.876846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:57.878519 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 383/758] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:57.879856 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1575e-8509-42fb-ae30-afbafa554e32 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:57.916778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:57.938012 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcd392-4588-45fd-975a-66d88cf51607 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:57.976140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1391990-edd5-40fd-b378-a372070287a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:57.977236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1391990-edd5-40fd-b378-a372070287a9 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:57.977236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1391990-edd5-40fd-b378-a372070287a9 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:57.977467 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1391990-edd5-40fd-b378-a372070287a9 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:57.978164 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1391990-edd5-40fd-b378-a372070287a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:57.978473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1391990-edd5-40fd-b378-a372070287a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:57.979933 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1391990-edd5-40fd-b378-a372070287a9 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:57.993533 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 376/759] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:57.994332 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1391990-edd5-40fd-b378-a372070287a9 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:58.017761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1575e-8509-42fb-ae30-afbafa554e32 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.056728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009f6c3-c3d0-4373-961b-295e3ae5f950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.058208 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009f6c3-c3d0-4373-961b-295e3ae5f950 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.058497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009f6c3-c3d0-4373-961b-295e3ae5f950 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.058775 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009f6c3-c3d0-4373-961b-295e3ae5f950 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.059515 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009f6c3-c3d0-4373-961b-295e3ae5f950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:58.059828 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009f6c3-c3d0-4373-961b-295e3ae5f950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:58.061908 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009f6c3-c3d0-4373-961b-295e3ae5f950 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:58.083137 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 384/760] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:58.083237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009f6c3-c3d0-4373-961b-295e3ae5f950 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:58.120669 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:58.129067 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1391990-edd5-40fd-b378-a372070287a9 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.165591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:58.169470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac08673-40e4-4c88-a62a-ca31dc74c2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.171490 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08673-40e4-4c88-a62a-ca31dc74c2e1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.172276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08673-40e4-4c88-a62a-ca31dc74c2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.173054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac08673-40e4-4c88-a62a-ca31dc74c2e1 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.174327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac08673-40e4-4c88-a62a-ca31dc74c2e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:58.175157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac08673-40e4-4c88-a62a-ca31dc74c2e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:58.176952 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac08673-40e4-4c88-a62a-ca31dc74c2e1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:58.190092 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009f6c3-c3d0-4373-961b-295e3ae5f950 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.199253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 377/761] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:58.200469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac08673-40e4-4c88-a62a-ca31dc74c2e1 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:58.232153 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8a41b-ecad-41d5-9a57-e0aa98d26b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.232944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8a41b-ecad-41d5-9a57-e0aa98d26b01 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.233752 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8a41b-ecad-41d5-9a57-e0aa98d26b01 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.234029 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8a41b-ecad-41d5-9a57-e0aa98d26b01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.234938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8a41b-ecad-41d5-9a57-e0aa98d26b01 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=7777c807131c4d9886c6aeff5f756711) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:58.235257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8a41b-ecad-41d5-9a57-e0aa98d26b01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:58.237808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8a41b-ecad-41d5-9a57-e0aa98d26b01 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:58.246664 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 385/762] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:58 2022] GET /identity/v3/role_assignments?user.id=7777c807131c4d9886c6aeff5f756711&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:58.247552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8a41b-ecad-41d5-9a57-e0aa98d26b01 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:58.403353 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac08673-40e4-4c88-a62a-ca31dc74c2e1 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.430497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8a41b-ecad-41d5-9a57-e0aa98d26b01 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.458760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1cba6-904f-4320-a1fd-3bfe2f166ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.464759 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1cba6-904f-4320-a1fd-3bfe2f166ee1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.465090 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1cba6-904f-4320-a1fd-3bfe2f166ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.465337 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1cba6-904f-4320-a1fd-3bfe2f166ee1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.471376 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e1cb6-fc12-49f6-b7ae-30d4293ab6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.473453 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e1cb6-fc12-49f6-b7ae-30d4293ab6fe None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.474284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e1cb6-fc12-49f6-b7ae-30d4293ab6fe None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.475076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e1cb6-fc12-49f6-b7ae-30d4293ab6fe None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/f6d44c613f524fe1804d6c9b65fe97ca/roles/ab7a63cec3a744848972fcbb89ecd0ca` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.479095 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e1cb6-fc12-49f6-b7ae-30d4293ab6fe None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=f6d44c613f524fe1804d6c9b65fe97ca, role_id=ab7a63cec3a744848972fcbb89ecd0ca)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:58.479517 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1cba6-904f-4320-a1fd-3bfe2f166ee1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:58.481030 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e1cba6-904f-4320-a1fd-3bfe2f166ee1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:58.482757 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e1cb6-fc12-49f6-b7ae-30d4293ab6fe None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-c0e1cba6-904f-4320-a1fd-3bfe2f166ee1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:07:58.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:58.507328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:58.509362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:07:58.509362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:07:58.509362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:07:58.509362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:07:58.509362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:07:58.509362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:07:58.509362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:07:58.509362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:07:58.509362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 378/763] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:07:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:58.509362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1cba6-904f-4320-a1fd-3bfe2f166ee1 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:58.529570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:58.563594 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:58.574584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 386/764] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:07:58 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/f6d44c613f524fe1804d6c9b65fe97ca/roles/ab7a63cec3a744848972fcbb89ecd0ca => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:07:58.709984 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1cba6-904f-4320-a1fd-3bfe2f166ee1 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:58.777055 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291abd4-ef62-4ba6-97f3-b74f933d7024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:58.778588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291abd4-ef62-4ba6-97f3-b74f933d7024 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:58.779316 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291abd4-ef62-4ba6-97f3-b74f933d7024 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:58.780020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291abd4-ef62-4ba6-97f3-b74f933d7024 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:58.781482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291abd4-ef62-4ba6-97f3-b74f933d7024 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:58.782221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291abd4-ef62-4ba6-97f3-b74f933d7024 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:58.784647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291abd4-ef62-4ba6-97f3-b74f933d7024 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:58.814416 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 379/765] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:58.820686 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e1cb6-fc12-49f6-b7ae-30d4293ab6fe None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:59.107264 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e1cb6-fc12-49f6-b7ae-30d4293ab6fe None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:59.185101 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e0f95-a4c3-421b-a91c-2f980b57a8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:59.186766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e0f95-a4c3-421b-a91c-2f980b57a8fb None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:59.187261 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e0f95-a4c3-421b-a91c-2f980b57a8fb None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:59.187749 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e0f95-a4c3-421b-a91c-2f980b57a8fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:59.189364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e0f95-a4c3-421b-a91c-2f980b57a8fb None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=3ed06928b2214d2cb10c89f095feb5cc) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:59.189926 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e0f95-a4c3-421b-a91c-2f980b57a8fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:59.194919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7680092-3eab-464b-9a44-e21eddcb6282 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:59.195413 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7680092-3eab-464b-9a44-e21eddcb6282 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:59.195898 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7680092-3eab-464b-9a44-e21eddcb6282 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:59.198378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 380/766] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:07:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:07:59.198738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e0f95-a4c3-421b-a91c-2f980b57a8fb None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:59.205364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82e9c-bcb3-4545-aedc-fe750268fef8 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:59.205791 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82e9c-bcb3-4545-aedc-fe750268fef8 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:59.206201 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82e9c-bcb3-4545-aedc-fe750268fef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:59.209926 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 387/767] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:07:58 2022] GET /identity/v3/role_assignments?user.id=3ed06928b2214d2cb10c89f095feb5cc&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 259 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:59.241123 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-32b82e9c-bcb3-4545-aedc-fe750268fef8 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:07:59.242610 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-32b82e9c-bcb3-4545-aedc-fe750268fef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:07:59.374156 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 381/768] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:07:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:07:59.406014 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e0f95-a4c3-421b-a91c-2f980b57a8fb None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:59.510710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:59.559425 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e0f95-a4c3-421b-a91c-2f980b57a8fb None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:59.617691 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8579b0-8157-4fc4-950c-1550eae0b041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:59.618563 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8579b0-8157-4fc4-950c-1550eae0b041 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:59.618803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8579b0-8157-4fc4-950c-1550eae0b041 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:59.619084 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8579b0-8157-4fc4-950c-1550eae0b041 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:59.619746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8579b0-8157-4fc4-950c-1550eae0b041 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:59.621175 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8579b0-8157-4fc4-950c-1550eae0b041 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-ab8579b0-8157-4fc4-950c-1550eae0b041 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:07:59.640254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:07:59.641508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:07:59.643813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 388/769] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 20:07:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:07:59.648154 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b82e9c-bcb3-4545-aedc-fe750268fef8 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:07:59.653755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:07:59.758784 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b82e9c-bcb3-4545-aedc-fe750268fef8 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:07:59.821674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0648f4-9edc-474a-a501-2ba7663176b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:07:59.822420 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0648f4-9edc-474a-a501-2ba7663176b1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:07:59.822704 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0648f4-9edc-474a-a501-2ba7663176b1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:07:59.822977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0648f4-9edc-474a-a501-2ba7663176b1 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:07:59.823696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0648f4-9edc-474a-a501-2ba7663176b1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:07:59.824007 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0648f4-9edc-474a-a501-2ba7663176b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:07:59.825424 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0648f4-9edc-474a-a501-2ba7663176b1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:07:59.836670 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 382/770] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:07:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:07:59.841918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8579b0-8157-4fc4-950c-1550eae0b041 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:00.021576 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8579b0-8157-4fc4-950c-1550eae0b041 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:00.060370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:00.080702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58aab3-10d4-4973-a882-a72d5bae5de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:00.081645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58aab3-10d4-4973-a882-a72d5bae5de2 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:00.082159 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58aab3-10d4-4973-a882-a72d5bae5de2 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:00.082592 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58aab3-10d4-4973-a882-a72d5bae5de2 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:00.083504 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58aab3-10d4-4973-a882-a72d5bae5de2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:00.083977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58aab3-10d4-4973-a882-a72d5bae5de2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:00.085583 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58aab3-10d4-4973-a882-a72d5bae5de2 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:00.104491 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0648f4-9edc-474a-a501-2ba7663176b1 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:00.107692 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 389/771] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:07:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:00.213156 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:00.256549 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0648f4-9edc-474a-a501-2ba7663176b1 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:00.294090 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d66b7b4-6a02-465e-91cb-4648526d3b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:00.294778 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66b7b4-6a02-465e-91cb-4648526d3b24 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:00.295065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66b7b4-6a02-465e-91cb-4648526d3b24 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:00.295366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66b7b4-6a02-465e-91cb-4648526d3b24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:00.296183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66b7b4-6a02-465e-91cb-4648526d3b24 None admin] RBAC: Adding query filter params (interface=public, service_id=49a84bdeea7e4e4f8b3403237c40ddab, region_id=RegionOne) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:00.296533 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66b7b4-6a02-465e-91cb-4648526d3b24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:00.298346 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d66b7b4-6a02-465e-91cb-4648526d3b24 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:00.307333 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 383/772] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:08:00 2022] GET /identity/v3/endpoints?service_id=49a84bdeea7e4e4f8b3403237c40ddab&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:00.313049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58aab3-10d4-4973-a882-a72d5bae5de2 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:00.390444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e650e46-16f7-4f20-b791-b01682806c25 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:00.390811 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e650e46-16f7-4f20-b791-b01682806c25 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:00.391110 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e650e46-16f7-4f20-b791-b01682806c25 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:00.392511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 384/773] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:08:00.398140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b93c5-7845-41b4-a9ef-ef9eb61102e0 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:00.398750 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b93c5-7845-41b4-a9ef-ef9eb61102e0 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:00.399298 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b93c5-7845-41b4-a9ef-ef9eb61102e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:00.428103 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-601b93c5-7845-41b4-a9ef-ef9eb61102e0 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:00.429606 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-601b93c5-7845-41b4-a9ef-ef9eb61102e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:00.438255 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58aab3-10d4-4973-a882-a72d5bae5de2 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:00.504984 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc9eb5-3192-44ba-b87d-af362eadcec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:00.505983 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc9eb5-3192-44ba-b87d-af362eadcec9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:00.506442 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc9eb5-3192-44ba-b87d-af362eadcec9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:00.506882 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc9eb5-3192-44ba-b87d-af362eadcec9 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:00.507763 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc9eb5-3192-44ba-b87d-af362eadcec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:00.510143 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc9eb5-3192-44ba-b87d-af362eadcec9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:00.519518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 390/774] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:08:00 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:00.581713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 385/775] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:00.616443 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc9eb5-3192-44ba-b87d-af362eadcec9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:00.704598 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:00.776749 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc9eb5-3192-44ba-b87d-af362eadcec9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:00.820658 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573a600-5972-4933-88e2-2f21f69ea615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:00.821585 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a600-5972-4933-88e2-2f21f69ea615 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:00.821867 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a600-5972-4933-88e2-2f21f69ea615 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:00.822141 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a600-5972-4933-88e2-2f21f69ea615 None admin] PATH_INFO: `/v3/services/network` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:00.822814 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573a600-5972-4933-88e2-2f21f69ea615 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:00.824369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573a600-5972-4933-88e2-2f21f69ea615 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-6573a600-5972-4933-88e2-2f21f69ea615 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:00.837728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:00.839322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:00.840268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 391/776] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:00 2022] GET /identity/v3/services/network => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:00.845250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b93c5-7845-41b4-a9ef-ef9eb61102e0 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:01.048740 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b213d-88cd-4175-bf9d-13ee4f900e37 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:01.049163 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b213d-88cd-4175-bf9d-13ee4f900e37 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:01.049334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b213d-88cd-4175-bf9d-13ee4f900e37 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:01.051001 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 392/777] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:01.056405 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34989e18-2d28-42a0-b583-d7759beae04a None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:01.056694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34989e18-2d28-42a0-b583-d7759beae04a None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:01.056973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34989e18-2d28-42a0-b583-d7759beae04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:01.092944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-34989e18-2d28-42a0-b583-d7759beae04a None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:01.094078 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-34989e18-2d28-42a0-b583-d7759beae04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:01.103636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:01.190632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b93c5-7845-41b4-a9ef-ef9eb61102e0 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:01.204979 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 393/778] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:01.236228 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34989e18-2d28-42a0-b583-d7759beae04a None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:01.239620 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84157703-4184-46cb-a0d8-2bec30893df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:01.244702 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84157703-4184-46cb-a0d8-2bec30893df4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:01.244992 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84157703-4184-46cb-a0d8-2bec30893df4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:01.245266 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84157703-4184-46cb-a0d8-2bec30893df4 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:01.247798 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84157703-4184-46cb-a0d8-2bec30893df4 None admin] RBAC: Adding query filter params (name=network) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:01.248114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84157703-4184-46cb-a0d8-2bec30893df4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:01.251371 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84157703-4184-46cb-a0d8-2bec30893df4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:01.268464 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 386/779] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:08:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:01.274449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84157703-4184-46cb-a0d8-2bec30893df4 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:01.338187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:01.425597 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34989e18-2d28-42a0-b583-d7759beae04a None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:01.431023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84157703-4184-46cb-a0d8-2bec30893df4 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:01.459417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:01.466039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11641d63-0872-4246-af1f-6a89621a7a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:01.466849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641d63-0872-4246-af1f-6a89621a7a0b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:01.467131 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641d63-0872-4246-af1f-6a89621a7a0b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:01.467990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11641d63-0872-4246-af1f-6a89621a7a0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:01.476456 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ac4fa-a6f4-4e45-8c0f-9f1779b2308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:01.477127 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ac4fa-a6f4-4e45-8c0f-9f1779b2308d None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:01.477405 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ac4fa-a6f4-4e45-8c0f-9f1779b2308d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:01.479002 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ac4fa-a6f4-4e45-8c0f-9f1779b2308d None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:01.479002 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ac4fa-a6f4-4e45-8c0f-9f1779b2308d None admin] RBAC: Adding query filter params (type=network) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:01.479002 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ac4fa-a6f4-4e45-8c0f-9f1779b2308d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:01.483917 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ac4fa-a6f4-4e45-8c0f-9f1779b2308d None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:01.488918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11641d63-0872-4246-af1f-6a89621a7a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-11641d63-0872-4246-af1f-6a89621a7a0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:01.497333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:01.498473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:01.500012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:01.502166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 387/780] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:08:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:01.502330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 394/781] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:08:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:01.508705 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ac4fa-a6f4-4e45-8c0f-9f1779b2308d None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:01.510059 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11641d63-0872-4246-af1f-6a89621a7a0b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:01.663406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ac4fa-a6f4-4e45-8c0f-9f1779b2308d None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:01.663787 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11641d63-0872-4246-af1f-6a89621a7a0b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:01.675898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:01.700928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afcd31-121b-4c5c-89ff-3ad12a386c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:01.701638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afcd31-121b-4c5c-89ff-3ad12a386c85 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:01.701832 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4db41-3eb8-4ced-846e-b265ba7e0480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:01.701971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afcd31-121b-4c5c-89ff-3ad12a386c85 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:01.702193 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afcd31-121b-4c5c-89ff-3ad12a386c85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:01.702959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4db41-3eb8-4ced-846e-b265ba7e0480 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:01.703159 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afcd31-121b-4c5c-89ff-3ad12a386c85 None admin] RBAC: Adding query filter params (interface=public, service_id=160701ff1af34958942426d1040a8154, region_id=RegionOne) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:01.703304 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afcd31-121b-4c5c-89ff-3ad12a386c85 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:01.703721 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4db41-3eb8-4ced-846e-b265ba7e0480 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:01.704147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4db41-3eb8-4ced-846e-b265ba7e0480 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:01.704761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afcd31-121b-4c5c-89ff-3ad12a386c85 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:01.705595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4db41-3eb8-4ced-846e-b265ba7e0480 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:01.706074 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4db41-3eb8-4ced-846e-b265ba7e0480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:01.707607 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4db41-3eb8-4ced-846e-b265ba7e0480 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:01.713772 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 395/782] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:08:01 2022] GET /identity/v3/endpoints?service_id=160701ff1af34958942426d1040a8154&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:01.715010 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66564a6b-c90b-469a-b173-6b0bcc873211 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:01.715301 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66564a6b-c90b-469a-b173-6b0bcc873211 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:01.715581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66564a6b-c90b-469a-b173-6b0bcc873211 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:01.720640 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 388/784] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:08:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:01.721692 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 396/784] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:01.722242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4db41-3eb8-4ced-846e-b265ba7e0480 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:01.723045 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ccf15-a7fb-4020-8067-6539503a6409 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:01.723338 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ccf15-a7fb-4020-8067-6539503a6409 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:01.723623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ccf15-a7fb-4020-8067-6539503a6409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:01.741790 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-042ccf15-a7fb-4020-8067-6539503a6409 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:01.742986 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-042ccf15-a7fb-4020-8067-6539503a6409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:01.782633 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:01.824595 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 397/785] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:01.825541 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ccf15-a7fb-4020-8067-6539503a6409 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:01.835428 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4db41-3eb8-4ced-846e-b265ba7e0480 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:01.872710 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961dd62-acfc-4d36-9164-5391e5521259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:01.874931 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961dd62-acfc-4d36-9164-5391e5521259 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:01.875471 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961dd62-acfc-4d36-9164-5391e5521259 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:01.875975 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961dd62-acfc-4d36-9164-5391e5521259 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:01.876921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961dd62-acfc-4d36-9164-5391e5521259 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:01.878721 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961dd62-acfc-4d36-9164-5391e5521259 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:01.885037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 389/786] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:08:01 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:01.886393 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961dd62-acfc-4d36-9164-5391e5521259 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:01.972099 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ccf15-a7fb-4020-8067-6539503a6409 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:01.974650 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:02.026009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4477a94-0e3c-4f55-bf8e-700e4ec3f72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.026824 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4477a94-0e3c-4f55-bf8e-700e4ec3f72a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.027109 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4477a94-0e3c-4f55-bf8e-700e4ec3f72a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.027391 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4477a94-0e3c-4f55-bf8e-700e4ec3f72a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.041091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4477a94-0e3c-4f55-bf8e-700e4ec3f72a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:02.042611 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4477a94-0e3c-4f55-bf8e-700e4ec3f72a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:02.042806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961dd62-acfc-4d36-9164-5391e5521259 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-d4477a94-0e3c-4f55-bf8e-700e4ec3f72a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:02.065353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:02.066502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:02.067508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:02.067508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:02.067508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:02.067508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:02.068094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 398/787] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:02.072901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4477a94-0e3c-4f55-bf8e-700e4ec3f72a None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.081220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:02.120065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5ed74-d698-4e7b-9a41-4afe1598d1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.120932 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5ed74-d698-4e7b-9a41-4afe1598d1a4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.121219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5ed74-d698-4e7b-9a41-4afe1598d1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.121501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5ed74-d698-4e7b-9a41-4afe1598d1a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.133469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5ed74-d698-4e7b-9a41-4afe1598d1a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-6ba5ed74-d698-4e7b-9a41-4afe1598d1a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:02.142406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:02.143503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:02.144515 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:02.147827 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 390/788] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:08:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:02.149536 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff35cf-78d4-4ab6-a65d-639fb8f9f955 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.149828 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff35cf-78d4-4ab6-a65d-639fb8f9f955 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.150109 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff35cf-78d4-4ab6-a65d-639fb8f9f955 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.151589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 391/789] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:02.157140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff35cf-78d4-4ab6-a65d-639fb8f9f955 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.214522 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4477a94-0e3c-4f55-bf8e-700e4ec3f72a None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.258903 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda2da6-b0b9-4246-95b9-69cceb159845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.259617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda2da6-b0b9-4246-95b9-69cceb159845 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.259901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda2da6-b0b9-4246-95b9-69cceb159845 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.260175 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda2da6-b0b9-4246-95b9-69cceb159845 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.260932 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda2da6-b0b9-4246-95b9-69cceb159845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:02.261238 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda2da6-b0b9-4246-95b9-69cceb159845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:02.262850 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda2da6-b0b9-4246-95b9-69cceb159845 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:02.276456 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 399/790] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:02.277831 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffe3df-28b8-451c-8dcd-8382e51c50a8 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.278126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffe3df-28b8-451c-8dcd-8382e51c50a8 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.278405 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffe3df-28b8-451c-8dcd-8382e51c50a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.290159 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff35cf-78d4-4ab6-a65d-639fb8f9f955 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.301697 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-feffe3df-28b8-451c-8dcd-8382e51c50a8 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:02.303080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-feffe3df-28b8-451c-8dcd-8382e51c50a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:02.315620 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:02.328874 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423a2a2-1a4b-4057-bd55-b5f0bd82d77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.329843 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423a2a2-1a4b-4057-bd55-b5f0bd82d77e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.329843 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423a2a2-1a4b-4057-bd55-b5f0bd82d77e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.330135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423a2a2-1a4b-4057-bd55-b5f0bd82d77e None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.330846 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423a2a2-1a4b-4057-bd55-b5f0bd82d77e None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:02.331160 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423a2a2-1a4b-4057-bd55-b5f0bd82d77e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:02.332568 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423a2a2-1a4b-4057-bd55-b5f0bd82d77e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:02.343327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 392/791] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:02.344288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423a2a2-1a4b-4057-bd55-b5f0bd82d77e None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.382016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 400/792] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:02.382948 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffe3df-28b8-451c-8dcd-8382e51c50a8 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.394196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:02.476275 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423a2a2-1a4b-4057-bd55-b5f0bd82d77e None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.496020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffe3df-28b8-451c-8dcd-8382e51c50a8 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.513547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388e785-7987-4aa5-8b49-77af9dce868f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.514505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388e785-7987-4aa5-8b49-77af9dce868f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.514654 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388e785-7987-4aa5-8b49-77af9dce868f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.514848 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388e785-7987-4aa5-8b49-77af9dce868f None admin] PATH_INFO: `/v3/users/swift` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.523640 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388e785-7987-4aa5-8b49-77af9dce868f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:02.525207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388e785-7987-4aa5-8b49-77af9dce868f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:02.538621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0614d-1fd5-4aec-8b54-724a9da35d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.539910 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0614d-1fd5-4aec-8b54-724a9da35d00 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.540367 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0614d-1fd5-4aec-8b54-724a9da35d00 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.540848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0614d-1fd5-4aec-8b54-724a9da35d00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-e388e785-7987-4aa5-8b49-77af9dce868f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:02.542194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:02.543445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:02.544994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:02.544994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:02.544994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:02.544994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:02.544994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:02.544994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:02.544994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:02.544994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:02.544994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:08:02.544994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:02.548626 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 393/793] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:02.549544 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388e785-7987-4aa5-8b49-77af9dce868f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.557252 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0614d-1fd5-4aec-8b54-724a9da35d00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:02.559084 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0614d-1fd5-4aec-8b54-724a9da35d00 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-61b0614d-1fd5-4aec-8b54-724a9da35d00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:02.585492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:02.586817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:02.588025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:02.588025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:02.588025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:02.588025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:02.591162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 401/794] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:02.592306 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0614d-1fd5-4aec-8b54-724a9da35d00 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.632928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:02.659996 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:02.667872 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388e785-7987-4aa5-8b49-77af9dce868f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.709246 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151c46e-07fc-47cc-969e-33478089f83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.710055 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151c46e-07fc-47cc-969e-33478089f83f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.710344 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151c46e-07fc-47cc-969e-33478089f83f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.710617 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151c46e-07fc-47cc-969e-33478089f83f None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.722101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151c46e-07fc-47cc-969e-33478089f83f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-d151c46e-07fc-47cc-969e-33478089f83f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:02.730336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:02.731534 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:08:02.734730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:02.736586 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 394/795] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:02.737510 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151c46e-07fc-47cc-969e-33478089f83f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.739682 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0614d-1fd5-4aec-8b54-724a9da35d00 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.797215 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f6a3b-9e01-4786-95b6-bae16f49f2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.798121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f6a3b-9e01-4786-95b6-bae16f49f2ec None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.800967 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f6a3b-9e01-4786-95b6-bae16f49f2ec None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.800967 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f6a3b-9e01-4786-95b6-bae16f49f2ec None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.800967 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f6a3b-9e01-4786-95b6-bae16f49f2ec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:02.800967 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f6a3b-9e01-4786-95b6-bae16f49f2ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:02.802558 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f6a3b-9e01-4786-95b6-bae16f49f2ec None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:02.828618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 402/796] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:02.829885 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f6a3b-9e01-4786-95b6-bae16f49f2ec None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.866472 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151c46e-07fc-47cc-969e-33478089f83f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.893588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:02.915560 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80c441-1e3f-4935-9cb7-4537744d2c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.916949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80c441-1e3f-4935-9cb7-4537744d2c0b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.917401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80c441-1e3f-4935-9cb7-4537744d2c0b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.917843 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80c441-1e3f-4935-9cb7-4537744d2c0b None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.918965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80c441-1e3f-4935-9cb7-4537744d2c0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:02.919504 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80c441-1e3f-4935-9cb7-4537744d2c0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:02.921689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80c441-1e3f-4935-9cb7-4537744d2c0b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:02.937377 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 395/797] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:02.938669 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80c441-1e3f-4935-9cb7-4537744d2c0b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:02.947235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f6a3b-9e01-4786-95b6-bae16f49f2ec None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:02.960983 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:02.993717 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf3627-1c51-4847-8d8e-940cdee4eacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:02.994643 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf3627-1c51-4847-8d8e-940cdee4eacb None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:02.994927 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf3627-1c51-4847-8d8e-940cdee4eacb None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:02.995204 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf3627-1c51-4847-8d8e-940cdee4eacb None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:02.995954 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf3627-1c51-4847-8d8e-940cdee4eacb None admin] RBAC: Adding query filter params (name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:02.996259 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf3627-1c51-4847-8d8e-940cdee4eacb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:02.998332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf3627-1c51-4847-8d8e-940cdee4eacb None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.015053 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 403/798] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:03.015053 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf3627-1c51-4847-8d8e-940cdee4eacb None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.062417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80c441-1e3f-4935-9cb7-4537744d2c0b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.114077 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c0996-885d-4e93-b1fc-cc5ba01f77dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.115352 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c0996-885d-4e93-b1fc-cc5ba01f77dc None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.116218 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c0996-885d-4e93-b1fc-cc5ba01f77dc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.116761 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c0996-885d-4e93-b1fc-cc5ba01f77dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.130726 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c0996-885d-4e93-b1fc-cc5ba01f77dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.132724 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c0996-885d-4e93-b1fc-cc5ba01f77dc None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.143183 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf3627-1c51-4847-8d8e-940cdee4eacb None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-4f3c0996-885d-4e93-b1fc-cc5ba01f77dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:03.157091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:03.158187 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:03.159216 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:03.159216 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:03.159216 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:03.159216 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:03.159216 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 396/799] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:03.159541 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c0996-885d-4e93-b1fc-cc5ba01f77dc None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.168734 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.193458 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35fb31-c412-4baa-b42e-e213118cf504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.194569 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35fb31-c412-4baa-b42e-e213118cf504 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.194852 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35fb31-c412-4baa-b42e-e213118cf504 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.195127 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35fb31-c412-4baa-b42e-e213118cf504 None admin] PATH_INFO: `/v3/users/nova` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.207964 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b35fb31-c412-4baa-b42e-e213118cf504 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.209619 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b35fb31-c412-4baa-b42e-e213118cf504 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-7b35fb31-c412-4baa-b42e-e213118cf504 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:03.237091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:03.238280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 404/800] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:03.239420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35fb31-c412-4baa-b42e-e213118cf504 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.258576 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.292225 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c0996-885d-4e93-b1fc-cc5ba01f77dc None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.335892 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8785d-a5a5-41b8-973d-3e8b305be361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.336673 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8785d-a5a5-41b8-973d-3e8b305be361 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.336963 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8785d-a5a5-41b8-973d-3e8b305be361 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.337243 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8785d-a5a5-41b8-973d-3e8b305be361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.354571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8785d-a5a5-41b8-973d-3e8b305be361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.356465 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8785d-a5a5-41b8-973d-3e8b305be361 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.365637 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35fb31-c412-4baa-b42e-e213118cf504 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-09d8785d-a5a5-41b8-973d-3e8b305be361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:03.376462 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:03.377480 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:03.378519 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:03.378519 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:03.378519 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:03.378519 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:03.381834 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 397/801] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:03.383013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8785d-a5a5-41b8-973d-3e8b305be361 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.402123 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fbb89-0a78-42fd-88cc-56a2e0a03914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.402796 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fbb89-0a78-42fd-88cc-56a2e0a03914 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.403116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fbb89-0a78-42fd-88cc-56a2e0a03914 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.403345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fbb89-0a78-42fd-88cc-56a2e0a03914 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.404112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fbb89-0a78-42fd-88cc-56a2e0a03914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:03.404386 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fbb89-0a78-42fd-88cc-56a2e0a03914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.405796 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fbb89-0a78-42fd-88cc-56a2e0a03914 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.418082 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 405/802] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:03.418743 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fbb89-0a78-42fd-88cc-56a2e0a03914 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.461405 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.496603 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8785d-a5a5-41b8-973d-3e8b305be361 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.522104 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.525237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fbb89-0a78-42fd-88cc-56a2e0a03914 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.535958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36780ccd-b5a4-42ef-a56b-f980204b2177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.536968 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36780ccd-b5a4-42ef-a56b-f980204b2177 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.537250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36780ccd-b5a4-42ef-a56b-f980204b2177 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.537679 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36780ccd-b5a4-42ef-a56b-f980204b2177 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.538516 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36780ccd-b5a4-42ef-a56b-f980204b2177 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:03.538978 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36780ccd-b5a4-42ef-a56b-f980204b2177 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.540366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36780ccd-b5a4-42ef-a56b-f980204b2177 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.558738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 398/803] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:03.560522 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fb9c5-c18a-43a5-a95e-bfe7e81242e5 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.560809 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fb9c5-c18a-43a5-a95e-bfe7e81242e5 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.561637 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fb9c5-c18a-43a5-a95e-bfe7e81242e5 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.563049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 399/804] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:03.563631 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fb9c5-c18a-43a5-a95e-bfe7e81242e5 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.571498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbadc7-6d7f-47a3-b11d-46458ed31800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.572655 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbadc7-6d7f-47a3-b11d-46458ed31800 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.573274 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbadc7-6d7f-47a3-b11d-46458ed31800 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.573616 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dbadc7-6d7f-47a3-b11d-46458ed31800 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.574453 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbadc7-6d7f-47a3-b11d-46458ed31800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:03.574872 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbadc7-6d7f-47a3-b11d-46458ed31800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.576457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dbadc7-6d7f-47a3-b11d-46458ed31800 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.593991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 406/805] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:03.594937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbadc7-6d7f-47a3-b11d-46458ed31800 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.668616 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fb9c5-c18a-43a5-a95e-bfe7e81242e5 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.698499 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.703195 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dbadc7-6d7f-47a3-b11d-46458ed31800 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:03.707569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c0661-49b7-4e3c-9916-7936cdad8867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.707569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c0661-49b7-4e3c-9916-7936cdad8867 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.707569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c0661-49b7-4e3c-9916-7936cdad8867 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.707569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c0661-49b7-4e3c-9916-7936cdad8867 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.714414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c0661-49b7-4e3c-9916-7936cdad8867 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.715858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c0661-49b7-4e3c-9916-7936cdad8867 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-c41c0661-49b7-4e3c-9916-7936cdad8867 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:03.730373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:03.731632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:03.733065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:03.733065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:03.733065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:03.733065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:03.733065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:03.733065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:03.733065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:03.733065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:03.736654 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 400/806] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:03.737976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bda071-2b52-4111-acd8-32e6bdbcdfb1 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.738267 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bda071-2b52-4111-acd8-32e6bdbcdfb1 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.738544 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bda071-2b52-4111-acd8-32e6bdbcdfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.744892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b27e1a-87ba-453c-8f24-9e78b446b7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:03.746650 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27e1a-87ba-453c-8f24-9e78b446b7bc None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:03.746942 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27e1a-87ba-453c-8f24-9e78b446b7bc None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:03.747500 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b27e1a-87ba-453c-8f24-9e78b446b7bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:03.756019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-17bda071-2b52-4111-acd8-32e6bdbcdfb1 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:03.757246 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-17bda071-2b52-4111-acd8-32e6bdbcdfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:03.796553 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.821360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b27e1a-87ba-453c-8f24-9e78b446b7bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:03.822825 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b27e1a-87ba-453c-8f24-9e78b446b7bc None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:03.860447 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 401/807] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:03.860447 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bda071-2b52-4111-acd8-32e6bdbcdfb1 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.939571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:03.945399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 407/808] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:08:03.946467 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b27e1a-87ba-453c-8f24-9e78b446b7bc None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:03.977865 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bda071-2b52-4111-acd8-32e6bdbcdfb1 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.015649 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438425d7-f187-4eb3-8b70-5f87f9cf4dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.016499 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438425d7-f187-4eb3-8b70-5f87f9cf4dae None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.016787 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438425d7-f187-4eb3-8b70-5f87f9cf4dae None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.017063 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438425d7-f187-4eb3-8b70-5f87f9cf4dae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.024573 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438425d7-f187-4eb3-8b70-5f87f9cf4dae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.026030 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438425d7-f187-4eb3-8b70-5f87f9cf4dae None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-438425d7-f187-4eb3-8b70-5f87f9cf4dae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:04.041629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:04.042741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 402/809] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:04.043796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438425d7-f187-4eb3-8b70-5f87f9cf4dae None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.060025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b27e1a-87ba-453c-8f24-9e78b446b7bc None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.081615 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:04.100094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8675e-1200-4722-807d-e4d8162f28ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.103551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8675e-1200-4722-807d-e4d8162f28ff None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.103551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8675e-1200-4722-807d-e4d8162f28ff None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.103551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8675e-1200-4722-807d-e4d8162f28ff None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.103551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8675e-1200-4722-807d-e4d8162f28ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:04.103551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8675e-1200-4722-807d-e4d8162f28ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.302956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8675e-1200-4722-807d-e4d8162f28ff None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.302956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 408/810] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:08:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:04.302956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8675e-1200-4722-807d-e4d8162f28ff None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.303469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438425d7-f187-4eb3-8b70-5f87f9cf4dae None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.303670 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:04.304234 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5ce09-c5dc-49f1-adf0-03ebfdada8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.304234 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5ce09-c5dc-49f1-adf0-03ebfdada8ba None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.304234 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5ce09-c5dc-49f1-adf0-03ebfdada8ba None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.304234 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5ce09-c5dc-49f1-adf0-03ebfdada8ba None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5ce09-c5dc-49f1-adf0-03ebfdada8ba None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5ce09-c5dc-49f1-adf0-03ebfdada8ba None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-eda5ce09-c5dc-49f1-adf0-03ebfdada8ba None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:04.304927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:04.307572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:04.310870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:04.312879 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:08:04.312879 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:04.312879 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 403/811] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:04.312879 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5ce09-c5dc-49f1-adf0-03ebfdada8ba None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.313376 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8675e-1200-4722-807d-e4d8162f28ff None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.313376 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebae4dd-0635-4183-8b47-27b697f3c187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.313376 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebae4dd-0635-4183-8b47-27b697f3c187 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.313376 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebae4dd-0635-4183-8b47-27b697f3c187 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.313842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebae4dd-0635-4183-8b47-27b697f3c187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.317530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebae4dd-0635-4183-8b47-27b697f3c187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.318965 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebae4dd-0635-4183-8b47-27b697f3c187 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-bebae4dd-0635-4183-8b47-27b697f3c187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:04.341581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:04.343525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:04.344450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:04.344450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:04.344450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:04.344450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:04.344450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 409/812] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:04.344450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebae4dd-0635-4183-8b47-27b697f3c187 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.361703 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:04.368597 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5ce09-c5dc-49f1-adf0-03ebfdada8ba None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.402574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965b255-c6d7-4551-a484-899e2705b585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.403250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965b255-c6d7-4551-a484-899e2705b585 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.403529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965b255-c6d7-4551-a484-899e2705b585 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.403806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965b255-c6d7-4551-a484-899e2705b585 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.404744 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965b255-c6d7-4551-a484-899e2705b585 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:04.405063 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965b255-c6d7-4551-a484-899e2705b585 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.406543 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965b255-c6d7-4551-a484-899e2705b585 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.425197 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 404/813] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:04.426163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965b255-c6d7-4551-a484-899e2705b585 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.450756 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebae4dd-0635-4183-8b47-27b697f3c187 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.461632 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:04.485710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf6e43-2c73-428c-9107-6193914f7081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.486600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf6e43-2c73-428c-9107-6193914f7081 None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.486889 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf6e43-2c73-428c-9107-6193914f7081 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.487166 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf6e43-2c73-428c-9107-6193914f7081 None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/7777c807131c4d9886c6aeff5f756711/roles/ab7a63cec3a744848972fcbb89ecd0ca` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.489181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf6e43-2c73-428c-9107-6193914f7081 None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=7777c807131c4d9886c6aeff5f756711, role_id=ab7a63cec3a744848972fcbb89ecd0ca)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.490630 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf6e43-2c73-428c-9107-6193914f7081 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.527489 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965b255-c6d7-4551-a484-899e2705b585 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.570664 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6190faf-072e-4338-91e9-bed63d9c7763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.571104 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 410/814] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:08:04 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/7777c807131c4d9886c6aeff5f756711/roles/ab7a63cec3a744848972fcbb89ecd0ca => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:08:04.571321 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6190faf-072e-4338-91e9-bed63d9c7763 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.571601 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6190faf-072e-4338-91e9-bed63d9c7763 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.571926 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6190faf-072e-4338-91e9-bed63d9c7763 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.572767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6190faf-072e-4338-91e9-bed63d9c7763 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:04.573073 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d396491-b60e-42ca-b3bd-b49addaa9a2f None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.573185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6190faf-072e-4338-91e9-bed63d9c7763 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.573254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d396491-b60e-42ca-b3bd-b49addaa9a2f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.573516 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d396491-b60e-42ca-b3bd-b49addaa9a2f None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.574495 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6190faf-072e-4338-91e9-bed63d9c7763 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.575040 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 411/815] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:04.577268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d396491-b60e-42ca-b3bd-b49addaa9a2f None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.595166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 405/816] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:04.596363 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6190faf-072e-4338-91e9-bed63d9c7763 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.619371 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:04.702944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d396491-b60e-42ca-b3bd-b49addaa9a2f None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.714840 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6190faf-072e-4338-91e9-bed63d9c7763 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.743375 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8778a056-ef53-4fba-85db-95321c484820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.744391 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778a056-ef53-4fba-85db-95321c484820 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.744852 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778a056-ef53-4fba-85db-95321c484820 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.745304 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8778a056-ef53-4fba-85db-95321c484820 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.746690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778a056-ef53-4fba-85db-95321c484820 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:04.747191 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778a056-ef53-4fba-85db-95321c484820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.748671 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8778a056-ef53-4fba-85db-95321c484820 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.755401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c600d14-433f-4b24-95a4-cdfa8978ea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.756200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c600d14-433f-4b24-95a4-cdfa8978ea50 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.756517 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c600d14-433f-4b24-95a4-cdfa8978ea50 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.757411 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c600d14-433f-4b24-95a4-cdfa8978ea50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.765750 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 412/817] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:04.768371 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0aec4-79a3-4a1f-8aff-77f87ef68355 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.768685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0aec4-79a3-4a1f-8aff-77f87ef68355 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.768970 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0aec4-79a3-4a1f-8aff-77f87ef68355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.770429 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c600d14-433f-4b24-95a4-cdfa8978ea50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.772316 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c600d14-433f-4b24-95a4-cdfa8978ea50 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.787959 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-19c0aec4-79a3-4a1f-8aff-77f87ef68355 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:04.789214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-19c0aec4-79a3-4a1f-8aff-77f87ef68355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-1c600d14-433f-4b24-95a4-cdfa8978ea50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:04.794799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:04.796158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:04.797289 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:04.797289 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:04.797289 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:04.797289 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:04.797289 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 406/818] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:04.799073 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c600d14-433f-4b24-95a4-cdfa8978ea50 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.816056 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:04.863622 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 413/819] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:04.864519 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0aec4-79a3-4a1f-8aff-77f87ef68355 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.879563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:04.915502 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c600d14-433f-4b24-95a4-cdfa8978ea50 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:04.954693 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727c328-b331-4f23-a9d2-51f9fd98b290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:04.955399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727c328-b331-4f23-a9d2-51f9fd98b290 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:04.955666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727c328-b331-4f23-a9d2-51f9fd98b290 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:04.955929 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727c328-b331-4f23-a9d2-51f9fd98b290 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:04.956692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727c328-b331-4f23-a9d2-51f9fd98b290 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:04.957003 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727c328-b331-4f23-a9d2-51f9fd98b290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:04.958406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727c328-b331-4f23-a9d2-51f9fd98b290 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:04.963506 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 407/820] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:04.964308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727c328-b331-4f23-a9d2-51f9fd98b290 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:04.978656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0aec4-79a3-4a1f-8aff-77f87ef68355 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:05.015724 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78139b-cde0-40d8-9909-26e183f4d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:05.016555 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78139b-cde0-40d8-9909-26e183f4d712 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:05.016842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78139b-cde0-40d8-9909-26e183f4d712 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:05.017124 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78139b-cde0-40d8-9909-26e183f4d712 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:05.024167 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78139b-cde0-40d8-9909-26e183f4d712 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:05.025629 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78139b-cde0-40d8-9909-26e183f4d712 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-5c78139b-cde0-40d8-9909-26e183f4d712 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:05.039976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:05.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:05.041551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:05.041551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:05.041551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:05.041551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:05.041551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:05.041551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:05.041551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:05.041551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:05.041551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 414/821] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:05.042502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78139b-cde0-40d8-9909-26e183f4d712 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:05.067129 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:05.083778 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727c328-b331-4f23-a9d2-51f9fd98b290 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:05.123014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:05.128495 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25258cc3-07b5-49e5-b7e8-9e16d5b055fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:05.129165 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25258cc3-07b5-49e5-b7e8-9e16d5b055fd None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:05.129446 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25258cc3-07b5-49e5-b7e8-9e16d5b055fd None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:05.129773 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25258cc3-07b5-49e5-b7e8-9e16d5b055fd None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:05.130494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25258cc3-07b5-49e5-b7e8-9e16d5b055fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:05.130802 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25258cc3-07b5-49e5-b7e8-9e16d5b055fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:05.132208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25258cc3-07b5-49e5-b7e8-9e16d5b055fd None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:05.145059 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 408/822] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:05.145878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25258cc3-07b5-49e5-b7e8-9e16d5b055fd None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:05.156828 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78139b-cde0-40d8-9909-26e183f4d712 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:05.192317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce2453-2a1a-4b2b-a805-e72e4c1d428d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:05.193153 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce2453-2a1a-4b2b-a805-e72e4c1d428d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:05.193442 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce2453-2a1a-4b2b-a805-e72e4c1d428d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:05.193721 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce2453-2a1a-4b2b-a805-e72e4c1d428d None admin] PATH_INFO: `/v3/services/network` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:05.194373 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce2453-2a1a-4b2b-a805-e72e4c1d428d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:05.195764 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce2453-2a1a-4b2b-a805-e72e4c1d428d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-09ce2453-2a1a-4b2b-a805-e72e4c1d428d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:05.208377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:05.209243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:08:05.210243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:05.214714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 415/823] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:05 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:05.215678 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce2453-2a1a-4b2b-a805-e72e4c1d428d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:05.264859 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25258cc3-07b5-49e5-b7e8-9e16d5b055fd None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:05.302184 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43657925-89c0-4cb7-9f74-e72419f6cc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:05.303268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43657925-89c0-4cb7-9f74-e72419f6cc09 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:05.303563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43657925-89c0-4cb7-9f74-e72419f6cc09 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:05.304136 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43657925-89c0-4cb7-9f74-e72419f6cc09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:05.304930 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43657925-89c0-4cb7-9f74-e72419f6cc09 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:05.307118 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43657925-89c0-4cb7-9f74-e72419f6cc09 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:05.309229 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:05.325492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce2453-2a1a-4b2b-a805-e72e4c1d428d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:05.360121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3f6a1-41ce-4607-b7d0-dc3b15717d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:05.360819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3f6a1-41ce-4607-b7d0-dc3b15717d7a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:05.361103 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3f6a1-41ce-4607-b7d0-dc3b15717d7a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:05.361378 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3f6a1-41ce-4607-b7d0-dc3b15717d7a None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:05.362163 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3f6a1-41ce-4607-b7d0-dc3b15717d7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:05.362473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3f6a1-41ce-4607-b7d0-dc3b15717d7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:05.363965 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3f6a1-41ce-4607-b7d0-dc3b15717d7a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:05.381144 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 416/824] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:05.381144 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3f6a1-41ce-4607-b7d0-dc3b15717d7a None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:05.482256 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3f6a1-41ce-4607-b7d0-dc3b15717d7a None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:05.492411 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 409/825] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:08:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:05.495591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43657925-89c0-4cb7-9f74-e72419f6cc09 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:05.539257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd749082-d88e-49d9-973b-2bcfb281b951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:05.540384 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd749082-d88e-49d9-973b-2bcfb281b951 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:05.540729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd749082-d88e-49d9-973b-2bcfb281b951 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:05.541113 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd749082-d88e-49d9-973b-2bcfb281b951 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:05.550015 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd749082-d88e-49d9-973b-2bcfb281b951 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:05.551639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd749082-d88e-49d9-973b-2bcfb281b951 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:05.556370 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:05.566826 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-cd749082-d88e-49d9-973b-2bcfb281b951 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:05.571112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:05.576222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:05.577649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:05.577649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:05.577649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:05.577649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:05.577649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:05.577649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:05.577649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:05.577649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:05.577649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 417/826] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:05.577649 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd749082-d88e-49d9-973b-2bcfb281b951 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:05.647278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43657925-89c0-4cb7-9f74-e72419f6cc09 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:05.694323 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5942a9d-f687-4ca7-b542-a4324e0cb6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:05.695316 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5942a9d-f687-4ca7-b542-a4324e0cb6c7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:05.695819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5942a9d-f687-4ca7-b542-a4324e0cb6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:05.696103 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5942a9d-f687-4ca7-b542-a4324e0cb6c7 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:05.696932 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5942a9d-f687-4ca7-b542-a4324e0cb6c7 None admin] RBAC: Adding query filter params (name=network) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:05.697253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5942a9d-f687-4ca7-b542-a4324e0cb6c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:05.698690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5942a9d-f687-4ca7-b542-a4324e0cb6c7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:05.703124 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd749082-d88e-49d9-973b-2bcfb281b951 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:05.705794 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 410/827] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:08:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:05.707507 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5942a9d-f687-4ca7-b542-a4324e0cb6c7 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:05.749306 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f3e2-eadc-4d89-bc9d-b113d9d42df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:05.750582 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f3e2-eadc-4d89-bc9d-b113d9d42df9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:05.751042 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f3e2-eadc-4d89-bc9d-b113d9d42df9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:05.751511 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568f3e2-eadc-4d89-bc9d-b113d9d42df9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:05.752616 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568f3e2-eadc-4d89-bc9d-b113d9d42df9 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=f6d44c613f524fe1804d6c9b65fe97ca) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:05.752941 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568f3e2-eadc-4d89-bc9d-b113d9d42df9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:05.754934 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568f3e2-eadc-4d89-bc9d-b113d9d42df9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:05.761453 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 418/828] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:08:05 2022] GET /identity/v3/role_assignments?user.id=f6d44c613f524fe1804d6c9b65fe97ca&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:05.762049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f3e2-eadc-4d89-bc9d-b113d9d42df9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:05.845183 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:05.892860 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568f3e2-eadc-4d89-bc9d-b113d9d42df9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:05.933090 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecf4c4-d952-4f24-bcdd-b4a510ec95a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:05.933694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecf4c4-d952-4f24-bcdd-b4a510ec95a8 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:05.933978 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecf4c4-d952-4f24-bcdd-b4a510ec95a8 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:05.934445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecf4c4-d952-4f24-bcdd-b4a510ec95a8 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:05.935182 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecf4c4-d952-4f24-bcdd-b4a510ec95a8 None admin] RBAC: Adding query filter params (type=network) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:05.935493 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecf4c4-d952-4f24-bcdd-b4a510ec95a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:05.936993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecf4c4-d952-4f24-bcdd-b4a510ec95a8 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:05.943533 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 419/829] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:08:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:05.948947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecf4c4-d952-4f24-bcdd-b4a510ec95a8 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:05.960747 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5942a9d-f687-4ca7-b542-a4324e0cb6c7 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:05.999988 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:06.034695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd1971-cf33-4dda-9f75-d4bc784e4240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:06.035725 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd1971-cf33-4dda-9f75-d4bc784e4240 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:06.036205 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd1971-cf33-4dda-9f75-d4bc784e4240 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:06.036713 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd1971-cf33-4dda-9f75-d4bc784e4240 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:06.037718 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd1971-cf33-4dda-9f75-d4bc784e4240 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:06.038222 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd1971-cf33-4dda-9f75-d4bc784e4240 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:06.039896 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd1971-cf33-4dda-9f75-d4bc784e4240 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:06.058181 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 411/830] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:06.064253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd1971-cf33-4dda-9f75-d4bc784e4240 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:06.073428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecf4c4-d952-4f24-bcdd-b4a510ec95a8 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:06.113372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:06.127823 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b8607-9e97-4783-8834-13f74396e9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:06.128527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b8607-9e97-4783-8834-13f74396e9d6 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:06.128815 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b8607-9e97-4783-8834-13f74396e9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:06.129097 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b8607-9e97-4783-8834-13f74396e9d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:06.129858 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b8607-9e97-4783-8834-13f74396e9d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:06.131368 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b8607-9e97-4783-8834-13f74396e9d6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:06.205825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd1971-cf33-4dda-9f75-d4bc784e4240 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:06.241962 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 420/831] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:08:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:06.263292 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b44e6-05d6-4437-997d-52e4f802cdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:06.264247 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b44e6-05d6-4437-997d-52e4f802cdfb None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:06.264566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b44e6-05d6-4437-997d-52e4f802cdfb None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:06.264899 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b44e6-05d6-4437-997d-52e4f802cdfb None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/3ed06928b2214d2cb10c89f095feb5cc/roles/afaf4273358148d3b0b9cac8dbeeba97` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:06.282862 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b44e6-05d6-4437-997d-52e4f802cdfb None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=3ed06928b2214d2cb10c89f095feb5cc, role_id=afaf4273358148d3b0b9cac8dbeeba97)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:06.284472 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b44e6-05d6-4437-997d-52e4f802cdfb None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:06.361005 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 412/832] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:08:06 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/3ed06928b2214d2cb10c89f095feb5cc/roles/afaf4273358148d3b0b9cac8dbeeba97 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:08:06.891422 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1cc8a-698d-4c46-b4fb-4356b00ffabc None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:06.891944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1cc8a-698d-4c46-b4fb-4356b00ffabc None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:06.892016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1cc8a-698d-4c46-b4fb-4356b00ffabc None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:06.893810 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 421/833] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:06.899889 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe173d5-f045-48f5-9a3c-5a63da929d08 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:06.900189 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe173d5-f045-48f5-9a3c-5a63da929d08 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:06.900514 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe173d5-f045-48f5-9a3c-5a63da929d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:06.917859 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-ebe173d5-f045-48f5-9a3c-5a63da929d08 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:06.919060 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-ebe173d5-f045-48f5-9a3c-5a63da929d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:06.933528 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:06.992752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 413/834] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:06 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:07.015929 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1cc8a-698d-4c46-b4fb-4356b00ffabc None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:07.135025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1cc8a-698d-4c46-b4fb-4356b00ffabc None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:07.172396 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba09470-4e1e-470a-8d20-7291c725923d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:07.173124 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09470-4e1e-470a-8d20-7291c725923d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:07.173413 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09470-4e1e-470a-8d20-7291c725923d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:07.173697 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba09470-4e1e-470a-8d20-7291c725923d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:07.184954 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba09470-4e1e-470a-8d20-7291c725923d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-aba09470-4e1e-470a-8d20-7291c725923d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:07.192861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:07.193713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:07.194469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:07.199554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 422/835] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:08:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:07.201096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe173d5-f045-48f5-9a3c-5a63da929d08 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:07.310172 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe173d5-f045-48f5-9a3c-5a63da929d08 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:07.347727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810027-6917-4b02-b625-4ee568c38645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:07.348438 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810027-6917-4b02-b625-4ee568c38645 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:07.348728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810027-6917-4b02-b625-4ee568c38645 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:07.349007 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810027-6917-4b02-b625-4ee568c38645 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:07.349760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef810027-6917-4b02-b625-4ee568c38645 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:07.350070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef810027-6917-4b02-b625-4ee568c38645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:07.351466 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef810027-6917-4b02-b625-4ee568c38645 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:07.364493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 414/836] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:08:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:07.366427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba09470-4e1e-470a-8d20-7291c725923d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:07.416578 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:07.474518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba09470-4e1e-470a-8d20-7291c725923d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:07.511509 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc78393-acb4-4b81-92b5-024614b1c622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:07.512299 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc78393-acb4-4b81-92b5-024614b1c622 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:07.512640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc78393-acb4-4b81-92b5-024614b1c622 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:07.512913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc78393-acb4-4b81-92b5-024614b1c622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:07.524021 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc78393-acb4-4b81-92b5-024614b1c622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:07.525753 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc78393-acb4-4b81-92b5-024614b1c622 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-9cc78393-acb4-4b81-92b5-024614b1c622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:07.544674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:07.546685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:07.548874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:07.548874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:07.548874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:07.548874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:07.548874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 423/837] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:07.553434 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810027-6917-4b02-b625-4ee568c38645 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:07.566694 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:07.660314 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810027-6917-4b02-b625-4ee568c38645 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:07.699465 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c176bc-4b41-4839-926e-14cd43298950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:07.700152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c176bc-4b41-4839-926e-14cd43298950 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:07.700469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c176bc-4b41-4839-926e-14cd43298950 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:07.700753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c176bc-4b41-4839-926e-14cd43298950 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:07.702108 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c176bc-4b41-4839-926e-14cd43298950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:07.702435 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c176bc-4b41-4839-926e-14cd43298950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:07.703822 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c176bc-4b41-4839-926e-14cd43298950 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:07.716892 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 415/838] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:07.722206 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc78393-acb4-4b81-92b5-024614b1c622 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:07.831185 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:07.838408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc78393-acb4-4b81-92b5-024614b1c622 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:07.889642 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bc1b3-815d-4d56-a848-d50f441c2159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:07.890512 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc1b3-815d-4d56-a848-d50f441c2159 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:07.890799 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc1b3-815d-4d56-a848-d50f441c2159 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:07.891072 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc1b3-815d-4d56-a848-d50f441c2159 None admin] PATH_INFO: `/v3/users/swift` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:07.900654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bc1b3-815d-4d56-a848-d50f441c2159 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:07.902138 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bc1b3-815d-4d56-a848-d50f441c2159 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-699bc1b3-815d-4d56-a848-d50f441c2159 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:07.917044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:07.919126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:07.921334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:07.921334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:07.921334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:07.921334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:07.921334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:07.921334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:07.921334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:07.921334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:07.921334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:08:07.921334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:07.924389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 424/839] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:08:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:07.925776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c176bc-4b41-4839-926e-14cd43298950 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:07.999492 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:08.030964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c176bc-4b41-4839-926e-14cd43298950 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:08.050278 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aa8ff-3fcd-4503-9616-817a856881d6 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.050907 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aa8ff-3fcd-4503-9616-817a856881d6 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.051382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aa8ff-3fcd-4503-9616-817a856881d6 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.053269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 425/840] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:08:08.059844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a53ae-e605-47d2-98b1-6b8d394615dd None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.060254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a53ae-e605-47d2-98b1-6b8d394615dd None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.060654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a53ae-e605-47d2-98b1-6b8d394615dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.071346 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47d39b-af9c-4709-888f-cc8ed5b1fc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:08.072217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47d39b-af9c-4709-888f-cc8ed5b1fc23 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.072677 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47d39b-af9c-4709-888f-cc8ed5b1fc23 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.073124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47d39b-af9c-4709-888f-cc8ed5b1fc23 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.074118 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47d39b-af9c-4709-888f-cc8ed5b1fc23 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:08.074576 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47d39b-af9c-4709-888f-cc8ed5b1fc23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:08.076158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a47d39b-af9c-4709-888f-cc8ed5b1fc23 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:08.079615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-808a53ae-e605-47d2-98b1-6b8d394615dd None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:08.081577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-808a53ae-e605-47d2-98b1-6b8d394615dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:08.100086 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 416/841] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:08:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:08.109572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47d39b-af9c-4709-888f-cc8ed5b1fc23 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:08.172322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 426/842] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:08.195371 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a53ae-e605-47d2-98b1-6b8d394615dd None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:08.251590 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a47d39b-af9c-4709-888f-cc8ed5b1fc23 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:08.290759 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:08.302728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cc022-9af5-44ae-900f-c7056f1ea563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:08.304194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cc022-9af5-44ae-900f-c7056f1ea563 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.304678 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cc022-9af5-44ae-900f-c7056f1ea563 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.307596 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cc022-9af5-44ae-900f-c7056f1ea563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.316127 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:08.335461 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a53ae-e605-47d2-98b1-6b8d394615dd None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:08.372310 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc66a8-7323-449d-8733-360241e27c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:08.374070 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc66a8-7323-449d-8733-360241e27c6e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.374583 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc66a8-7323-449d-8733-360241e27c6e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.375060 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc66a8-7323-449d-8733-360241e27c6e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.375978 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc66a8-7323-449d-8733-360241e27c6e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:08.377679 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc66a8-7323-449d-8733-360241e27c6e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-9cfc66a8-7323-449d-8733-360241e27c6e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:08.392154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:08.393251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:08:08.394022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:08.395164 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cc022-9af5-44ae-900f-c7056f1ea563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:08.396550 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cc022-9af5-44ae-900f-c7056f1ea563 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:08.397590 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 427/843] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:08:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:08.405131 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc66a8-7323-449d-8733-360241e27c6e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:08.511839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 417/844] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:08 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:08:08.513620 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245961-55e3-4856-9682-adcadefd19fd None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.514153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245961-55e3-4856-9682-adcadefd19fd None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.514635 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245961-55e3-4856-9682-adcadefd19fd None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.516423 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 418/845] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:08.517556 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb245961-55e3-4856-9682-adcadefd19fd None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:08.531265 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc66a8-7323-449d-8733-360241e27c6e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:08.547898 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:08.568201 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fb760-1f8c-4a8a-80c7-6ff93586fc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:08.568911 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fb760-1f8c-4a8a-80c7-6ff93586fc04 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.569202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fb760-1f8c-4a8a-80c7-6ff93586fc04 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.569481 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fb760-1f8c-4a8a-80c7-6ff93586fc04 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.570255 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fb760-1f8c-4a8a-80c7-6ff93586fc04 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:08.570569 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fb760-1f8c-4a8a-80c7-6ff93586fc04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:08.572241 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5fb760-1f8c-4a8a-80c7-6ff93586fc04 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:08.582369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 428/846] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:08:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:08.582369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8fc1-3b0b-4cea-85d3-31f20a6335e3 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.582369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8fc1-3b0b-4cea-85d3-31f20a6335e3 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.582369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe8fc1-3b0b-4cea-85d3-31f20a6335e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.597156 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-57fe8fc1-3b0b-4cea-85d3-31f20a6335e3 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:08.598280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-57fe8fc1-3b0b-4cea-85d3-31f20a6335e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:08.619650 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:08.636481 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb245961-55e3-4856-9682-adcadefd19fd None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:08.681669 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb1515-8ed5-4d38-a67a-f1719131fef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:08.682800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb1515-8ed5-4d38-a67a-f1719131fef7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.683122 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb1515-8ed5-4d38-a67a-f1719131fef7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.683356 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb1515-8ed5-4d38-a67a-f1719131fef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.692609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 429/847] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:08.692609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe8fc1-3b0b-4cea-85d3-31f20a6335e3 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:08.705154 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb1515-8ed5-4d38-a67a-f1719131fef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:08.707475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb1515-8ed5-4d38-a67a-f1719131fef7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-99eb1515-8ed5-4d38-a67a-f1719131fef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:08.728273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:08.729129 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:08.729893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:08.729893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:08.729893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:08.729893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:08.729893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 419/848] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:08.730657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb1515-8ed5-4d38-a67a-f1719131fef7 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:08.805358 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe8fc1-3b0b-4cea-85d3-31f20a6335e3 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:08.820021 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:08.840457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb1515-8ed5-4d38-a67a-f1719131fef7 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:08.843415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c41c8-3746-4821-b0f9-e199ce24076f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:08.844136 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c41c8-3746-4821-b0f9-e199ce24076f None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.844447 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c41c8-3746-4821-b0f9-e199ce24076f None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.844744 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c41c8-3746-4821-b0f9-e199ce24076f None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.845489 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c41c8-3746-4821-b0f9-e199ce24076f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:08.845795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c41c8-3746-4821-b0f9-e199ce24076f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:08.847231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c41c8-3746-4821-b0f9-e199ce24076f None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:08.858049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 430/849] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:08:08 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:08.858049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c41c8-3746-4821-b0f9-e199ce24076f None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:08.869971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:08.887913 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2db1353-bbce-4798-afd3-a1ad93b053fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:08.888781 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db1353-bbce-4798-afd3-a1ad93b053fe None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.889940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db1353-bbce-4798-afd3-a1ad93b053fe None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.889940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2db1353-bbce-4798-afd3-a1ad93b053fe None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.890600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db1353-bbce-4798-afd3-a1ad93b053fe None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:08.892035 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2db1353-bbce-4798-afd3-a1ad93b053fe None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-a2db1353-bbce-4798-afd3-a1ad93b053fe None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:08.905717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:08.906608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:08.907382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 420/850] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 20:08:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:08.908799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57287f-361f-4750-bc12-15655cf9b56c None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:08.908799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57287f-361f-4750-bc12-15655cf9b56c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:08.908952 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57287f-361f-4750-bc12-15655cf9b56c None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:08.910518 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 421/851] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:08.913444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57287f-361f-4750-bc12-15655cf9b56c None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:08.983878 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c41c8-3746-4821-b0f9-e199ce24076f None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:09.026569 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd93ab4-dc13-4e74-8c50-c17034dc927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:09.027551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd93ab4-dc13-4e74-8c50-c17034dc927a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.027973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd93ab4-dc13-4e74-8c50-c17034dc927a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.028413 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd93ab4-dc13-4e74-8c50-c17034dc927a None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.029301 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd93ab4-dc13-4e74-8c50-c17034dc927a None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:09.029803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd93ab4-dc13-4e74-8c50-c17034dc927a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:09.031192 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd93ab4-dc13-4e74-8c50-c17034dc927a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:09.033744 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57287f-361f-4750-bc12-15655cf9b56c None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:09.049656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 431/852] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:09.049656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5e8ee-5642-46ba-bcdb-860fa5cf5dc9 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.049656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5e8ee-5642-46ba-bcdb-860fa5cf5dc9 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.049656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf5e8ee-5642-46ba-bcdb-860fa5cf5dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.072364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-8cf5e8ee-5642-46ba-bcdb-860fa5cf5dc9 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:09.072364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-8cf5e8ee-5642-46ba-bcdb-860fa5cf5dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:09.082282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cbafa-c38a-4a0d-999f-27957424307a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:09.082893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cbafa-c38a-4a0d-999f-27957424307a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.083184 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cbafa-c38a-4a0d-999f-27957424307a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.083469 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cbafa-c38a-4a0d-999f-27957424307a None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.084200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cbafa-c38a-4a0d-999f-27957424307a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:09.084542 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cbafa-c38a-4a0d-999f-27957424307a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:09.085932 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cbafa-c38a-4a0d-999f-27957424307a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:09.091046 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 422/853] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:08:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:09.096844 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cbafa-c38a-4a0d-999f-27957424307a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:09.110580 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:09.143565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:09.156366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 432/854] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:09 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:09.156366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5e8ee-5642-46ba-bcdb-860fa5cf5dc9 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:09.226334 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cbafa-c38a-4a0d-999f-27957424307a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:09.263078 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76f782-d293-421b-bb46-150766d78afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:09.264604 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76f782-d293-421b-bb46-150766d78afa None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.264985 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76f782-d293-421b-bb46-150766d78afa None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.265327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76f782-d293-421b-bb46-150766d78afa None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.272820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76f782-d293-421b-bb46-150766d78afa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:09.274232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76f782-d293-421b-bb46-150766d78afa None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:09.277974 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf5e8ee-5642-46ba-bcdb-860fa5cf5dc9 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-cd76f782-d293-421b-bb46-150766d78afa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:09.288671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:09.289653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:09.290676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:09.290676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:09.290676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:09.290676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:09.290676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:09.290676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:09.290676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:09.290676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:09.294668 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 423/855] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:09.295517 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76f782-d293-421b-bb46-150766d78afa None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:09.319709 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ab73a-5994-47d4-b8f6-ce75acc7d82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:09.320414 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ab73a-5994-47d4-b8f6-ce75acc7d82c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.320699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ab73a-5994-47d4-b8f6-ce75acc7d82c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.320980 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ab73a-5994-47d4-b8f6-ce75acc7d82c None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.321691 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ab73a-5994-47d4-b8f6-ce75acc7d82c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:09.322000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ab73a-5994-47d4-b8f6-ce75acc7d82c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:09.323383 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1ab73a-5994-47d4-b8f6-ce75acc7d82c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:09.328697 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 433/856] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:08:09 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:09.329498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ab73a-5994-47d4-b8f6-ce75acc7d82c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:09.384869 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:09.407897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:09.422788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76f782-d293-421b-bb46-150766d78afa None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:09.455941 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ab73a-5994-47d4-b8f6-ce75acc7d82c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:09.463594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bc7f7-2547-478f-98e1-8a91c87fd520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:09.464582 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bc7f7-2547-478f-98e1-8a91c87fd520 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.464939 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bc7f7-2547-478f-98e1-8a91c87fd520 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.465292 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bc7f7-2547-478f-98e1-8a91c87fd520 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.477939 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bc7f7-2547-478f-98e1-8a91c87fd520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-282bc7f7-2547-478f-98e1-8a91c87fd520 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:09.486183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:09.487089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:09.487842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 424/857] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:08:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:09.493738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bc7f7-2547-478f-98e1-8a91c87fd520 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:09.507298 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef9308-bcc0-42f8-afd3-384b33e235dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:09.508234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef9308-bcc0-42f8-afd3-384b33e235dd None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.508797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef9308-bcc0-42f8-afd3-384b33e235dd None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.509216 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef9308-bcc0-42f8-afd3-384b33e235dd None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.510643 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef9308-bcc0-42f8-afd3-384b33e235dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:09.511310 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef9308-bcc0-42f8-afd3-384b33e235dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:09.513206 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef9308-bcc0-42f8-afd3-384b33e235dd None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:09.529113 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 434/858] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:08:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:09.536502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef9308-bcc0-42f8-afd3-384b33e235dd None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:09.627066 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bc7f7-2547-478f-98e1-8a91c87fd520 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:09.656437 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:09.667283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c710c-81d8-4490-85b5-3cf1fea4e413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:09.667959 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c710c-81d8-4490-85b5-3cf1fea4e413 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.668236 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c710c-81d8-4490-85b5-3cf1fea4e413 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.669547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c710c-81d8-4490-85b5-3cf1fea4e413 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.669547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c710c-81d8-4490-85b5-3cf1fea4e413 None admin] RBAC: Adding query filter params (name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:09.669704 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c710c-81d8-4490-85b5-3cf1fea4e413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:09.671503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c710c-81d8-4490-85b5-3cf1fea4e413 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:09.677588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef9308-bcc0-42f8-afd3-384b33e235dd None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:09.681656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 425/859] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:08:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:09.687744 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c710c-81d8-4490-85b5-3cf1fea4e413 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:09.703503 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:09.716284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff6e7a-9b2a-487e-b49b-e6ec387715a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:09.717312 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff6e7a-9b2a-487e-b49b-e6ec387715a4 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.720257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff6e7a-9b2a-487e-b49b-e6ec387715a4 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.720257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff6e7a-9b2a-487e-b49b-e6ec387715a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.720257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff6e7a-9b2a-487e-b49b-e6ec387715a4 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=7777c807131c4d9886c6aeff5f756711) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:09.720257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff6e7a-9b2a-487e-b49b-e6ec387715a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:09.721107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff6e7a-9b2a-487e-b49b-e6ec387715a4 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:09.736515 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 435/860] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:08:09 2022] GET /identity/v3/role_assignments?user.id=7777c807131c4d9886c6aeff5f756711&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:09.813661 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c710c-81d8-4490-85b5-3cf1fea4e413 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:09.861656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ade5df-b670-4568-83c4-ad00c56733d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:09.863113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade5df-b670-4568-83c4-ad00c56733d7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:09.863538 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade5df-b670-4568-83c4-ad00c56733d7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:09.863960 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ade5df-b670-4568-83c4-ad00c56733d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:09.882639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ade5df-b670-4568-83c4-ad00c56733d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:09.885188 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ade5df-b670-4568-83c4-ad00c56733d7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-39ade5df-b670-4568-83c4-ad00c56733d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:09.915934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:09.916830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:09.917657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:09.917657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:09.917657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:09.917657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:09.917657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 426/861] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:09.924330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff6e7a-9b2a-487e-b49b-e6ec387715a4 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:10.039819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff6e7a-9b2a-487e-b49b-e6ec387715a4 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:10.079039 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34865f03-1a85-4b4e-8db5-6e3e220fa241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:10.079725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34865f03-1a85-4b4e-8db5-6e3e220fa241 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:10.080108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34865f03-1a85-4b4e-8db5-6e3e220fa241 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:10.080247 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34865f03-1a85-4b4e-8db5-6e3e220fa241 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:10.081029 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34865f03-1a85-4b4e-8db5-6e3e220fa241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:10.081335 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34865f03-1a85-4b4e-8db5-6e3e220fa241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:10.082738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34865f03-1a85-4b4e-8db5-6e3e220fa241 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:10.096570 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 436/862] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:10.106616 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ade5df-b670-4568-83c4-ad00c56733d7 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:10.155973 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:10.234969 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ade5df-b670-4568-83c4-ad00c56733d7 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:10.271175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4560b-99aa-405c-ac2e-68275e080ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:10.272189 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4560b-99aa-405c-ac2e-68275e080ed0 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:10.272513 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4560b-99aa-405c-ac2e-68275e080ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:10.272793 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4560b-99aa-405c-ac2e-68275e080ed0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:10.281247 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4560b-99aa-405c-ac2e-68275e080ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:10.282711 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4560b-99aa-405c-ac2e-68275e080ed0 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-93f4560b-99aa-405c-ac2e-68275e080ed0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:10.298261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:10.299158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:10.300017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:10.300017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:10.300017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:10.300017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:10.300017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:10.300017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:10.300017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:10.300017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:10.300017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:08:10.300017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:10.300017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 427/863] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:08:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:10.305403 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34865f03-1a85-4b4e-8db5-6e3e220fa241 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:10.355660 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:10.411270 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34865f03-1a85-4b4e-8db5-6e3e220fa241 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:10.447100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef36d34-0d49-46a5-a508-843e4fa95e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:10.447768 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef36d34-0d49-46a5-a508-843e4fa95e87 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:10.448049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef36d34-0d49-46a5-a508-843e4fa95e87 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:10.448323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef36d34-0d49-46a5-a508-843e4fa95e87 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:10.449184 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef36d34-0d49-46a5-a508-843e4fa95e87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:10.449490 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef36d34-0d49-46a5-a508-843e4fa95e87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:10.450931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef36d34-0d49-46a5-a508-843e4fa95e87 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:10.469531 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 437/864] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:08:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:10.475197 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4560b-99aa-405c-ac2e-68275e080ed0 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:10.581547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4560b-99aa-405c-ac2e-68275e080ed0 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:10.583315 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:10.620127 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722a77d-370c-4eab-a151-721867b5a5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:10.620831 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722a77d-370c-4eab-a151-721867b5a5b1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:10.621107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722a77d-370c-4eab-a151-721867b5a5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:10.621382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722a77d-370c-4eab-a151-721867b5a5b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:10.701336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722a77d-370c-4eab-a151-721867b5a5b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:10.703403 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722a77d-370c-4eab-a151-721867b5a5b1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:10.808807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 428/865] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:10 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:08:10.814976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef36d34-0d49-46a5-a508-843e4fa95e87 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:10.921902 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef36d34-0d49-46a5-a508-843e4fa95e87 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:10.924570 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:10.964551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a992f-84fb-4aa9-8852-3d7a3878d6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:10.965601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a992f-84fb-4aa9-8852-3d7a3878d6ec None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:10.966071 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a992f-84fb-4aa9-8852-3d7a3878d6ec None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:10.966672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a992f-84fb-4aa9-8852-3d7a3878d6ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:10.977444 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a992f-84fb-4aa9-8852-3d7a3878d6ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:10.978919 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a992f-84fb-4aa9-8852-3d7a3878d6ec None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-128a992f-84fb-4aa9-8852-3d7a3878d6ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:10.997284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:10.998237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:10.999653 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:10.999653 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:10.999653 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:10.999653 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:10.999653 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 438/866] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:11.006127 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722a77d-370c-4eab-a151-721867b5a5b1 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:11.008918 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:11.127552 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722a77d-370c-4eab-a151-721867b5a5b1 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:11.163422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fe16b-ccf5-46da-af9b-b239614e1f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:11.164095 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fe16b-ccf5-46da-af9b-b239614e1f9c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:11.164414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fe16b-ccf5-46da-af9b-b239614e1f9c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:11.164696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fe16b-ccf5-46da-af9b-b239614e1f9c None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:11.165429 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fe16b-ccf5-46da-af9b-b239614e1f9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:11.165742 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fe16b-ccf5-46da-af9b-b239614e1f9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:11.167116 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fe16b-ccf5-46da-af9b-b239614e1f9c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:11.180146 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 429/867] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:11.185759 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a992f-84fb-4aa9-8852-3d7a3878d6ec None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:11.235537 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939322-69b3-4cff-8121-f0d565bb1aae None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:11.235875 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939322-69b3-4cff-8121-f0d565bb1aae None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:11.236108 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939322-69b3-4cff-8121-f0d565bb1aae None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:11.237900 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 430/868] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:11.242984 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514fb94-8061-4a28-a876-df292a9c7487 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:11.243276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514fb94-8061-4a28-a876-df292a9c7487 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:11.243559 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514fb94-8061-4a28-a876-df292a9c7487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:11.262182 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-e514fb94-8061-4a28-a876-df292a9c7487 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:11.263674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-e514fb94-8061-4a28-a876-df292a9c7487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:11.308790 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a992f-84fb-4aa9-8852-3d7a3878d6ec None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:11.327851 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:11.340560 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 431/869] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:11.354158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034557ab-a2d4-41bc-aab6-7845659cd3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:11.354956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034557ab-a2d4-41bc-aab6-7845659cd3a1 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:11.355244 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034557ab-a2d4-41bc-aab6-7845659cd3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:11.355521 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034557ab-a2d4-41bc-aab6-7845659cd3a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:11.363193 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034557ab-a2d4-41bc-aab6-7845659cd3a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:11.364739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034557ab-a2d4-41bc-aab6-7845659cd3a1 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:11.365938 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514fb94-8061-4a28-a876-df292a9c7487 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-034557ab-a2d4-41bc-aab6-7845659cd3a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:11.381030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:11.381899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:11.382718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:11.382718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:11.382718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:11.382718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:11.382718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:11.382718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:11.382718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:11.382718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:11.385280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 439/870] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:11.396360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034557ab-a2d4-41bc-aab6-7845659cd3a1 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:11.421364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:11.476581 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514fb94-8061-4a28-a876-df292a9c7487 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:11.511205 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034557ab-a2d4-41bc-aab6-7845659cd3a1 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:11.515386 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5016476a-7080-47e1-9c24-f7ac1346ccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:11.516264 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016476a-7080-47e1-9c24-f7ac1346ccd8 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:11.516732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016476a-7080-47e1-9c24-f7ac1346ccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:11.517284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5016476a-7080-47e1-9c24-f7ac1346ccd8 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:11.518690 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016476a-7080-47e1-9c24-f7ac1346ccd8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:11.521060 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5016476a-7080-47e1-9c24-f7ac1346ccd8 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:11.548518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6d1cc-7c64-4a5c-955d-5790e0490469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:11.549410 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6d1cc-7c64-4a5c-955d-5790e0490469 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:11.549859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6d1cc-7c64-4a5c-955d-5790e0490469 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:11.550405 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6d1cc-7c64-4a5c-955d-5790e0490469 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:11.551377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6d1cc-7c64-4a5c-955d-5790e0490469 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:11.551824 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6d1cc-7c64-4a5c-955d-5790e0490469 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:11.553510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6d1cc-7c64-4a5c-955d-5790e0490469 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:11.566730 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 440/871] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:08:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:11.572851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6d1cc-7c64-4a5c-955d-5790e0490469 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:11.608382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 432/872] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:08:11 2022] POST /identity/v3/services => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:08:11.677138 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:11.704635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6d1cc-7c64-4a5c-955d-5790e0490469 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:11.753090 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b3ba9-8084-4516-a155-d9969c26126c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:11.753684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b3ba9-8084-4516-a155-d9969c26126c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:11.753965 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b3ba9-8084-4516-a155-d9969c26126c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:11.754241 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b3ba9-8084-4516-a155-d9969c26126c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:11.755097 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b3ba9-8084-4516-a155-d9969c26126c None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=3ed06928b2214d2cb10c89f095feb5cc) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:11.755409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b3ba9-8084-4516-a155-d9969c26126c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:11.756927 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b3ba9-8084-4516-a155-d9969c26126c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:11.765708 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 441/873] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:08:11 2022] GET /identity/v3/role_assignments?user.id=3ed06928b2214d2cb10c89f095feb5cc&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:11.813808 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e071b2f-9e5b-4f83-9516-ff5d4031adbe None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:11.814123 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e071b2f-9e5b-4f83-9516-ff5d4031adbe None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:11.814374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e071b2f-9e5b-4f83-9516-ff5d4031adbe None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:11.816188 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 433/874] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:11.822084 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d39da-69d0-4bdf-ada4-b8735c8d6c09 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:11.822352 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d39da-69d0-4bdf-ada4-b8735c8d6c09 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:11.822632 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d39da-69d0-4bdf-ada4-b8735c8d6c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:11.843310 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-fc5d39da-69d0-4bdf-ada4-b8735c8d6c09 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:11.852020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-fc5d39da-69d0-4bdf-ada4-b8735c8d6c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:11.930435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 442/875] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:11 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:11.954854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e071b2f-9e5b-4f83-9516-ff5d4031adbe None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:12.018888 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:12.071621 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e071b2f-9e5b-4f83-9516-ff5d4031adbe None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:12.117657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe50a19-b3c2-4fd7-968e-3d11ca18517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:12.118569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe50a19-b3c2-4fd7-968e-3d11ca18517c None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:12.118998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe50a19-b3c2-4fd7-968e-3d11ca18517c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:12.119439 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe50a19-b3c2-4fd7-968e-3d11ca18517c None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:12.120362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe50a19-b3c2-4fd7-968e-3d11ca18517c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:12.121900 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe50a19-b3c2-4fd7-968e-3d11ca18517c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:12.122801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d0f33-774d-420d-a109-b3c040b91812 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:12.123096 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d0f33-774d-420d-a109-b3c040b91812 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:12.123412 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d0f33-774d-420d-a109-b3c040b91812 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:12.125020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 443/876] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:12.130286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543386c-8820-477e-b3f3-5c4b7adb6498 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:12.130579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543386c-8820-477e-b3f3-5c4b7adb6498 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:12.130862 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543386c-8820-477e-b3f3-5c4b7adb6498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:12.151238 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-b543386c-8820-477e-b3f3-5c4b7adb6498 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:12.152441 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-b543386c-8820-477e-b3f3-5c4b7adb6498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:12.222349 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:12.235020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 434/877] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:08:11 2022] POST /identity/v3/services => generated 269 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:12.248089 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 444/878] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:12.272028 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe50a19-b3c2-4fd7-968e-3d11ca18517c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:12.393882 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe50a19-b3c2-4fd7-968e-3d11ca18517c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:12.409873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:12.434123 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fafc9-8ce6-4d8c-9b5d-d935cc035d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:12.435278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fafc9-8ce6-4d8c-9b5d-d935cc035d34 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:12.436353 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fafc9-8ce6-4d8c-9b5d-d935cc035d34 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:12.436850 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fafc9-8ce6-4d8c-9b5d-d935cc035d34 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:12.437732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fafc9-8ce6-4d8c-9b5d-d935cc035d34 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:12.439347 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fafc9-8ce6-4d8c-9b5d-d935cc035d34 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-382fafc9-8ce6-4d8c-9b5d-d935cc035d34 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:12.454818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:12.455833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:12.456627 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:12.457708 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 435/879] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:08:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:12.463906 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543386c-8820-477e-b3f3-5c4b7adb6498 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:12.576275 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543386c-8820-477e-b3f3-5c4b7adb6498 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:12.612350 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19dacf-1f5c-4532-ac7b-467e2358829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:12.613407 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19dacf-1f5c-4532-ac7b-467e2358829a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:12.613694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19dacf-1f5c-4532-ac7b-467e2358829a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:12.613970 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19dacf-1f5c-4532-ac7b-467e2358829a None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:12.614680 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19dacf-1f5c-4532-ac7b-467e2358829a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:12.614990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19dacf-1f5c-4532-ac7b-467e2358829a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:12.616394 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19dacf-1f5c-4532-ac7b-467e2358829a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:12.621574 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 445/880] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:08:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:12.627589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fafc9-8ce6-4d8c-9b5d-d935cc035d34 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:12.735784 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fafc9-8ce6-4d8c-9b5d-d935cc035d34 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:12.773584 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056b98b-b5a2-4f05-99a8-fd729c552abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:12.774623 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056b98b-b5a2-4f05-99a8-fd729c552abe None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:12.775116 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056b98b-b5a2-4f05-99a8-fd729c552abe None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:12.775594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056b98b-b5a2-4f05-99a8-fd729c552abe None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:12.776543 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056b98b-b5a2-4f05-99a8-fd729c552abe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:12.777057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056b98b-b5a2-4f05-99a8-fd729c552abe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:12.778634 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056b98b-b5a2-4f05-99a8-fd729c552abe None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:12.784245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 436/881] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:08:12 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:13.942524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfa285-a7c2-406c-92b7-1877be6da817 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:13.943045 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfa285-a7c2-406c-92b7-1877be6da817 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:13.943121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfa285-a7c2-406c-92b7-1877be6da817 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:13.944761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 446/882] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:13.950500 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8f872-7f91-4132-ba12-a83105cb01e6 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:13.950917 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8f872-7f91-4132-ba12-a83105cb01e6 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:13.951089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8f872-7f91-4132-ba12-a83105cb01e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:13.968905 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-7ee8f872-7f91-4132-ba12-a83105cb01e6 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:13.970106 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-7ee8f872-7f91-4132-ba12-a83105cb01e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:14.006432 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:14.048516 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 437/883] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:14.065799 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfa285-a7c2-406c-92b7-1877be6da817 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:14.134602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:14.182853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfa285-a7c2-406c-92b7-1877be6da817 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:14.219993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937242b-5ccc-45a4-aac8-92f1d329df15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:14.220754 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937242b-5ccc-45a4-aac8-92f1d329df15 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:14.221041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937242b-5ccc-45a4-aac8-92f1d329df15 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:14.221323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937242b-5ccc-45a4-aac8-92f1d329df15 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:14.221989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937242b-5ccc-45a4-aac8-92f1d329df15 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:14.223389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937242b-5ccc-45a4-aac8-92f1d329df15 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-b937242b-5ccc-45a4-aac8-92f1d329df15 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:14.235921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:14.236931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:08:14.237687 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:14.241390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 447/884] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:08:14 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:14.243766 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8f872-7f91-4132-ba12-a83105cb01e6 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:14.258727 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9975d-e014-408e-9ca1-3526430ec8d4 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:14.259011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9975d-e014-408e-9ca1-3526430ec8d4 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:14.259296 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9975d-e014-408e-9ca1-3526430ec8d4 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:14.261005 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 448/885] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:08:14.269844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7580-aa20-4ee3-a593-0b7cdae32f32 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:14.270135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7580-aa20-4ee3-a593-0b7cdae32f32 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:14.270421 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce7580-aa20-4ee3-a593-0b7cdae32f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:14.291939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-56ce7580-aa20-4ee3-a593-0b7cdae32f32 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:14.293599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-56ce7580-aa20-4ee3-a593-0b7cdae32f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:14.357927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8f872-7f91-4132-ba12-a83105cb01e6 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:14.384374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 449/886] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:14.401094 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9dc7f-117c-4391-a53a-0a02c656b2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:14.402084 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9dc7f-117c-4391-a53a-0a02c656b2be None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:14.402554 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9dc7f-117c-4391-a53a-0a02c656b2be None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:14.403020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9dc7f-117c-4391-a53a-0a02c656b2be None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:14.403923 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9dc7f-117c-4391-a53a-0a02c656b2be None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:14.404493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9dc7f-117c-4391-a53a-0a02c656b2be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:14.406088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9dc7f-117c-4391-a53a-0a02c656b2be None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:14.408136 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce7580-aa20-4ee3-a593-0b7cdae32f32 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:14.418376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 438/887] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:08:14 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:14.425265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9dc7f-117c-4391-a53a-0a02c656b2be None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:14.471331 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:14.473635 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:14.536413 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce7580-aa20-4ee3-a593-0b7cdae32f32 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:14.553813 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9dc7f-117c-4391-a53a-0a02c656b2be None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:14.582095 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea1659-755e-4222-bef9-6f7b005a3004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:14.583095 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea1659-755e-4222-bef9-6f7b005a3004 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:14.583389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea1659-755e-4222-bef9-6f7b005a3004 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:14.583672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea1659-755e-4222-bef9-6f7b005a3004 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:14.601558 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea1659-755e-4222-bef9-6f7b005a3004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:14.603770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd746fb0-7092-4d54-ae22-37f6ec4931ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:14.604698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746fb0-7092-4d54-ae22-37f6ec4931ed None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:14.605227 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746fb0-7092-4d54-ae22-37f6ec4931ed None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:14.605704 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746fb0-7092-4d54-ae22-37f6ec4931ed None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:14.606619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd746fb0-7092-4d54-ae22-37f6ec4931ed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:14.607125 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd746fb0-7092-4d54-ae22-37f6ec4931ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:14.609293 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd746fb0-7092-4d54-ae22-37f6ec4931ed None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-34ea1659-755e-4222-bef9-6f7b005a3004 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:14.611330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:14.612202 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:14.613013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 450/888] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:08:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:14.619482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea1659-755e-4222-bef9-6f7b005a3004 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:14.620048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 439/889] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:08:14 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:14.624436 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca7fab-ddeb-4222-9e6f-f902f377096a None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:14.624436 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca7fab-ddeb-4222-9e6f-f902f377096a None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:14.624436 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca7fab-ddeb-4222-9e6f-f902f377096a None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:14.627358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 440/890] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:14.628112 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca7fab-ddeb-4222-9e6f-f902f377096a None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:14.735734 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea1659-755e-4222-bef9-6f7b005a3004 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:14.741036 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:14.746151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:14.746633 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca7fab-ddeb-4222-9e6f-f902f377096a None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:14.773628 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09278b0-6df8-4174-a05b-559b85ef486c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:14.774298 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09278b0-6df8-4174-a05b-559b85ef486c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:14.774575 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09278b0-6df8-4174-a05b-559b85ef486c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:14.774847 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09278b0-6df8-4174-a05b-559b85ef486c None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:14.775589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09278b0-6df8-4174-a05b-559b85ef486c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:14.775909 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09278b0-6df8-4174-a05b-559b85ef486c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:14.777361 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09278b0-6df8-4174-a05b-559b85ef486c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:14.792518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 451/891] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:08:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:14.792518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291fcdf-e7c0-4628-a776-2e782e3845c7 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:14.792518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291fcdf-e7c0-4628-a776-2e782e3845c7 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:14.792518 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5291fcdf-e7c0-4628-a776-2e782e3845c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:14.799644 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487db35-26ea-4945-961b-c48e11a9dc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:14.800319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487db35-26ea-4945-961b-c48e11a9dc1f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:14.808416 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487db35-26ea-4945-961b-c48e11a9dc1f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:14.808416 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487db35-26ea-4945-961b-c48e11a9dc1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:14.808416 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487db35-26ea-4945-961b-c48e11a9dc1f None admin] RBAC: Adding query filter params (interface=public, service_id=a2afbac3b99d4d9fbe44795bb2c860a3, region_id=RegionOne) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:14.808416 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487db35-26ea-4945-961b-c48e11a9dc1f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:14.808416 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6487db35-26ea-4945-961b-c48e11a9dc1f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:14.813927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 441/892] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:08:14 2022] GET /identity/v3/endpoints?service_id=a2afbac3b99d4d9fbe44795bb2c860a3&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:14.813927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487db35-26ea-4945-961b-c48e11a9dc1f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:14.814672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-5291fcdf-e7c0-4628-a776-2e782e3845c7 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:14.815842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-5291fcdf-e7c0-4628-a776-2e782e3845c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:14.892109 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 452/893] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:14.892830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291fcdf-e7c0-4628-a776-2e782e3845c7 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:14.951838 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487db35-26ea-4945-961b-c48e11a9dc1f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:14.990866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c38370-e957-41dc-b80c-aea74e924ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:14.991586 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c38370-e957-41dc-b80c-aea74e924ad8 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:14.991873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c38370-e957-41dc-b80c-aea74e924ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:14.992154 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c38370-e957-41dc-b80c-aea74e924ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.004093 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c38370-e957-41dc-b80c-aea74e924ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.005607 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c38370-e957-41dc-b80c-aea74e924ad8 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.017789 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5291fcdf-e7c0-4628-a776-2e782e3845c7 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.019488 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-e9c38370-e957-41dc-b80c-aea74e924ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:15.024575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:15.025389 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:15.026217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:15.026217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:15.026217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:15.026217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:15.026217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 442/894] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:15.027035 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c38370-e957-41dc-b80c-aea74e924ad8 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.043421 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:15.063612 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905043ee-b34d-499a-8884-7ec7b198205f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.064285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905043ee-b34d-499a-8884-7ec7b198205f None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.064594 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905043ee-b34d-499a-8884-7ec7b198205f None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.064873 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905043ee-b34d-499a-8884-7ec7b198205f None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.065565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905043ee-b34d-499a-8884-7ec7b198205f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.066940 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905043ee-b34d-499a-8884-7ec7b198205f None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.073173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 453/895] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:08:14 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:08:15.073996 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905043ee-b34d-499a-8884-7ec7b198205f None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.151057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c38370-e957-41dc-b80c-aea74e924ad8 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.192859 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c3abd6-1037-4139-a2dd-e4b41b8b6da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.193843 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3abd6-1037-4139-a2dd-e4b41b8b6da7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.194248 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3abd6-1037-4139-a2dd-e4b41b8b6da7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.194527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c3abd6-1037-4139-a2dd-e4b41b8b6da7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.195202 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3abd6-1037-4139-a2dd-e4b41b8b6da7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.196761 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c3abd6-1037-4139-a2dd-e4b41b8b6da7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.200429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905043ee-b34d-499a-8884-7ec7b198205f None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-62c3abd6-1037-4139-a2dd-e4b41b8b6da7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:15.210895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:15.211845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:15.212655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 443/896] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 20:08:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:15.214063 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0643e2b-dd86-403d-96c5-1df162303014 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.214063 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0643e2b-dd86-403d-96c5-1df162303014 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.214278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0643e2b-dd86-403d-96c5-1df162303014 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.215868 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 444/897] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:15.221212 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0643e2b-dd86-403d-96c5-1df162303014 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.249626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c3753-c87f-4cd5-a8f2-f30993bb2a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.250228 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c3753-c87f-4cd5-a8f2-f30993bb2a32 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.250514 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c3753-c87f-4cd5-a8f2-f30993bb2a32 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.250793 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c3753-c87f-4cd5-a8f2-f30993bb2a32 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.251530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c3753-c87f-4cd5-a8f2-f30993bb2a32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:15.251839 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c3753-c87f-4cd5-a8f2-f30993bb2a32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.253288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c3753-c87f-4cd5-a8f2-f30993bb2a32 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.268218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 454/898] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:15.270348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c9444-5f6f-4b61-9bc4-ce3b65e8beb7 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.270717 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c9444-5f6f-4b61-9bc4-ce3b65e8beb7 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.271065 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c9444-5f6f-4b61-9bc4-ce3b65e8beb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.289010 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-4e2c9444-5f6f-4b61-9bc4-ce3b65e8beb7 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:15.290018 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-4e2c9444-5f6f-4b61-9bc4-ce3b65e8beb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:15.314718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:15.325753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:15.351113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0643e2b-dd86-403d-96c5-1df162303014 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.370092 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 455/899] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:15 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:15.370092 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c9444-5f6f-4b61-9bc4-ce3b65e8beb7 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.394796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aefa82-b7f1-41a9-80e4-4d9bf44bd604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.395591 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aefa82-b7f1-41a9-80e4-4d9bf44bd604 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.395950 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aefa82-b7f1-41a9-80e4-4d9bf44bd604 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.396301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aefa82-b7f1-41a9-80e4-4d9bf44bd604 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.397123 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aefa82-b7f1-41a9-80e4-4d9bf44bd604 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:15.397517 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aefa82-b7f1-41a9-80e4-4d9bf44bd604 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.398974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aefa82-b7f1-41a9-80e4-4d9bf44bd604 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.404209 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 445/900] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:08:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:15.407409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aefa82-b7f1-41a9-80e4-4d9bf44bd604 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.486460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c9444-5f6f-4b61-9bc4-ce3b65e8beb7 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.517329 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aefa82-b7f1-41a9-80e4-4d9bf44bd604 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.523596 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dfe5e-0af7-4365-8e7c-3ce699094e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.524447 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dfe5e-0af7-4365-8e7c-3ce699094e1a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.524732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dfe5e-0af7-4365-8e7c-3ce699094e1a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.525012 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1dfe5e-0af7-4365-8e7c-3ce699094e1a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.532306 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dfe5e-0af7-4365-8e7c-3ce699094e1a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.533792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1dfe5e-0af7-4365-8e7c-3ce699094e1a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-9e1dfe5e-0af7-4365-8e7c-3ce699094e1a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:15.550322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:15.551397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:15.552830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:15.552830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:15.552830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:15.552830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:15.552830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:15.552830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:15.552830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:15.552830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:15.552830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:08:15.552830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:15.555176 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2f1ee-a2c2-43ec-9f1f-523cf4b3f228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.556173 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2f1ee-a2c2-43ec-9f1f-523cf4b3f228 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.556323 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 456/901] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:08:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:15.556610 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2f1ee-a2c2-43ec-9f1f-523cf4b3f228 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.556954 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2f1ee-a2c2-43ec-9f1f-523cf4b3f228 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.557051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dfe5e-0af7-4365-8e7c-3ce699094e1a None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.557809 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2f1ee-a2c2-43ec-9f1f-523cf4b3f228 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.559297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd2f1ee-a2c2-43ec-9f1f-523cf4b3f228 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.575706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:15.641836 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 446/902] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:08:15 2022] POST /identity/v3/services => generated 241 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:15.643075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2f1ee-a2c2-43ec-9f1f-523cf4b3f228 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.646109 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:15.674484 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1dfe5e-0af7-4365-8e7c-3ce699094e1a None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.735458 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a3c1a-0a07-4302-bd97-4ec4fc7003b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.736345 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a3c1a-0a07-4302-bd97-4ec4fc7003b4 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.736990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a3c1a-0a07-4302-bd97-4ec4fc7003b4 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.737140 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a3c1a-0a07-4302-bd97-4ec4fc7003b4 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.738070 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a3c1a-0a07-4302-bd97-4ec4fc7003b4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:15.738478 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a3c1a-0a07-4302-bd97-4ec4fc7003b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.740033 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a3c1a-0a07-4302-bd97-4ec4fc7003b4 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.751332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 457/903] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:08:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:15.752807 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a3c1a-0a07-4302-bd97-4ec4fc7003b4 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.796557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2f1ee-a2c2-43ec-9f1f-523cf4b3f228 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.845744 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06430f-cee6-4167-832b-b63e8fefaea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.848178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06430f-cee6-4167-832b-b63e8fefaea3 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.848178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06430f-cee6-4167-832b-b63e8fefaea3 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.848729 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06430f-cee6-4167-832b-b63e8fefaea3 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.849600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06430f-cee6-4167-832b-b63e8fefaea3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:15.849953 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06430f-cee6-4167-832b-b63e8fefaea3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.851433 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06430f-cee6-4167-832b-b63e8fefaea3 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.866048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:15.873858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 447/904] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:08:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:15.879492 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06430f-cee6-4167-832b-b63e8fefaea3 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.900238 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a3c1a-0a07-4302-bd97-4ec4fc7003b4 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:15.941234 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f34b54-e5ac-42e9-a966-5e71e5e6bb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:15.942435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f34b54-e5ac-42e9-a966-5e71e5e6bb0f None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:15.943000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f34b54-e5ac-42e9-a966-5e71e5e6bb0f None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:15.943504 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f34b54-e5ac-42e9-a966-5e71e5e6bb0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:15.944580 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f34b54-e5ac-42e9-a966-5e71e5e6bb0f None admin] RBAC: Adding query filter params (interface=public, service_id=185576f9ec8b4a0ebc5b0d3e957b55c5, region_id=RegionOne) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:15.945451 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f34b54-e5ac-42e9-a966-5e71e5e6bb0f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:15.947226 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f34b54-e5ac-42e9-a966-5e71e5e6bb0f None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:15.955225 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 458/905] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:08:15 2022] GET /identity/v3/endpoints?service_id=185576f9ec8b4a0ebc5b0d3e957b55c5&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:15.960909 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f34b54-e5ac-42e9-a966-5e71e5e6bb0f None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:15.965131 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:16.025370 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06430f-cee6-4167-832b-b63e8fefaea3 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:16.074280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a380d5d-508c-4114-9273-19163c5b0645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:16.075019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a380d5d-508c-4114-9273-19163c5b0645 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:16.075606 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a380d5d-508c-4114-9273-19163c5b0645 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:16.075958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a380d5d-508c-4114-9273-19163c5b0645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:16.087689 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f34b54-e5ac-42e9-a966-5e71e5e6bb0f None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:16.132601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65d9e8-ed6b-4b14-b7c3-f0d8b5c76885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:16.133376 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65d9e8-ed6b-4b14-b7c3-f0d8b5c76885 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:16.133764 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65d9e8-ed6b-4b14-b7c3-f0d8b5c76885 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:16.134444 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65d9e8-ed6b-4b14-b7c3-f0d8b5c76885 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:16.135399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65d9e8-ed6b-4b14-b7c3-f0d8b5c76885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:16.137070 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65d9e8-ed6b-4b14-b7c3-f0d8b5c76885 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:16.148453 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 459/906] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:08:15 2022] GET /identity/v3/services => generated 1497 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 20:08:16.173568 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a380d5d-508c-4114-9273-19163c5b0645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:16.175481 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a380d5d-508c-4114-9273-19163c5b0645 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:16.246387 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:16.295920 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 448/907] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:15 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:08:16.302309 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65d9e8-ed6b-4b14-b7c3-f0d8b5c76885 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:16.308148 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:16.417049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65d9e8-ed6b-4b14-b7c3-f0d8b5c76885 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:16.455750 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3a31-c04d-4798-a463-c1c6f5179f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:16.456761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3a31-c04d-4798-a463-c1c6f5179f86 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:16.457150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3a31-c04d-4798-a463-c1c6f5179f86 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:16.457528 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3a31-c04d-4798-a463-c1c6f5179f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:16.469210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3a31-c04d-4798-a463-c1c6f5179f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:16.470841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3a31-c04d-4798-a463-c1c6f5179f86 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-825b3a31-c04d-4798-a463-c1c6f5179f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:16.489497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:16.490524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:16.491527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:16.491527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:16.491527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:16.491527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:16.493213 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 460/908] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:16.498626 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a380d5d-508c-4114-9273-19163c5b0645 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:16.608524 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a380d5d-508c-4114-9273-19163c5b0645 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:16.649465 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61409f0f-601e-45fc-bb8a-59984c717e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:16.650185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61409f0f-601e-45fc-bb8a-59984c717e47 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:16.650478 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61409f0f-601e-45fc-bb8a-59984c717e47 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:16.650757 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61409f0f-601e-45fc-bb8a-59984c717e47 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:16.651493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61409f0f-601e-45fc-bb8a-59984c717e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:16.651805 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61409f0f-601e-45fc-bb8a-59984c717e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:16.653223 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61409f0f-601e-45fc-bb8a-59984c717e47 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:16.666801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 449/909] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:16.672827 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3a31-c04d-4798-a463-c1c6f5179f86 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:16.738638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:16.787122 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3a31-c04d-4798-a463-c1c6f5179f86 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:16.825554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d511f5b-e33f-4182-a7b4-4e8a3e423b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:16.827077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d511f5b-e33f-4182-a7b4-4e8a3e423b8d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:16.827363 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d511f5b-e33f-4182-a7b4-4e8a3e423b8d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:16.827637 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d511f5b-e33f-4182-a7b4-4e8a3e423b8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:16.839795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d511f5b-e33f-4182-a7b4-4e8a3e423b8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:16.841561 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d511f5b-e33f-4182-a7b4-4e8a3e423b8d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-3d511f5b-e33f-4182-a7b4-4e8a3e423b8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:16.856635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:16.857746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:16.858794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:16.858794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:16.858794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:16.858794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:16.858794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:16.858794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:16.858794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:16.858794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:16.858794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 461/910] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:16.864599 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61409f0f-601e-45fc-bb8a-59984c717e47 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:16.890407 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:16.978919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61409f0f-601e-45fc-bb8a-59984c717e47 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:17.017578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3427fe5-e489-45a5-b253-a01c2277ade1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:17.018522 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3427fe5-e489-45a5-b253-a01c2277ade1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:17.018961 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3427fe5-e489-45a5-b253-a01c2277ade1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:17.019391 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3427fe5-e489-45a5-b253-a01c2277ade1 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:17.020361 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3427fe5-e489-45a5-b253-a01c2277ade1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:17.020822 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3427fe5-e489-45a5-b253-a01c2277ade1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:17.022468 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3427fe5-e489-45a5-b253-a01c2277ade1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:17.036557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 450/911] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:08:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:17.043064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d511f5b-e33f-4182-a7b4-4e8a3e423b8d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:17.151617 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d511f5b-e33f-4182-a7b4-4e8a3e423b8d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:17.168670 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:17.188179 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb23d6-d634-43ce-923f-a0e5733080bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:17.188875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb23d6-d634-43ce-923f-a0e5733080bb None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:17.189161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb23d6-d634-43ce-923f-a0e5733080bb None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:17.189442 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb23d6-d634-43ce-923f-a0e5733080bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:17.190277 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb23d6-d634-43ce-923f-a0e5733080bb None admin] RBAC: Adding query filter params (role.id=07c8382535c74f80a514602a3dbe9a7f, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=3ed06928b2214d2cb10c89f095feb5cc) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:17.190594 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb23d6-d634-43ce-923f-a0e5733080bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:17.192055 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb23d6-d634-43ce-923f-a0e5733080bb None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:17.198269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 462/912] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:08:17 2022] GET /identity/v3/role_assignments?user.id=3ed06928b2214d2cb10c89f095feb5cc&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=07c8382535c74f80a514602a3dbe9a7f => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:17.436108 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce685199-41e5-42db-982e-f0ed047d6fe6 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:17.436787 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce685199-41e5-42db-982e-f0ed047d6fe6 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:17.437268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce685199-41e5-42db-982e-f0ed047d6fe6 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:17.439150 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 451/913] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:17.445007 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acf652-74bb-426a-8c01-941219136761 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:17.445542 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acf652-74bb-426a-8c01-941219136761 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:17.446179 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acf652-74bb-426a-8c01-941219136761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:17.463880 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-c8acf652-74bb-426a-8c01-941219136761 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:17.465431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-c8acf652-74bb-426a-8c01-941219136761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:17.549669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 463/914] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:17 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:17.572636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce685199-41e5-42db-982e-f0ed047d6fe6 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:17.664714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:17.686811 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce685199-41e5-42db-982e-f0ed047d6fe6 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:17.723457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62b315-04e9-4c37-8529-fc04a1771a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:17.724639 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62b315-04e9-4c37-8529-fc04a1771a71 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:17.725127 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62b315-04e9-4c37-8529-fc04a1771a71 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:17.725603 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e62b315-04e9-4c37-8529-fc04a1771a71 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:17.726503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62b315-04e9-4c37-8529-fc04a1771a71 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:17.728082 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e62b315-04e9-4c37-8529-fc04a1771a71 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-8e62b315-04e9-4c37-8529-fc04a1771a71 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:17.741140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:17.742071 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:08:17.742820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:17.750313 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 452/915] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:08:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:17.752489 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acf652-74bb-426a-8c01-941219136761 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:17.864530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acf652-74bb-426a-8c01-941219136761 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:17.879998 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:17.905376 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52357f-146b-4fcb-b746-cfe1c3d06f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:17.906053 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52357f-146b-4fcb-b746-cfe1c3d06f0c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:17.906338 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52357f-146b-4fcb-b746-cfe1c3d06f0c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:17.906620 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52357f-146b-4fcb-b746-cfe1c3d06f0c None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:17.907331 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52357f-146b-4fcb-b746-cfe1c3d06f0c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:17.907638 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52357f-146b-4fcb-b746-cfe1c3d06f0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:17.909108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52357f-146b-4fcb-b746-cfe1c3d06f0c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:17.919246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 464/916] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:08:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:17.921382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62b315-04e9-4c37-8529-fc04a1771a71 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:18.004641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c099558-546e-4557-8337-913121003825 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:18.005022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c099558-546e-4557-8337-913121003825 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:18.005424 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c099558-546e-4557-8337-913121003825 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:18.007324 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 465/917] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:18.012397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c404a-9e76-417d-9b79-a2bdb236e9b2 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:18.012698 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c404a-9e76-417d-9b79-a2bdb236e9b2 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:18.012988 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c404a-9e76-417d-9b79-a2bdb236e9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:18.031020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e62b315-04e9-4c37-8529-fc04a1771a71 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:18.032312 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-379c404a-9e76-417d-9b79-a2bdb236e9b2 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:18.034573 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-379c404a-9e76-417d-9b79-a2bdb236e9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:18.077147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b6f6a-40f6-4abf-9214-ad39faea10ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:18.081181 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6f6a-40f6-4abf-9214-ad39faea10ed None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:18.081181 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6f6a-40f6-4abf-9214-ad39faea10ed None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:18.081433 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6f6a-40f6-4abf-9214-ad39faea10ed None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:18.082138 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b6f6a-40f6-4abf-9214-ad39faea10ed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:18.082458 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b6f6a-40f6-4abf-9214-ad39faea10ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:18.083864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06b6f6a-40f6-4abf-9214-ad39faea10ed None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:18.091571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 453/918] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:08:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:18.097216 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b6f6a-40f6-4abf-9214-ad39faea10ed None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:18.110202 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:18.128763 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 466/919] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:18.146359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c404a-9e76-417d-9b79-a2bdb236e9b2 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:18.210687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b6f6a-40f6-4abf-9214-ad39faea10ed None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:18.249470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c54b54-98f8-4c34-b779-b95f1750ba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:18.250255 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c54b54-98f8-4c34-b779-b95f1750ba43 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:18.250548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c54b54-98f8-4c34-b779-b95f1750ba43 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:18.250824 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c54b54-98f8-4c34-b779-b95f1750ba43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:18.251577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c54b54-98f8-4c34-b779-b95f1750ba43 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:18.253106 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c54b54-98f8-4c34-b779-b95f1750ba43 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:18.259640 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c404a-9e76-417d-9b79-a2bdb236e9b2 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:18.262504 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:18.295115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8b2be-25f2-494b-8f4d-2359938752a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:18.295955 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8b2be-25f2-494b-8f4d-2359938752a3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:18.296245 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8b2be-25f2-494b-8f4d-2359938752a3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:18.296557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8b2be-25f2-494b-8f4d-2359938752a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:18.297484 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8b2be-25f2-494b-8f4d-2359938752a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:18.298955 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8b2be-25f2-494b-8f4d-2359938752a3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-a8a8b2be-25f2-494b-8f4d-2359938752a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:18.311995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:18.318557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:18.319377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 467/920] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:08:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:18.320111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8b2be-25f2-494b-8f4d-2359938752a3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:18.384473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 454/921] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:08:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:18.445411 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8b2be-25f2-494b-8f4d-2359938752a3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:18.496853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760b7ef-4135-45ac-bdef-07aac0c8c197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:18.497921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760b7ef-4135-45ac-bdef-07aac0c8c197 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:18.498958 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760b7ef-4135-45ac-bdef-07aac0c8c197 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:18.499443 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760b7ef-4135-45ac-bdef-07aac0c8c197 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:18.500510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760b7ef-4135-45ac-bdef-07aac0c8c197 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:18.501078 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760b7ef-4135-45ac-bdef-07aac0c8c197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:18.502835 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760b7ef-4135-45ac-bdef-07aac0c8c197 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:18.509805 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 468/922] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:08:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:18.516254 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c54b54-98f8-4c34-b779-b95f1750ba43 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:18.598945 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:18.647172 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c54b54-98f8-4c34-b779-b95f1750ba43 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:18.684779 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2110c96-3ac9-4b29-a677-c5674a5a8d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:18.685459 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2110c96-3ac9-4b29-a677-c5674a5a8d14 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:18.686308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2110c96-3ac9-4b29-a677-c5674a5a8d14 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:18.686595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2110c96-3ac9-4b29-a677-c5674a5a8d14 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:18.687313 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2110c96-3ac9-4b29-a677-c5674a5a8d14 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:18.687623 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2110c96-3ac9-4b29-a677-c5674a5a8d14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:18.689038 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2110c96-3ac9-4b29-a677-c5674a5a8d14 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:18.694697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 455/923] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:08:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:18.700506 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760b7ef-4135-45ac-bdef-07aac0c8c197 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:18.711965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c33d33-a873-4770-bea5-e5ff07b8f084 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:18.712240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c33d33-a873-4770-bea5-e5ff07b8f084 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:18.712555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c33d33-a873-4770-bea5-e5ff07b8f084 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:18.714082 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 456/924] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:18.719706 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0feead4-260e-4e28-87a8-528cc564934d None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:18.720329 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0feead4-260e-4e28-87a8-528cc564934d None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:18.720861 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0feead4-260e-4e28-87a8-528cc564934d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:18.735751 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:18.740496 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-b0feead4-260e-4e28-87a8-528cc564934d None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:18.741927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-b0feead4-260e-4e28-87a8-528cc564934d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:18.822371 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760b7ef-4135-45ac-bdef-07aac0c8c197 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:18.840359 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 457/925] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:18 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:18.858150 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0feead4-260e-4e28-87a8-528cc564934d None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:18.875461 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4329d55-b92e-467d-ac6e-495985de9644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:18.876491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4329d55-b92e-467d-ac6e-495985de9644 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:18.876914 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4329d55-b92e-467d-ac6e-495985de9644 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:18.877605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4329d55-b92e-467d-ac6e-495985de9644 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:18.879053 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4329d55-b92e-467d-ac6e-495985de9644 None admin] RBAC: Adding query filter params (interface=public, service_id=114388ac41414ab2871c06099f5823df, region_id=RegionOne) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:18.879516 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4329d55-b92e-467d-ac6e-495985de9644 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:18.881337 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4329d55-b92e-467d-ac6e-495985de9644 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:18.889006 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 469/926] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:08:18 2022] GET /identity/v3/endpoints?service_id=114388ac41414ab2871c06099f5823df&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:18.894989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4329d55-b92e-467d-ac6e-495985de9644 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:18.933265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:18.983752 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0feead4-260e-4e28-87a8-528cc564934d None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:19.005207 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:19.016748 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4329d55-b92e-467d-ac6e-495985de9644 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:19.025532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac453e-eb9d-4160-a295-0d6a4e4b4c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:19.027155 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac453e-eb9d-4160-a295-0d6a4e4b4c97 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:19.027651 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac453e-eb9d-4160-a295-0d6a4e4b4c97 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:19.028473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac453e-eb9d-4160-a295-0d6a4e4b4c97 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:19.029750 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac453e-eb9d-4160-a295-0d6a4e4b4c97 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:19.031989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac453e-eb9d-4160-a295-0d6a4e4b4c97 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-16ac453e-eb9d-4160-a295-0d6a4e4b4c97 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:19.052288 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:19.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:08:19.054376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:19.056006 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a29e30-59f5-470e-928a-676c96acd7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:19.056819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29e30-59f5-470e-928a-676c96acd7d9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:19.057471 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 458/927] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 20:08:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:19.062563 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29e30-59f5-470e-928a-676c96acd7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:19.062969 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a29e30-59f5-470e-928a-676c96acd7d9 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:19.063757 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a29e30-59f5-470e-928a-676c96acd7d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:19.065318 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a29e30-59f5-470e-928a-676c96acd7d9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:19.065961 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac453e-eb9d-4160-a295-0d6a4e4b4c97 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:19.079091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 470/928] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:08:18 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:08:19.197644 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac453e-eb9d-4160-a295-0d6a4e4b4c97 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:19.224651 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:19.237981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a25ec-e6ed-4854-bed8-1cfbe3ad6cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:19.238666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a25ec-e6ed-4854-bed8-1cfbe3ad6cc4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:19.238949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a25ec-e6ed-4854-bed8-1cfbe3ad6cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:19.249096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a25ec-e6ed-4854-bed8-1cfbe3ad6cc4 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:19.249096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a25ec-e6ed-4854-bed8-1cfbe3ad6cc4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:19.249096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a25ec-e6ed-4854-bed8-1cfbe3ad6cc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:19.249096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a25ec-e6ed-4854-bed8-1cfbe3ad6cc4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:19.249096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 459/929] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:08:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:19.254124 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a29e30-59f5-470e-928a-676c96acd7d9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:19.371727 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a29e30-59f5-470e-928a-676c96acd7d9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:19.413401 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfad28d-67a8-40b5-91d8-7f4c83356bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:19.414104 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad28d-67a8-40b5-91d8-7f4c83356bb0 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:19.414528 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad28d-67a8-40b5-91d8-7f4c83356bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:19.414849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad28d-67a8-40b5-91d8-7f4c83356bb0 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:19.415603 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfad28d-67a8-40b5-91d8-7f4c83356bb0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:19.415950 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfad28d-67a8-40b5-91d8-7f4c83356bb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:19.417510 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfad28d-67a8-40b5-91d8-7f4c83356bb0 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:19.426110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 471/930] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:08:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:19.431877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a25ec-e6ed-4854-bed8-1cfbe3ad6cc4 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:19.539520 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a25ec-e6ed-4854-bed8-1cfbe3ad6cc4 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:19.577576 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677aa5d-32ee-4f7d-896f-fd4b278bac7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:19.578255 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677aa5d-32ee-4f7d-896f-fd4b278bac7b None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:19.578549 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677aa5d-32ee-4f7d-896f-fd4b278bac7b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:19.578832 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677aa5d-32ee-4f7d-896f-fd4b278bac7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:19.579590 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677aa5d-32ee-4f7d-896f-fd4b278bac7b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:19.581188 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677aa5d-32ee-4f7d-896f-fd4b278bac7b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:19.582921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f382d3b-67ad-4545-95aa-2a72aece0b02 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:19.583207 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f382d3b-67ad-4545-95aa-2a72aece0b02 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:19.583505 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f382d3b-67ad-4545-95aa-2a72aece0b02 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:19.585173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 472/931] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:19.590314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e50d74-67d2-48fd-bcd7-271f006b2706 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:19.590598 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e50d74-67d2-48fd-bcd7-271f006b2706 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:19.590874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e50d74-67d2-48fd-bcd7-271f006b2706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:19.615076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-17e50d74-67d2-48fd-bcd7-271f006b2706 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:19.616489 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-17e50d74-67d2-48fd-bcd7-271f006b2706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:19.633163 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:19.695552 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 460/932] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:08:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:19.735668 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 473/933] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:19.762084 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677aa5d-32ee-4f7d-896f-fd4b278bac7b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:19.834141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:19.963620 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677aa5d-32ee-4f7d-896f-fd4b278bac7b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:20.015763 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbe1a7-50e0-4600-8cae-6848b9fb4c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:20.016623 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbe1a7-50e0-4600-8cae-6848b9fb4c33 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:20.016908 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbe1a7-50e0-4600-8cae-6848b9fb4c33 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:20.017186 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbe1a7-50e0-4600-8cae-6848b9fb4c33 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:20.031737 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dbe1a7-50e0-4600-8cae-6848b9fb4c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-a8dbe1a7-50e0-4600-8cae-6848b9fb4c33 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:20.040076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:20.041379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:20.042140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 461/934] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:08:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:20.048659 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e50d74-67d2-48fd-bcd7-271f006b2706 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:20.163250 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e50d74-67d2-48fd-bcd7-271f006b2706 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:20.199806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fcea1-0f10-4d6a-b116-a5b2d7c53b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:20.201040 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fcea1-0f10-4d6a-b116-a5b2d7c53b6b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:20.201550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fcea1-0f10-4d6a-b116-a5b2d7c53b6b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:20.202000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fcea1-0f10-4d6a-b116-a5b2d7c53b6b None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:20.202924 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fcea1-0f10-4d6a-b116-a5b2d7c53b6b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:20.203409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fcea1-0f10-4d6a-b116-a5b2d7c53b6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:20.205077 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fcea1-0f10-4d6a-b116-a5b2d7c53b6b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:20.214730 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 474/935] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:08:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:20.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbe1a7-50e0-4600-8cae-6848b9fb4c33 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:20.332676 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbe1a7-50e0-4600-8cae-6848b9fb4c33 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:20.334593 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:20.369818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f4634-671e-4477-85c6-d88551146495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:20.371049 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f4634-671e-4477-85c6-d88551146495 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:20.371532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f4634-671e-4477-85c6-d88551146495 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:20.372000 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f4634-671e-4477-85c6-d88551146495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:20.385743 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f4634-671e-4477-85c6-d88551146495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:20.387734 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f4634-671e-4477-85c6-d88551146495 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-d91f4634-671e-4477-85c6-d88551146495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:20.407101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:20.407949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:20.408810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:20.408810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:20.408810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:20.408810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:20.411051 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 462/936] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:20.416513 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fcea1-0f10-4d6a-b116-a5b2d7c53b6b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:20.430296 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:20.524685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fcea1-0f10-4d6a-b116-a5b2d7c53b6b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:20.560250 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480a96f-5d02-44a3-ab23-38f0cfdcc887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:20.560976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480a96f-5d02-44a3-ab23-38f0cfdcc887 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:20.561259 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480a96f-5d02-44a3-ab23-38f0cfdcc887 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:20.561530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480a96f-5d02-44a3-ab23-38f0cfdcc887 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:20.562242 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480a96f-5d02-44a3-ab23-38f0cfdcc887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:20.562547 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480a96f-5d02-44a3-ab23-38f0cfdcc887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:20.563914 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480a96f-5d02-44a3-ab23-38f0cfdcc887 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:20.576614 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 475/937] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:20.582151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f4634-671e-4477-85c6-d88551146495 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:20.692265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f4634-671e-4477-85c6-d88551146495 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:20.730391 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c6d6f-6c83-40cc-b614-ff858986460e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:20.731210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c6d6f-6c83-40cc-b614-ff858986460e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:20.731494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c6d6f-6c83-40cc-b614-ff858986460e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:20.731764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c6d6f-6c83-40cc-b614-ff858986460e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:20.740290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c6d6f-6c83-40cc-b614-ff858986460e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:20.741842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c6d6f-6c83-40cc-b614-ff858986460e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-806c6d6f-6c83-40cc-b614-ff858986460e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:20.757153 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:20.758054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:20.758902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:20.758902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:20.758902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:20.758902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:20.758902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:20.758902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:20.758902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:20.758902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:20.758902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:08:20.758902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:20.758902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 463/938] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:08:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:20.766300 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480a96f-5d02-44a3-ab23-38f0cfdcc887 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:20.849705 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:20.880232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282667e-2726-4b04-836b-f45ae6d46e66 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:20.880736 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282667e-2726-4b04-836b-f45ae6d46e66 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:20.881127 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282667e-2726-4b04-836b-f45ae6d46e66 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:20.882873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 464/939] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:20.884042 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480a96f-5d02-44a3-ab23-38f0cfdcc887 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:20.887977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d309a7-2b9d-4aad-b1d6-c9e3546ee623 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:20.888410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d309a7-2b9d-4aad-b1d6-c9e3546ee623 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:20.889338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d309a7-2b9d-4aad-b1d6-c9e3546ee623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:20.906997 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-60d309a7-2b9d-4aad-b1d6-c9e3546ee623 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:20.908368 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-60d309a7-2b9d-4aad-b1d6-c9e3546ee623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:20.921547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:20.927864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b401ce5-1939-4dfc-bcaf-4bc4b4c127ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:20.928599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401ce5-1939-4dfc-bcaf-4bc4b4c127ec None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:20.928883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401ce5-1939-4dfc-bcaf-4bc4b4c127ec None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:20.929163 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401ce5-1939-4dfc-bcaf-4bc4b4c127ec None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:20.930007 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b401ce5-1939-4dfc-bcaf-4bc4b4c127ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:20.930322 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b401ce5-1939-4dfc-bcaf-4bc4b4c127ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:20.931895 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b401ce5-1939-4dfc-bcaf-4bc4b4c127ec None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:20.951224 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 476/940] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:08:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:20.956876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b401ce5-1939-4dfc-bcaf-4bc4b4c127ec None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:20.994392 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 465/941] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:21.019593 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d309a7-2b9d-4aad-b1d6-c9e3546ee623 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:21.074195 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b401ce5-1939-4dfc-bcaf-4bc4b4c127ec None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:21.106589 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:21.115406 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fe2dd-ab22-4f3c-aa17-bbedd9059dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:21.116465 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fe2dd-ab22-4f3c-aa17-bbedd9059dfa None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:21.116991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fe2dd-ab22-4f3c-aa17-bbedd9059dfa None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:21.117408 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98fe2dd-ab22-4f3c-aa17-bbedd9059dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:21.128726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fe2dd-ab22-4f3c-aa17-bbedd9059dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:21.130656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98fe2dd-ab22-4f3c-aa17-bbedd9059dfa None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:21.151146 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d309a7-2b9d-4aad-b1d6-c9e3546ee623 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-b98fe2dd-ab22-4f3c-aa17-bbedd9059dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:21.151672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:21.152634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:21.153446 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:21.153446 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:21.153446 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:21.153446 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:21.153659 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 477/942] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:21.159873 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fe2dd-ab22-4f3c-aa17-bbedd9059dfa None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:21.201675 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaa0b3-0e38-4853-8f96-7362ed1436f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:21.202723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaa0b3-0e38-4853-8f96-7362ed1436f4 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:21.203241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaa0b3-0e38-4853-8f96-7362ed1436f4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:21.204256 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaa0b3-0e38-4853-8f96-7362ed1436f4 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:21.205399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaa0b3-0e38-4853-8f96-7362ed1436f4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:21.207211 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaa0b3-0e38-4853-8f96-7362ed1436f4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:21.208923 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: WARNING keystone.server.flask.common [None req-dbfaa0b3-0e38-4853-8f96-7362ed1436f4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:08:21.226325 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:21.274430 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.sql.core [None req-dbfaa0b3-0e38-4853-8f96-7362ed1436f4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:08:21.274430 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:08:21.274430 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [parameters: {'id': '9228dddac6cd471daa2bc0031231d354', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:08:21.274430 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=144624) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:08:21.283709 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98fe2dd-ab22-4f3c-aa17-bbedd9059dfa None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-dbfaa0b3-0e38-4853-8f96-7362ed1436f4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9228dddac6cd471daa2bc0031231d354.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9228dddac6cd471daa2bc0031231d354. May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application conn.query(q) May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:08:21.285588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application result.read() May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 20:08:21.286305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application next(self.gen) May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application yield resource May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application next(self.gen) May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise self.value May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application session.commit() May 19 20:08:21.287308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self.session.flush() May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application compat.raise_( May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:08:21.288241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:08:21.289141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application conn.query(q) May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application result.read() May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:08:21.289962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [parameters: {'id': '9228dddac6cd471daa2bc0031231d354', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:21.290768 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:21.291494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:21.296388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:21.296388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 20:08:21.296388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 20:08:21.296388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:21.296388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:21.296388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 20:08:21.296388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 20:08:21.296388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9228dddac6cd471daa2bc0031231d354. May 19 20:08:21.296388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:21.296388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 466/943] 10.222.0.88 () {60 vars in 1140 bytes} [Thu May 19 20:08:21 2022] POST /identity/v3/projects => generated 262 bytes in 272 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:21.298748 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaa0b3-0e38-4853-8f96-7362ed1436f4 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:21.349932 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6135183-96d4-4965-a423-8f23d834aa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:21.350625 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6135183-96d4-4965-a423-8f23d834aa69 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:21.350910 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6135183-96d4-4965-a423-8f23d834aa69 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:21.351194 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6135183-96d4-4965-a423-8f23d834aa69 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:21.351932 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6135183-96d4-4965-a423-8f23d834aa69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:21.352244 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6135183-96d4-4965-a423-8f23d834aa69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:21.353730 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6135183-96d4-4965-a423-8f23d834aa69 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:21.375409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 478/944] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:21.377000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6135183-96d4-4965-a423-8f23d834aa69 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:21.431802 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:21.452401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaa0b3-0e38-4853-8f96-7362ed1436f4 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:21.492699 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222596ff-bc02-4386-a7e0-871f9840a124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:21.493950 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222596ff-bc02-4386-a7e0-871f9840a124 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:21.494457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222596ff-bc02-4386-a7e0-871f9840a124 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:21.494996 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222596ff-bc02-4386-a7e0-871f9840a124 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:21.503543 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222596ff-bc02-4386-a7e0-871f9840a124 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:21.504362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6135183-96d4-4965-a423-8f23d834aa69 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:21.505930 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222596ff-bc02-4386-a7e0-871f9840a124 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-222596ff-bc02-4386-a7e0-871f9840a124 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:21.522372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:21.523242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:21.524758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:21.524758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:21.524758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:21.524758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:21.524758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:21.524758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:21.524758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:21.524758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:21.524758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 467/945] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:21.525392 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f44fc0-cfc7-4231-8f48-99262090ab9a None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:21.527866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f44fc0-cfc7-4231-8f48-99262090ab9a None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:21.527866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f44fc0-cfc7-4231-8f48-99262090ab9a None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:21.530346 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 468/946] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:21.531089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f44fc0-cfc7-4231-8f48-99262090ab9a None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:21.548500 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096af61f-e89b-43ca-8b51-df1258f2209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:21.549453 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af61f-e89b-43ca-8b51-df1258f2209f None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:21.549748 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af61f-e89b-43ca-8b51-df1258f2209f None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:21.550027 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096af61f-e89b-43ca-8b51-df1258f2209f None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:21.557100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096af61f-e89b-43ca-8b51-df1258f2209f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:21.558557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096af61f-e89b-43ca-8b51-df1258f2209f None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-096af61f-e89b-43ca-8b51-df1258f2209f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:21.573738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:21.574731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:21.575613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:21.575613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:21.575613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:21.575613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:21.575613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:21.575613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:21.575613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:21.575613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:21.575613 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 479/947] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:21.576611 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c1df9-b25c-4be0-907e-27d17922ecab None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:21.576898 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c1df9-b25c-4be0-907e-27d17922ecab None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:21.577178 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c1df9-b25c-4be0-907e-27d17922ecab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:21.593113 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-0a8c1df9-b25c-4be0-907e-27d17922ecab None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:21.594287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-0a8c1df9-b25c-4be0-907e-27d17922ecab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:21.603174 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:21.648988 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f44fc0-cfc7-4231-8f48-99262090ab9a None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:21.666168 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 480/948] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:21.667026 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c1df9-b25c-4be0-907e-27d17922ecab None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:21.696788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ff3fb-0a46-4a8d-9773-db59ff3e5b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:21.697618 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ff3fb-0a46-4a8d-9773-db59ff3e5b1c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:21.697901 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ff3fb-0a46-4a8d-9773-db59ff3e5b1c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:21.698175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ff3fb-0a46-4a8d-9773-db59ff3e5b1c None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:21.698936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ff3fb-0a46-4a8d-9773-db59ff3e5b1c None admin] RBAC: Adding query filter params (name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:21.699243 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ff3fb-0a46-4a8d-9773-db59ff3e5b1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:21.700708 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ff3fb-0a46-4a8d-9773-db59ff3e5b1c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:21.700852 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:21.724754 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 469/949] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:08:21 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:21.725951 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ff3fb-0a46-4a8d-9773-db59ff3e5b1c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:21.792530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c1df9-b25c-4be0-907e-27d17922ecab None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:21.832572 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19867df9-642d-4545-bdf4-c790429cd287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:21.833417 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19867df9-642d-4545-bdf4-c790429cd287 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:21.833776 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19867df9-642d-4545-bdf4-c790429cd287 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:21.834127 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19867df9-642d-4545-bdf4-c790429cd287 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:21.835010 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19867df9-642d-4545-bdf4-c790429cd287 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:21.835390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19867df9-642d-4545-bdf4-c790429cd287 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:21.837134 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19867df9-642d-4545-bdf4-c790429cd287 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:21.851572 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 481/950] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:08:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:21.862206 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19867df9-642d-4545-bdf4-c790429cd287 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:21.864465 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ff3fb-0a46-4a8d-9773-db59ff3e5b1c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:21.890148 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:21.908840 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f201a9-89c0-42c9-a250-69eec538e48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:21.909918 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f201a9-89c0-42c9-a250-69eec538e48a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:21.910352 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f201a9-89c0-42c9-a250-69eec538e48a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:21.910774 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f201a9-89c0-42c9-a250-69eec538e48a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:21.911719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f201a9-89c0-42c9-a250-69eec538e48a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:21.913384 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f201a9-89c0-42c9-a250-69eec538e48a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-20f201a9-89c0-42c9-a250-69eec538e48a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:21.929912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:21.930858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:21.936625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 470/951] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:08:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:21.939241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f201a9-89c0-42c9-a250-69eec538e48a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:21.990808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:22.007302 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19867df9-642d-4545-bdf4-c790429cd287 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:22.061481 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a17a8-3812-41be-81e3-7bf05784404a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:22.063719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a17a8-3812-41be-81e3-7bf05784404a None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:22.064325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a17a8-3812-41be-81e3-7bf05784404a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:22.064816 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a17a8-3812-41be-81e3-7bf05784404a None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/3ed06928b2214d2cb10c89f095feb5cc/roles/07c8382535c74f80a514602a3dbe9a7f` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:22.068872 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a17a8-3812-41be-81e3-7bf05784404a None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=3ed06928b2214d2cb10c89f095feb5cc, role_id=07c8382535c74f80a514602a3dbe9a7f)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:22.071881 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a17a8-3812-41be-81e3-7bf05784404a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:22.073488 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f201a9-89c0-42c9-a250-69eec538e48a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:22.119729 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95010d-b991-457d-a225-29708db060c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:22.120546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95010d-b991-457d-a225-29708db060c2 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:22.120836 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95010d-b991-457d-a225-29708db060c2 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:22.121124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95010d-b991-457d-a225-29708db060c2 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:22.121855 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95010d-b991-457d-a225-29708db060c2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:22.122178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95010d-b991-457d-a225-29708db060c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:22.123650 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95010d-b991-457d-a225-29708db060c2 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:22.129525 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 471/952] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:08:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:22.134969 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95010d-b991-457d-a225-29708db060c2 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:22.188658 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 482/953] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:08:21 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/3ed06928b2214d2cb10c89f095feb5cc/roles/07c8382535c74f80a514602a3dbe9a7f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:08:22.200683 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:22.259545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0cc60-5201-453b-9fb6-a2742da8f5b9 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:22.259842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0cc60-5201-453b-9fb6-a2742da8f5b9 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:22.260132 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0cc60-5201-453b-9fb6-a2742da8f5b9 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:22.261620 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 483/954] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:22.269774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489a978-c0eb-4e07-9d29-a8deae9ebe8b None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:22.270294 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489a978-c0eb-4e07-9d29-a8deae9ebe8b None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:22.270766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489a978-c0eb-4e07-9d29-a8deae9ebe8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:22.283771 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95010d-b991-457d-a225-29708db060c2 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:22.291922 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-5489a978-c0eb-4e07-9d29-a8deae9ebe8b None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:22.293576 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-5489a978-c0eb-4e07-9d29-a8deae9ebe8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:22.323817 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f85f-f5bb-435f-a08f-ccabd77b60c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:22.324570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f85f-f5bb-435f-a08f-ccabd77b60c1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:22.324854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f85f-f5bb-435f-a08f-ccabd77b60c1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:22.325146 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f85f-f5bb-435f-a08f-ccabd77b60c1 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:22.325854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1f85f-f5bb-435f-a08f-ccabd77b60c1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:22.326230 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1f85f-f5bb-435f-a08f-ccabd77b60c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:22.327693 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1f85f-f5bb-435f-a08f-ccabd77b60c1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:22.333828 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 472/955] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:08:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:22.339588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f85f-f5bb-435f-a08f-ccabd77b60c1 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:22.399739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 484/956] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:22.426440 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489a978-c0eb-4e07-9d29-a8deae9ebe8b None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:22.484465 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f85f-f5bb-435f-a08f-ccabd77b60c1 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:22.503230 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:22.518701 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:22.530057 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad5891-f516-490d-b84c-0004d20ae460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:22.530799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad5891-f516-490d-b84c-0004d20ae460 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:22.531082 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad5891-f516-490d-b84c-0004d20ae460 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:22.531795 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ad5891-f516-490d-b84c-0004d20ae460 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:22.532608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad5891-f516-490d-b84c-0004d20ae460 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:22.534248 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ad5891-f516-490d-b84c-0004d20ae460 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:22.546478 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489a978-c0eb-4e07-9d29-a8deae9ebe8b None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:22.582893 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5516b-3698-47b3-8c86-5f92707b103e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:22.584037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5516b-3698-47b3-8c86-5f92707b103e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:22.584574 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5516b-3698-47b3-8c86-5f92707b103e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:22.585075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5516b-3698-47b3-8c86-5f92707b103e None admin] PATH_INFO: `/v3/services/compute` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:22.585927 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5516b-3698-47b3-8c86-5f92707b103e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:22.587516 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5516b-3698-47b3-8c86-5f92707b103e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-cbe5516b-3698-47b3-8c86-5f92707b103e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:22.599710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:22.600675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:08:22.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:22.602801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 485/957] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:22.608224 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5516b-3698-47b3-8c86-5f92707b103e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:22.649911 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 473/958] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:08:22 2022] POST /identity/v3/endpoints => generated 349 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:08:22.736618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5516b-3698-47b3-8c86-5f92707b103e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:22.788196 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc063481-a0f6-4b9e-81ff-8bcff9d3e8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:22.789303 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc063481-a0f6-4b9e-81ff-8bcff9d3e8e6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:22.789819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc063481-a0f6-4b9e-81ff-8bcff9d3e8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:22.790299 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc063481-a0f6-4b9e-81ff-8bcff9d3e8e6 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:22.791249 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc063481-a0f6-4b9e-81ff-8bcff9d3e8e6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:22.791762 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc063481-a0f6-4b9e-81ff-8bcff9d3e8e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:22.793637 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc063481-a0f6-4b9e-81ff-8bcff9d3e8e6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:22.801007 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 486/959] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:08:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:22.815709 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad5891-f516-490d-b84c-0004d20ae460 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:22.930648 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ad5891-f516-490d-b84c-0004d20ae460 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:22.970373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696af3c-2804-4f6b-8128-8c77dbbaf6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:22.971417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696af3c-2804-4f6b-8128-8c77dbbaf6a1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:22.971906 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696af3c-2804-4f6b-8128-8c77dbbaf6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:22.972418 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696af3c-2804-4f6b-8128-8c77dbbaf6a1 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:22.973361 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696af3c-2804-4f6b-8128-8c77dbbaf6a1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:22.973871 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696af3c-2804-4f6b-8128-8c77dbbaf6a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:22.975447 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696af3c-2804-4f6b-8128-8c77dbbaf6a1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:22.981379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 474/960] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:08:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:24.187183 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c290bb-25b6-4cf9-9ec4-ee7094a60634 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:24.187693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c290bb-25b6-4cf9-9ec4-ee7094a60634 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:24.187918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c290bb-25b6-4cf9-9ec4-ee7094a60634 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:24.189587 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 487/961] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:24.195090 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad0de2-98a7-41af-9079-74351c9b09fe None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:24.195508 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad0de2-98a7-41af-9079-74351c9b09fe None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:24.195674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad0de2-98a7-41af-9079-74351c9b09fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:24.214325 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-c6ad0de2-98a7-41af-9079-74351c9b09fe None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:24.215854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-c6ad0de2-98a7-41af-9079-74351c9b09fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:24.244351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:24.292483 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 475/962] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:24.314975 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c290bb-25b6-4cf9-9ec4-ee7094a60634 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:24.317900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:24.435504 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c290bb-25b6-4cf9-9ec4-ee7094a60634 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:24.475304 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316f7f6-a058-4a6b-90c5-59337dacb96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:24.475980 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316f7f6-a058-4a6b-90c5-59337dacb96d None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:24.476253 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316f7f6-a058-4a6b-90c5-59337dacb96d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:24.476562 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316f7f6-a058-4a6b-90c5-59337dacb96d None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:24.477429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316f7f6-a058-4a6b-90c5-59337dacb96d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:24.478965 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316f7f6-a058-4a6b-90c5-59337dacb96d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:24.480460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: WARNING keystone.server.flask.common [None req-4316f7f6-a058-4a6b-90c5-59337dacb96d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:08:24.525527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.sql.core [None req-4316f7f6-a058-4a6b-90c5-59337dacb96d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:08:24.525527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:08:24.525527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [parameters: {'user_id': '8fe686f5302448d79b1529596a3450b1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:08:24.525527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=144623) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-4316f7f6-a058-4a6b-90c5-59337dacb96d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application conn.query(q) May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application result.read() May 19 20:08:24.543821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:08:24.544732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application next(self.gen) May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application yield resource May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application next(self.gen) May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise self.value May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application session.commit() May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:08:24.545621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self.session.flush() May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application compat.raise_( May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:08:24.547120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:08:24.549462 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application conn.query(q) May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application result.read() May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:08:24.551666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [parameters: {'user_id': '8fe686f5302448d79b1529596a3450b1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:24.553871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 20:08:24.555694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 20:08:24.557921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 20:08:24.557921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 20:08:24.557921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:08:24.557921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:08:24.557921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:08:24.557921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:24.557921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 488/963] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:08:24 2022] POST /identity/v3/users => generated 160 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:24.558380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad0de2-98a7-41af-9079-74351c9b09fe None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:24.559596 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f682296-4b24-4a30-8f16-7c25cd7e8fbf None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:24.559884 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f682296-4b24-4a30-8f16-7c25cd7e8fbf None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:24.560158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f682296-4b24-4a30-8f16-7c25cd7e8fbf None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:24.566902 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 489/964] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:08:24.571512 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af30c15-b1f1-41f1-bb25-fe6a37bb8be8 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:24.571831 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af30c15-b1f1-41f1-bb25-fe6a37bb8be8 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:24.572126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af30c15-b1f1-41f1-bb25-fe6a37bb8be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:24.596676 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-8af30c15-b1f1-41f1-bb25-fe6a37bb8be8 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:24.598304 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-8af30c15-b1f1-41f1-bb25-fe6a37bb8be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:24.668248 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad0de2-98a7-41af-9079-74351c9b09fe None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:24.678760 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:24.696715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 490/965] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:24.711799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eefd9-a548-4048-ba78-f60e08f6f5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:24.713087 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eefd9-a548-4048-ba78-f60e08f6f5fc None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:24.713731 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eefd9-a548-4048-ba78-f60e08f6f5fc None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:24.714261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eefd9-a548-4048-ba78-f60e08f6f5fc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:24.714470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af30c15-b1f1-41f1-bb25-fe6a37bb8be8 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:24.723570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752eefd9-a548-4048-ba78-f60e08f6f5fc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:24.725330 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752eefd9-a548-4048-ba78-f60e08f6f5fc None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-752eefd9-a548-4048-ba78-f60e08f6f5fc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:24.742158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:24.743175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:24.744079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:24.744079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:24.744079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:24.744079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:24.744079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:24.744079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:24.744079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:24.744079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:24.744079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:08:24.744079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:24.744079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 476/966] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:08:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:24.749247 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eefd9-a548-4048-ba78-f60e08f6f5fc None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:24.784737 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:24.859588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af30c15-b1f1-41f1-bb25-fe6a37bb8be8 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:24.896099 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eefd9-a548-4048-ba78-f60e08f6f5fc None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:24.900118 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b56e-9373-4b8d-a4e7-1822171feaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:24.900674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b56e-9373-4b8d-a4e7-1822171feaa9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:24.900963 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b56e-9373-4b8d-a4e7-1822171feaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:24.901241 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630b56e-9373-4b8d-a4e7-1822171feaa9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:24.914107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630b56e-9373-4b8d-a4e7-1822171feaa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-a630b56e-9373-4b8d-a4e7-1822171feaa9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:24.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:24.924601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:24.925395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 491/967] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:08:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:24.930570 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b56e-9373-4b8d-a4e7-1822171feaa9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:24.946975 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc2861-67f3-4e1e-a8de-876fa93f2158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:24.948298 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc2861-67f3-4e1e-a8de-876fa93f2158 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:24.949021 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc2861-67f3-4e1e-a8de-876fa93f2158 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:24.949848 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc2861-67f3-4e1e-a8de-876fa93f2158 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:24.951966 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc2861-67f3-4e1e-a8de-876fa93f2158 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:24.952924 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc2861-67f3-4e1e-a8de-876fa93f2158 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:24.955763 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc2861-67f3-4e1e-a8de-876fa93f2158 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:25.000087 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:25.004762 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 477/968] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:08:24 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:25.062002 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630b56e-9373-4b8d-a4e7-1822171feaa9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:25.076220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a885e-85e4-44ac-aec2-989e60710d06 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.076220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a885e-85e4-44ac-aec2-989e60710d06 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.076220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a885e-85e4-44ac-aec2-989e60710d06 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.078381 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 478/969] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:08:25.084210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efbc44-2711-4551-880f-d54ff8824b41 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.084544 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efbc44-2711-4551-880f-d54ff8824b41 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.084832 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efbc44-2711-4551-880f-d54ff8824b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.102940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-10efbc44-2711-4551-880f-d54ff8824b41 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:25.104286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-10efbc44-2711-4551-880f-d54ff8824b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:25.107859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c231a2-ee39-410a-81fe-47c61ee849bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:25.109585 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c231a2-ee39-410a-81fe-47c61ee849bb None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.110052 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c231a2-ee39-410a-81fe-47c61ee849bb None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.110531 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c231a2-ee39-410a-81fe-47c61ee849bb None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.111479 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c231a2-ee39-410a-81fe-47c61ee849bb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:25.111968 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c231a2-ee39-410a-81fe-47c61ee849bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:25.113830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c231a2-ee39-410a-81fe-47c61ee849bb None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:25.125094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 492/970] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:08:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:25.130477 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c231a2-ee39-410a-81fe-47c61ee849bb None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:25.212995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 479/971] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:25.235765 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efbc44-2711-4551-880f-d54ff8824b41 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:25.251255 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:25.277683 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c231a2-ee39-410a-81fe-47c61ee849bb None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:25.298659 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:25.320993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c9a7c-b974-4260-bc9f-2037ceb8e8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:25.321839 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c9a7c-b974-4260-bc9f-2037ceb8e8e7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.322278 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c9a7c-b974-4260-bc9f-2037ceb8e8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.322559 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c9a7c-b974-4260-bc9f-2037ceb8e8e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.337311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c9a7c-b974-4260-bc9f-2037ceb8e8e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:25.339532 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c9a7c-b974-4260-bc9f-2037ceb8e8e7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-a16c9a7c-b974-4260-bc9f-2037ceb8e8e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:25.361214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:25.362116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:25.362982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:25.362982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:25.362982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:25.362982 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:25.365667 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 493/972] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:25.369385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b5b58-e36a-4d6b-89f5-72e6c97de7a9 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.369673 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b5b58-e36a-4d6b-89f5-72e6c97de7a9 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.369947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b5b58-e36a-4d6b-89f5-72e6c97de7a9 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.371291 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 494/973] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:08:25.371720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10efbc44-2711-4551-880f-d54ff8824b41 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:25.371986 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b5b58-e36a-4d6b-89f5-72e6c97de7a9 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:25.407144 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30acd9-8698-4914-8ffc-46fb41b5e8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:25.407942 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30acd9-8698-4914-8ffc-46fb41b5e8e6 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.408224 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30acd9-8698-4914-8ffc-46fb41b5e8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.408532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30acd9-8698-4914-8ffc-46fb41b5e8e6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.409192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30acd9-8698-4914-8ffc-46fb41b5e8e6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:25.410569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30acd9-8698-4914-8ffc-46fb41b5e8e6 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-fd30acd9-8698-4914-8ffc-46fb41b5e8e6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:25.423301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:25.424249 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:25.429555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 480/974] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:08:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:25.430318 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09155b8-0b9d-4d58-bd3d-7ec7a8ceb2a9 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.430318 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09155b8-0b9d-4d58-bd3d-7ec7a8ceb2a9 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.430318 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09155b8-0b9d-4d58-bd3d-7ec7a8ceb2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.444863 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-e09155b8-0b9d-4d58-bd3d-7ec7a8ceb2a9 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:25.446042 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-e09155b8-0b9d-4d58-bd3d-7ec7a8ceb2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:25.476950 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b5b58-e36a-4d6b-89f5-72e6c97de7a9 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:25.517203 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16729687-5cdc-4ab4-a099-223fa0d756b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:25.518216 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16729687-5cdc-4ab4-a099-223fa0d756b3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.518675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16729687-5cdc-4ab4-a099-223fa0d756b3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.519048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16729687-5cdc-4ab4-a099-223fa0d756b3 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.520137 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16729687-5cdc-4ab4-a099-223fa0d756b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:25.520625 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16729687-5cdc-4ab4-a099-223fa0d756b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:25.522257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16729687-5cdc-4ab4-a099-223fa0d756b3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:25.527096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 481/975] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:25.527956 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09155b8-0b9d-4d58-bd3d-7ec7a8ceb2a9 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:25.532922 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:25.540180 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 495/976] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:25.541599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16729687-5cdc-4ab4-a099-223fa0d756b3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:25.570372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:25.631200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09155b8-0b9d-4d58-bd3d-7ec7a8ceb2a9 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:25.641895 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16729687-5cdc-4ab4-a099-223fa0d756b3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:25.665964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa99903-6b6e-42e9-be17-e7bf450660c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:25.666629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa99903-6b6e-42e9-be17-e7bf450660c2 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.666936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa99903-6b6e-42e9-be17-e7bf450660c2 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.667211 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa99903-6b6e-42e9-be17-e7bf450660c2 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.667921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa99903-6b6e-42e9-be17-e7bf450660c2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:25.668304 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa99903-6b6e-42e9-be17-e7bf450660c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:25.669641 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa99903-6b6e-42e9-be17-e7bf450660c2 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:25.674912 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 482/977] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:08:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:25.675582 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa99903-6b6e-42e9-be17-e7bf450660c2 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:25.680994 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92e0aa-c705-423f-9ba7-b75c9f873062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:25.681812 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92e0aa-c705-423f-9ba7-b75c9f873062 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.682094 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92e0aa-c705-423f-9ba7-b75c9f873062 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.682369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92e0aa-c705-423f-9ba7-b75c9f873062 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.691037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92e0aa-c705-423f-9ba7-b75c9f873062 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:25.692603 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92e0aa-c705-423f-9ba7-b75c9f873062 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-5c92e0aa-c705-423f-9ba7-b75c9f873062 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:25.708135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:25.709269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 496/978] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:08:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:25.715334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92e0aa-c705-423f-9ba7-b75c9f873062 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:25.750146 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:25.785259 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa99903-6b6e-42e9-be17-e7bf450660c2 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:25.804621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:25.817102 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92e0aa-c705-423f-9ba7-b75c9f873062 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:25.820853 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835f509-e3ff-4d31-a964-3ca6805cd6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:25.821529 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835f509-e3ff-4d31-a964-3ca6805cd6d6 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.821810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835f509-e3ff-4d31-a964-3ca6805cd6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.822089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f835f509-e3ff-4d31-a964-3ca6805cd6d6 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.822848 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835f509-e3ff-4d31-a964-3ca6805cd6d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:25.824214 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f835f509-e3ff-4d31-a964-3ca6805cd6d6 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:25.850912 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4cf5a4-0cff-4f67-90af-957809bf1480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:25.851579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cf5a4-0cff-4f67-90af-957809bf1480 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:25.851934 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cf5a4-0cff-4f67-90af-957809bf1480 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:25.852198 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4cf5a4-0cff-4f67-90af-957809bf1480 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:25.852936 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cf5a4-0cff-4f67-90af-957809bf1480 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:25.853248 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cf5a4-0cff-4f67-90af-957809bf1480 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:25.854644 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4cf5a4-0cff-4f67-90af-957809bf1480 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:25.860264 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 497/979] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:08:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:25.862995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4cf5a4-0cff-4f67-90af-957809bf1480 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:25.926172 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 483/980] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:08:25 2022] POST /identity/v3/services => generated 242 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:25.927023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835f509-e3ff-4d31-a964-3ca6805cd6d6 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:25.965853 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4cf5a4-0cff-4f67-90af-957809bf1480 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:26.018681 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08663b9b-71ac-45f9-9a47-384e5340c321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:26.019391 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08663b9b-71ac-45f9-9a47-384e5340c321 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:26.019768 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08663b9b-71ac-45f9-9a47-384e5340c321 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:26.020154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08663b9b-71ac-45f9-9a47-384e5340c321 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:26.021043 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08663b9b-71ac-45f9-9a47-384e5340c321 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:26.021361 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08663b9b-71ac-45f9-9a47-384e5340c321 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:26.022869 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08663b9b-71ac-45f9-9a47-384e5340c321 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:26.049320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 498/981] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:08:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:26.050569 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08663b9b-71ac-45f9-9a47-384e5340c321 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:26.061383 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f835f509-e3ff-4d31-a964-3ca6805cd6d6 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:26.061930 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:26.096577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:26.112641 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79877f-3fd1-45d0-a6f4-2f354bd5252c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:26.113330 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79877f-3fd1-45d0-a6f4-2f354bd5252c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:26.113644 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79877f-3fd1-45d0-a6f4-2f354bd5252c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:26.113894 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79877f-3fd1-45d0-a6f4-2f354bd5252c None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:26.114702 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79877f-3fd1-45d0-a6f4-2f354bd5252c None admin] RBAC: Adding query filter params (interface=admin, service_id=114388ac41414ab2871c06099f5823df, region_id=RegionOne) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:26.115019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79877f-3fd1-45d0-a6f4-2f354bd5252c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:26.116433 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79877f-3fd1-45d0-a6f4-2f354bd5252c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:26.127861 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 484/982] 10.222.0.88 () {56 vars in 1249 bytes} [Thu May 19 20:08:25 2022] GET /identity/v3/endpoints?service_id=114388ac41414ab2871c06099f5823df&interface=admin®ion_id=RegionOne => generated 193 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:26.129755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79877f-3fd1-45d0-a6f4-2f354bd5252c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:26.174231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08663b9b-71ac-45f9-9a47-384e5340c321 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:26.218011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74bf0e-4fa7-4515-9e6a-df3813597e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:26.218693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74bf0e-4fa7-4515-9e6a-df3813597e69 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:26.218985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74bf0e-4fa7-4515-9e6a-df3813597e69 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:26.219264 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74bf0e-4fa7-4515-9e6a-df3813597e69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:26.240245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79877f-3fd1-45d0-a6f4-2f354bd5252c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:26.280849 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b911d-d28a-49f9-84c4-90fd2e7b6c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:26.281605 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b911d-d28a-49f9-84c4-90fd2e7b6c45 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:26.281902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b911d-d28a-49f9-84c4-90fd2e7b6c45 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:26.282293 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b911d-d28a-49f9-84c4-90fd2e7b6c45 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:26.282992 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b911d-d28a-49f9-84c4-90fd2e7b6c45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:26.284606 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b911d-d28a-49f9-84c4-90fd2e7b6c45 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:26.296427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74bf0e-4fa7-4515-9e6a-df3813597e69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:26.296949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 485/983] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:08:26 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 20:08:26.297042 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74bf0e-4fa7-4515-9e6a-df3813597e69 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:26.311258 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:26.419002 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 499/984] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:26 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:08:26.421493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b911d-d28a-49f9-84c4-90fd2e7b6c45 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:26.489976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:26.538140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b911d-d28a-49f9-84c4-90fd2e7b6c45 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:26.573218 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b48b64-e3d0-41b9-b8a8-cbb0ea099386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:26.574021 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b48b64-e3d0-41b9-b8a8-cbb0ea099386 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:26.574441 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b48b64-e3d0-41b9-b8a8-cbb0ea099386 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:26.574613 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b48b64-e3d0-41b9-b8a8-cbb0ea099386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:26.585839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b48b64-e3d0-41b9-b8a8-cbb0ea099386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:26.587303 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b48b64-e3d0-41b9-b8a8-cbb0ea099386 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-14b48b64-e3d0-41b9-b8a8-cbb0ea099386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:26.605284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:26.606166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:26.607006 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:26.607006 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:26.607006 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:26.607006 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:26.608269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 486/985] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:26.613096 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74bf0e-4fa7-4515-9e6a-df3813597e69 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:26.716558 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74bf0e-4fa7-4515-9e6a-df3813597e69 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:26.725165 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:26.752721 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148af0e9-51c5-4a91-a71d-ce2a221c945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:26.753813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148af0e9-51c5-4a91-a71d-ce2a221c945b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:26.754271 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148af0e9-51c5-4a91-a71d-ce2a221c945b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:26.754723 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148af0e9-51c5-4a91-a71d-ce2a221c945b None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:26.755594 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148af0e9-51c5-4a91-a71d-ce2a221c945b None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:26.756172 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148af0e9-51c5-4a91-a71d-ce2a221c945b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:26.757610 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148af0e9-51c5-4a91-a71d-ce2a221c945b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:26.774285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 500/986] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:26.775947 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b48b64-e3d0-41b9-b8a8-cbb0ea099386 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:26.897954 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b48b64-e3d0-41b9-b8a8-cbb0ea099386 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:26.907733 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:26.933348 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bcf9e-e14e-4de7-8130-e70911274762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:26.934302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bcf9e-e14e-4de7-8130-e70911274762 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:26.934678 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bcf9e-e14e-4de7-8130-e70911274762 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:26.935056 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3bcf9e-e14e-4de7-8130-e70911274762 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:26.943117 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3bcf9e-e14e-4de7-8130-e70911274762 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:26.944689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3bcf9e-e14e-4de7-8130-e70911274762 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-0f3bcf9e-e14e-4de7-8130-e70911274762 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:26.958698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:26.959719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:26.960997 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:26.960997 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:26.960997 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:26.960997 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:26.960997 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:26.960997 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:26.960997 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:26.960997 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:26.962609 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 487/987] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:26.968194 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148af0e9-51c5-4a91-a71d-ce2a221c945b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:27.071263 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148af0e9-51c5-4a91-a71d-ce2a221c945b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:27.115557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e03319-d707-4a4b-80b0-6798c876efd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:27.116227 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e03319-d707-4a4b-80b0-6798c876efd5 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:27.116541 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e03319-d707-4a4b-80b0-6798c876efd5 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:27.116820 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e03319-d707-4a4b-80b0-6798c876efd5 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:27.117604 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e03319-d707-4a4b-80b0-6798c876efd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:27.117920 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e03319-d707-4a4b-80b0-6798c876efd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:27.119470 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e03319-d707-4a4b-80b0-6798c876efd5 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:27.137907 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 501/988] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:08:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:27.139733 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bcf9e-e14e-4de7-8130-e70911274762 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:27.257979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3bcf9e-e14e-4de7-8130-e70911274762 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:27.297714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744d511-15fc-4128-a034-26a61104c193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:27.298409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744d511-15fc-4128-a034-26a61104c193 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:27.298693 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744d511-15fc-4128-a034-26a61104c193 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:27.298966 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744d511-15fc-4128-a034-26a61104c193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:27.299799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744d511-15fc-4128-a034-26a61104c193 None admin] RBAC: Adding query filter params (role.id=07c8382535c74f80a514602a3dbe9a7f, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=3ed06928b2214d2cb10c89f095feb5cc) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:27.300113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744d511-15fc-4128-a034-26a61104c193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:27.301712 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f744d511-15fc-4128-a034-26a61104c193 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:27.307901 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 488/989] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:08:27 2022] GET /identity/v3/role_assignments?user.id=3ed06928b2214d2cb10c89f095feb5cc&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=07c8382535c74f80a514602a3dbe9a7f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:28.202637 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a35292-fa9f-41e7-8572-916b74abb92d None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:28.203159 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a35292-fa9f-41e7-8572-916b74abb92d None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:28.203159 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a35292-fa9f-41e7-8572-916b74abb92d None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:28.204825 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 502/990] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:28.210090 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b85f5-8155-4bef-9c93-5eab3ae3ec63 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:28.210460 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b85f5-8155-4bef-9c93-5eab3ae3ec63 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:28.210750 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b85f5-8155-4bef-9c93-5eab3ae3ec63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:28.228381 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-691b85f5-8155-4bef-9c93-5eab3ae3ec63 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:28.229574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-691b85f5-8155-4bef-9c93-5eab3ae3ec63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:28.257714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:28.300974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 489/991] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:28.324246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a35292-fa9f-41e7-8572-916b74abb92d None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:28.352380 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:28.434046 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a35292-fa9f-41e7-8572-916b74abb92d None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:28.472900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c0a9b-7829-46f8-bc90-aa8d3491e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:28.473712 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c0a9b-7829-46f8-bc90-aa8d3491e068 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:28.473996 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c0a9b-7829-46f8-bc90-aa8d3491e068 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:28.474274 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c0a9b-7829-46f8-bc90-aa8d3491e068 None admin] PATH_INFO: `/v3/services/compute` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:28.474934 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c0a9b-7829-46f8-bc90-aa8d3491e068 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:28.476348 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c0a9b-7829-46f8-bc90-aa8d3491e068 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-603c0a9b-7829-46f8-bc90-aa8d3491e068 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:28.488855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:28.489844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:08:28.490645 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:28.492412 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 503/992] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:28.496453 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b85f5-8155-4bef-9c93-5eab3ae3ec63 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:28.579526 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a4a4e-1ba1-4e47-8ec6-20124cc57ab0 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:28.580160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a4a4e-1ba1-4e47-8ec6-20124cc57ab0 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:28.580482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a4a4e-1ba1-4e47-8ec6-20124cc57ab0 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:28.582065 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 504/993] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:08:28.587224 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bde2cf-672d-424d-a33a-b82407157c49 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:28.587517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bde2cf-672d-424d-a33a-b82407157c49 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:28.587794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bde2cf-672d-424d-a33a-b82407157c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:28.602741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b85f5-8155-4bef-9c93-5eab3ae3ec63 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:28.605247 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-27bde2cf-672d-424d-a33a-b82407157c49 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:28.606443 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-27bde2cf-672d-424d-a33a-b82407157c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:28.639346 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54508e2-1ed2-4364-b69b-18537d12a22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:28.640011 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54508e2-1ed2-4364-b69b-18537d12a22f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:28.640290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54508e2-1ed2-4364-b69b-18537d12a22f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:28.640606 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54508e2-1ed2-4364-b69b-18537d12a22f None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:28.641310 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54508e2-1ed2-4364-b69b-18537d12a22f None admin] RBAC: Adding query filter params (name=compute) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:28.641617 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54508e2-1ed2-4364-b69b-18537d12a22f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:28.642981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54508e2-1ed2-4364-b69b-18537d12a22f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:28.648166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 490/994] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:08:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:28.653468 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54508e2-1ed2-4364-b69b-18537d12a22f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:28.683241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:28.684957 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 505/995] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:28.701625 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bde2cf-672d-424d-a33a-b82407157c49 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:28.723565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:28.765155 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54508e2-1ed2-4364-b69b-18537d12a22f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:28.803629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674dbae-f3ae-452b-905e-5ac2a7e3c506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:28.804245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674dbae-f3ae-452b-905e-5ac2a7e3c506 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:28.804564 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674dbae-f3ae-452b-905e-5ac2a7e3c506 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:28.805089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674dbae-f3ae-452b-905e-5ac2a7e3c506 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:28.805705 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674dbae-f3ae-452b-905e-5ac2a7e3c506 None admin] RBAC: Adding query filter params (type=compute) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:28.806066 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674dbae-f3ae-452b-905e-5ac2a7e3c506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:28.807409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674dbae-f3ae-452b-905e-5ac2a7e3c506 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:28.808176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bde2cf-672d-424d-a33a-b82407157c49 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:28.814294 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 491/996] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:08:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:28.820014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674dbae-f3ae-452b-905e-5ac2a7e3c506 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:28.845781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b791271-242b-430d-861b-bf7842bd64fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:28.846600 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b791271-242b-430d-861b-bf7842bd64fe None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:28.846885 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b791271-242b-430d-861b-bf7842bd64fe None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:28.847167 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b791271-242b-430d-861b-bf7842bd64fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:28.847825 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b791271-242b-430d-861b-bf7842bd64fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:28.849239 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b791271-242b-430d-861b-bf7842bd64fe None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-9b791271-242b-430d-861b-bf7842bd64fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:28.861729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:28.862693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:28.863498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 506/997] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:08:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:28.869702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b791271-242b-430d-861b-bf7842bd64fe None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:28.925215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:28.936849 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674dbae-f3ae-452b-905e-5ac2a7e3c506 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:28.960869 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:28.977325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b791271-242b-430d-861b-bf7842bd64fe None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:28.977685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2207a97-2f3a-4a8f-b0fa-3d2485d71a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:28.978353 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2207a97-2f3a-4a8f-b0fa-3d2485d71a7a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:28.978636 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2207a97-2f3a-4a8f-b0fa-3d2485d71a7a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:28.978909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2207a97-2f3a-4a8f-b0fa-3d2485d71a7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:28.980454 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2207a97-2f3a-4a8f-b0fa-3d2485d71a7a None admin] RBAC: Adding query filter params (interface=public, service_id=3a8837732af04509885ed31e665fc400, region_id=RegionOne) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:28.980454 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2207a97-2f3a-4a8f-b0fa-3d2485d71a7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:28.981400 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2207a97-2f3a-4a8f-b0fa-3d2485d71a7a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:28.991461 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 492/998] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:08:28 2022] GET /identity/v3/endpoints?service_id=3a8837732af04509885ed31e665fc400&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:28.992305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2207a97-2f3a-4a8f-b0fa-3d2485d71a7a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:29.019230 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864505f-b7de-4215-afd2-3d3f9808060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:29.019958 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864505f-b7de-4215-afd2-3d3f9808060e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:29.020225 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864505f-b7de-4215-afd2-3d3f9808060e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:29.020549 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864505f-b7de-4215-afd2-3d3f9808060e None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:29.021261 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864505f-b7de-4215-afd2-3d3f9808060e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:29.021576 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864505f-b7de-4215-afd2-3d3f9808060e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:29.022968 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864505f-b7de-4215-afd2-3d3f9808060e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:29.028369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 507/999] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:08:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:29.033792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864505f-b7de-4215-afd2-3d3f9808060e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:29.108421 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2207a97-2f3a-4a8f-b0fa-3d2485d71a7a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:29.146332 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158d304-e366-49bc-b805-4ad1da380a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:29.146929 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158d304-e366-49bc-b805-4ad1da380a8f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:29.147058 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864505f-b7de-4215-afd2-3d3f9808060e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:29.147223 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158d304-e366-49bc-b805-4ad1da380a8f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:29.147492 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158d304-e366-49bc-b805-4ad1da380a8f None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:29.148183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158d304-e366-49bc-b805-4ad1da380a8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:29.150757 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158d304-e366-49bc-b805-4ad1da380a8f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:29.172540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 493/1000] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:08:28 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:08:29.185529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e19e6-0031-4f74-b7e4-b3e6bd0f40d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:29.186226 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e19e6-0031-4f74-b7e4-b3e6bd0f40d9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:29.186512 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e19e6-0031-4f74-b7e4-b3e6bd0f40d9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:29.186825 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e19e6-0031-4f74-b7e4-b3e6bd0f40d9 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:29.187550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e19e6-0031-4f74-b7e4-b3e6bd0f40d9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:29.187864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e19e6-0031-4f74-b7e4-b3e6bd0f40d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:29.189336 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e19e6-0031-4f74-b7e4-b3e6bd0f40d9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:29.195478 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 508/1001] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:08:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:29.201921 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158d304-e366-49bc-b805-4ad1da380a8f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:29.227386 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:29.318777 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158d304-e366-49bc-b805-4ad1da380a8f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:29.357295 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b79cf8-2dbe-46ff-9bfa-6bf4a83d34a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:29.357984 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b79cf8-2dbe-46ff-9bfa-6bf4a83d34a3 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:29.358267 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b79cf8-2dbe-46ff-9bfa-6bf4a83d34a3 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:29.358542 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b79cf8-2dbe-46ff-9bfa-6bf4a83d34a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:29.359313 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b79cf8-2dbe-46ff-9bfa-6bf4a83d34a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:29.360732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b79cf8-2dbe-46ff-9bfa-6bf4a83d34a3 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:29.436265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 494/1002] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:08:29 2022] POST /identity/v3/endpoints => generated 325 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:08:29.592798 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07557aea-7b7a-4672-a59a-0f59c5d19508 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:29.593321 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07557aea-7b7a-4672-a59a-0f59c5d19508 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:29.593321 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07557aea-7b7a-4672-a59a-0f59c5d19508 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:29.594968 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 509/1003] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:29.600701 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27f341-4cae-43c7-abb0-c4c318a5a792 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:29.601102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27f341-4cae-43c7-abb0-c4c318a5a792 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:29.601387 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27f341-4cae-43c7-abb0-c4c318a5a792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:29.619547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-bc27f341-4cae-43c7-abb0-c4c318a5a792 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:29.620784 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-bc27f341-4cae-43c7-abb0-c4c318a5a792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:29.698268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 495/1004] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:29 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:29.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07557aea-7b7a-4672-a59a-0f59c5d19508 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:29.737286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:29.828579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07557aea-7b7a-4672-a59a-0f59c5d19508 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:29.912772 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081d361-db05-486b-9ff6-2453c63e697e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:29.913574 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081d361-db05-486b-9ff6-2453c63e697e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:29.913858 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081d361-db05-486b-9ff6-2453c63e697e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:29.914135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081d361-db05-486b-9ff6-2453c63e697e None admin] PATH_INFO: `/v3/services/image` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:29.914796 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081d361-db05-486b-9ff6-2453c63e697e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:29.916181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081d361-db05-486b-9ff6-2453c63e697e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-d081d361-db05-486b-9ff6-2453c63e697e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:29.928390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:29.929332 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:29.930110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 510/1005] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 20:08:29 2022] GET /identity/v3/services/image => generated 86 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:29.936920 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27f341-4cae-43c7-abb0-c4c318a5a792 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:29.940085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:30.041502 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27f341-4cae-43c7-abb0-c4c318a5a792 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:30.079927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5c1869-e5c2-42ed-80e3-02e2c0ea7816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:30.080629 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c1869-e5c2-42ed-80e3-02e2c0ea7816 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:30.080914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c1869-e5c2-42ed-80e3-02e2c0ea7816 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:30.081208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5c1869-e5c2-42ed-80e3-02e2c0ea7816 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:30.081922 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c1869-e5c2-42ed-80e3-02e2c0ea7816 None admin] RBAC: Adding query filter params (name=image) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:30.082233 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c1869-e5c2-42ed-80e3-02e2c0ea7816 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:30.083598 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5c1869-e5c2-42ed-80e3-02e2c0ea7816 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:30.093941 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 496/1006] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:30.095730 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081d361-db05-486b-9ff6-2453c63e697e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:30.184844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:30.209283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081d361-db05-486b-9ff6-2453c63e697e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:30.245989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104783a5-e494-4e7f-a127-58635dd7aaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:30.246664 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104783a5-e494-4e7f-a127-58635dd7aaa5 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:30.246951 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104783a5-e494-4e7f-a127-58635dd7aaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:30.247229 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104783a5-e494-4e7f-a127-58635dd7aaa5 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:30.247970 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104783a5-e494-4e7f-a127-58635dd7aaa5 None admin] RBAC: Adding query filter params (type=image) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:30.248280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104783a5-e494-4e7f-a127-58635dd7aaa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:30.249682 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104783a5-e494-4e7f-a127-58635dd7aaa5 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:30.260439 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 511/1007] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:30 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:31.471969 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d16c2-659f-49bd-a9d3-7774f43db0cf None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:31.472511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d16c2-659f-49bd-a9d3-7774f43db0cf None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:31.472511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d16c2-659f-49bd-a9d3-7774f43db0cf None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:31.473896 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 497/1008] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:31.479362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d862863-bf88-4fff-9775-3f776b24321c None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:31.479797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d862863-bf88-4fff-9775-3f776b24321c None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:31.479939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d862863-bf88-4fff-9775-3f776b24321c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:31.497808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-3d862863-bf88-4fff-9775-3f776b24321c None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:31.499746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-3d862863-bf88-4fff-9775-3f776b24321c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:31.572372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 512/1009] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:31 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:31.593887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d16c2-659f-49bd-a9d3-7774f43db0cf None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:31.672044 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:31.707020 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d16c2-659f-49bd-a9d3-7774f43db0cf None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:31.743608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae1718e-aeb8-4b96-aa5c-fd297a93547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:31.744327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1718e-aeb8-4b96-aa5c-fd297a93547d None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:31.744635 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1718e-aeb8-4b96-aa5c-fd297a93547d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:31.744913 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae1718e-aeb8-4b96-aa5c-fd297a93547d None admin] PATH_INFO: `/v3/services/compute` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:31.745569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1718e-aeb8-4b96-aa5c-fd297a93547d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:31.746948 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae1718e-aeb8-4b96-aa5c-fd297a93547d None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-0ae1718e-aeb8-4b96-aa5c-fd297a93547d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:31.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:31.759687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:08:31.760539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:31.764828 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 498/1010] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:31.766087 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d862863-bf88-4fff-9775-3f776b24321c None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:31.805252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9cd0-9906-4b09-b7a3-9730893c73aa None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:31.805568 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9cd0-9906-4b09-b7a3-9730893c73aa None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:31.805819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b9cd0-9906-4b09-b7a3-9730893c73aa None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:31.807378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 499/1011] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:08:31.812460 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a3c43-4054-491c-a7cd-98870c01a650 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:31.812754 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a3c43-4054-491c-a7cd-98870c01a650 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:31.813030 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a3c43-4054-491c-a7cd-98870c01a650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:31.830726 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-6e2a3c43-4054-491c-a7cd-98870c01a650 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:31.831905 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-6e2a3c43-4054-491c-a7cd-98870c01a650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:31.839521 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:31.872051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d862863-bf88-4fff-9775-3f776b24321c None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:31.907220 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 500/1012] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:31 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:31.909170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef29b-6de3-457c-92de-d016d15f43dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:31.909847 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef29b-6de3-457c-92de-d016d15f43dd None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:31.910243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef29b-6de3-457c-92de-d016d15f43dd None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:31.910414 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef29b-6de3-457c-92de-d016d15f43dd None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:31.911122 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef29b-6de3-457c-92de-d016d15f43dd None admin] RBAC: Adding query filter params (name=compute) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:31.911433 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef29b-6de3-457c-92de-d016d15f43dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:31.912851 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef29b-6de3-457c-92de-d016d15f43dd None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:31.924021 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 513/1013] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:08:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:31.925641 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a3c43-4054-491c-a7cd-98870c01a650 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:31.930823 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef29b-6de3-457c-92de-d016d15f43dd None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:31.991062 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:32.033721 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a3c43-4054-491c-a7cd-98870c01a650 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:32.040803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef29b-6de3-457c-92de-d016d15f43dd None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:32.069745 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204428b-cd93-4e9c-9542-570c26470321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:32.070422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204428b-cd93-4e9c-9542-570c26470321 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:32.070704 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204428b-cd93-4e9c-9542-570c26470321 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:32.070969 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204428b-cd93-4e9c-9542-570c26470321 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:32.071682 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204428b-cd93-4e9c-9542-570c26470321 None admin] RBAC: Adding query filter params (type=compute) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:32.071990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204428b-cd93-4e9c-9542-570c26470321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:32.073398 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204428b-cd93-4e9c-9542-570c26470321 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:32.081952 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba75a40-fc4d-4107-a5b1-3be54c097ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:32.082752 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba75a40-fc4d-4107-a5b1-3be54c097ad7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:32.083074 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 501/1014] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:08:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:32.084714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba75a40-fc4d-4107-a5b1-3be54c097ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:32.085070 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba75a40-fc4d-4107-a5b1-3be54c097ad7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:32.086309 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba75a40-fc4d-4107-a5b1-3be54c097ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:32.088614 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba75a40-fc4d-4107-a5b1-3be54c097ad7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:32.088714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204428b-cd93-4e9c-9542-570c26470321 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:32.095125 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 514/1015] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:32.096521 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba75a40-fc4d-4107-a5b1-3be54c097ad7 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:32.102497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:32.197075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204428b-cd93-4e9c-9542-570c26470321 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:32.227660 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:32.233804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba75a40-fc4d-4107-a5b1-3be54c097ad7 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:32.237192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631afe08-8dd7-4fb3-afde-38b5fcc4c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:32.237854 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631afe08-8dd7-4fb3-afde-38b5fcc4c4f2 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:32.238140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631afe08-8dd7-4fb3-afde-38b5fcc4c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:32.238420 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631afe08-8dd7-4fb3-afde-38b5fcc4c4f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:32.239161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631afe08-8dd7-4fb3-afde-38b5fcc4c4f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:32.240563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631afe08-8dd7-4fb3-afde-38b5fcc4c4f2 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:32.269850 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3444ab9-c0fa-4a84-b7b5-3cd6ca889919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:32.270509 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3444ab9-c0fa-4a84-b7b5-3cd6ca889919 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:32.270792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3444ab9-c0fa-4a84-b7b5-3cd6ca889919 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:32.271067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3444ab9-c0fa-4a84-b7b5-3cd6ca889919 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:32.271778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3444ab9-c0fa-4a84-b7b5-3cd6ca889919 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:32.273318 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3444ab9-c0fa-4a84-b7b5-3cd6ca889919 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:32.355282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 502/1016] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:08:32 2022] POST /identity/v3/endpoints => generated 334 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:08:32.405015 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 515/1017] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:08:32 2022] POST /identity/v3/projects => generated 311 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:08:32.507592 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341f72a-7153-40a9-bbd5-e54aa1111273 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:32.508241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341f72a-7153-40a9-bbd5-e54aa1111273 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:32.508241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341f72a-7153-40a9-bbd5-e54aa1111273 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:32.509941 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 503/1018] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:32.516560 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd265b6-86c4-4b54-9734-11b196bc30e9 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:32.517072 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd265b6-86c4-4b54-9734-11b196bc30e9 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:32.517143 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd265b6-86c4-4b54-9734-11b196bc30e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:32.536009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-5dd265b6-86c4-4b54-9734-11b196bc30e9 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:32.537303 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-5dd265b6-86c4-4b54-9734-11b196bc30e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:32.595678 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:32.621162 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 516/1019] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:32 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:32.644559 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341f72a-7153-40a9-bbd5-e54aa1111273 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:32.768303 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341f72a-7153-40a9-bbd5-e54aa1111273 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:32.806064 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3504bbdb-ee9b-4c1b-9b9d-c0742eb2dcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:32.806738 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504bbdb-ee9b-4c1b-9b9d-c0742eb2dcd7 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:32.807022 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504bbdb-ee9b-4c1b-9b9d-c0742eb2dcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:32.807300 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3504bbdb-ee9b-4c1b-9b9d-c0742eb2dcd7 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:32.808023 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504bbdb-ee9b-4c1b-9b9d-c0742eb2dcd7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:32.809431 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3504bbdb-ee9b-4c1b-9b9d-c0742eb2dcd7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:32.949857 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 504/1020] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:08:32 2022] POST /identity/v3/services => generated 242 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:34.726107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5582c35-2c1b-4e30-864b-92e552aa7dfd None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:34.726685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5582c35-2c1b-4e30-864b-92e552aa7dfd None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:34.726685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5582c35-2c1b-4e30-864b-92e552aa7dfd None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:34.728217 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 517/1021] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:34.733820 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c38da8-62a5-4cfc-a529-dca997da0aee None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:34.734252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c38da8-62a5-4cfc-a529-dca997da0aee None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:34.734421 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c38da8-62a5-4cfc-a529-dca997da0aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:34.752014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-76c38da8-62a5-4cfc-a529-dca997da0aee None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:34.753265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-76c38da8-62a5-4cfc-a529-dca997da0aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:34.791322 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:34.830791 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 505/1022] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:34.833433 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d07c0-3bc4-470e-ac67-e0e6ca0898c9 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:34.833731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d07c0-3bc4-470e-ac67-e0e6ca0898c9 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:34.834017 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d07c0-3bc4-470e-ac67-e0e6ca0898c9 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:34.835593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 518/1023] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:08:34.841201 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338331c-18ca-4a1e-8299-5aca6e36f15e None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:34.841509 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338331c-18ca-4a1e-8299-5aca6e36f15e None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:34.841870 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338331c-18ca-4a1e-8299-5aca6e36f15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:34.856232 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d07c0-3bc4-470e-ac67-e0e6ca0898c9 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:34.858904 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-a338331c-18ca-4a1e-8299-5aca6e36f15e None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:34.861314 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-a338331c-18ca-4a1e-8299-5aca6e36f15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:34.942296 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 506/1024] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:34.966132 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d07c0-3bc4-470e-ac67-e0e6ca0898c9 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:34.973840 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338331c-18ca-4a1e-8299-5aca6e36f15e None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:35.004772 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c3c62-0dd2-4e16-9107-3ca8765d852d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:35.005489 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c3c62-0dd2-4e16-9107-3ca8765d852d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.005774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c3c62-0dd2-4e16-9107-3ca8765d852d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.006044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c3c62-0dd2-4e16-9107-3ca8765d852d None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.007252 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c3c62-0dd2-4e16-9107-3ca8765d852d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:35.008914 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c3c62-0dd2-4e16-9107-3ca8765d852d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:35.016604 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 519/1025] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:35.017419 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c3c62-0dd2-4e16-9107-3ca8765d852d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:35.071431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:35.084131 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:35.088045 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338331c-18ca-4a1e-8299-5aca6e36f15e None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:35.126656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52561b15-ab3c-4380-a56a-31d3a02708bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:35.127473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52561b15-ab3c-4380-a56a-31d3a02708bd None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.127751 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52561b15-ab3c-4380-a56a-31d3a02708bd None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.128037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52561b15-ab3c-4380-a56a-31d3a02708bd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.130284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c3c62-0dd2-4e16-9107-3ca8765d852d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:35.140783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52561b15-ab3c-4380-a56a-31d3a02708bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-52561b15-ab3c-4380-a56a-31d3a02708bd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:35.148958 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:35.149907 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:35.150714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:35.153079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 507/1026] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:08:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:35.156283 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52561b15-ab3c-4380-a56a-31d3a02708bd None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:35.175364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615a4ff-fc18-428d-bd46-9fb5c05c3c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:35.176197 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a4ff-fc18-428d-bd46-9fb5c05c3c4f None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.176520 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a4ff-fc18-428d-bd46-9fb5c05c3c4f None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.176799 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615a4ff-fc18-428d-bd46-9fb5c05c3c4f None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.177520 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615a4ff-fc18-428d-bd46-9fb5c05c3c4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:35.178947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615a4ff-fc18-428d-bd46-9fb5c05c3c4f None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:35.263326 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52561b15-ab3c-4380-a56a-31d3a02708bd None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:35.269303 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 520/1027] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:08:35 2022] POST /identity/v3/users => generated 285 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:08:35.270682 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd2610-2e65-491d-b3d4-d6f90d3b13a9 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.270974 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd2610-2e65-491d-b3d4-d6f90d3b13a9 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.271250 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd2610-2e65-491d-b3d4-d6f90d3b13a9 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.272714 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 521/1028] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:35.278453 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d96e96-6f1c-40d5-a595-945f0556db8c None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.278724 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d96e96-6f1c-40d5-a595-945f0556db8c None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.278998 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d96e96-6f1c-40d5-a595-945f0556db8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.302684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-a7d96e96-6f1c-40d5-a595-945f0556db8c None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:35.304196 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-a7d96e96-6f1c-40d5-a595-945f0556db8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:35.306107 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af39c9-d177-4902-8e50-953d429179eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:35.306850 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af39c9-d177-4902-8e50-953d429179eb None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.307075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af39c9-d177-4902-8e50-953d429179eb None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.307352 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af39c9-d177-4902-8e50-953d429179eb None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.308115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af39c9-d177-4902-8e50-953d429179eb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:35.309266 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af39c9-d177-4902-8e50-953d429179eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:35.309969 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af39c9-d177-4902-8e50-953d429179eb None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:35.324564 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 508/1029] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:08:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:35.327211 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af39c9-d177-4902-8e50-953d429179eb None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:35.353574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:35.385203 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 522/1030] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:35.408528 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d96e96-6f1c-40d5-a595-945f0556db8c None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:35.445773 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:35.447695 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af39c9-d177-4902-8e50-953d429179eb None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:35.490638 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78201d97-0c0b-4dd4-a2b2-53f3e5f12d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:35.491442 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78201d97-0c0b-4dd4-a2b2-53f3e5f12d92 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.491723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78201d97-0c0b-4dd4-a2b2-53f3e5f12d92 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.491996 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78201d97-0c0b-4dd4-a2b2-53f3e5f12d92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.506916 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78201d97-0c0b-4dd4-a2b2-53f3e5f12d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:35.508442 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78201d97-0c0b-4dd4-a2b2-53f3e5f12d92 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:35.521988 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d96e96-6f1c-40d5-a595-945f0556db8c None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-78201d97-0c0b-4dd4-a2b2-53f3e5f12d92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:35.530290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:35.531151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:35.532171 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:35.532171 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:35.532171 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:35.532171 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:35.532461 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 509/1031] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:35.538787 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78201d97-0c0b-4dd4-a2b2-53f3e5f12d92 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:35.563195 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352a38-26cc-4b00-b98d-94f621966ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:35.564252 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352a38-26cc-4b00-b98d-94f621966ce0 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.564804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352a38-26cc-4b00-b98d-94f621966ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.565330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352a38-26cc-4b00-b98d-94f621966ce0 None admin] PATH_INFO: `/v3/services/image` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.565995 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8352a38-26cc-4b00-b98d-94f621966ce0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:35.567374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8352a38-26cc-4b00-b98d-94f621966ce0 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-c8352a38-26cc-4b00-b98d-94f621966ce0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:35.580151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:35.581282 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:35.582057 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 523/1032] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 20:08:35 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:35.587269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352a38-26cc-4b00-b98d-94f621966ce0 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:35.615640 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:35.648000 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78201d97-0c0b-4dd4-a2b2-53f3e5f12d92 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:35.686741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a329fd-c183-4fcf-8324-7f818c0280e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:35.687414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a329fd-c183-4fcf-8324-7f818c0280e7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.687704 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a329fd-c183-4fcf-8324-7f818c0280e7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.687977 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a329fd-c183-4fcf-8324-7f818c0280e7 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.688765 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a329fd-c183-4fcf-8324-7f818c0280e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:35.689329 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a329fd-c183-4fcf-8324-7f818c0280e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:35.690445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a329fd-c183-4fcf-8324-7f818c0280e7 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:35.694882 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:35.699299 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352a38-26cc-4b00-b98d-94f621966ce0 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:35.707361 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 510/1033] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:35.708539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a329fd-c183-4fcf-8324-7f818c0280e7 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:35.736058 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b8b11-5b15-465b-96be-c2454a9413bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:35.736936 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b8b11-5b15-465b-96be-c2454a9413bf None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.737227 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b8b11-5b15-465b-96be-c2454a9413bf None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.737505 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b8b11-5b15-465b-96be-c2454a9413bf None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.738229 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b8b11-5b15-465b-96be-c2454a9413bf None admin] RBAC: Adding query filter params (name=image) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:35.738540 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b8b11-5b15-465b-96be-c2454a9413bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:35.739934 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b8b11-5b15-465b-96be-c2454a9413bf None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:35.746023 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 524/1034] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:35.751660 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b8b11-5b15-465b-96be-c2454a9413bf None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:35.822567 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a329fd-c183-4fcf-8324-7f818c0280e7 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:35.858464 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:35.859585 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b8b11-5b15-465b-96be-c2454a9413bf None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:35.862914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144ff90-19a4-4844-bc21-4bf50c7c8b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:35.863710 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144ff90-19a4-4844-bc21-4bf50c7c8b04 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.864282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144ff90-19a4-4844-bc21-4bf50c7c8b04 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.864586 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144ff90-19a4-4844-bc21-4bf50c7c8b04 None admin] PATH_INFO: `/v3/users/nova` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.872501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144ff90-19a4-4844-bc21-4bf50c7c8b04 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:35.874230 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144ff90-19a4-4844-bc21-4bf50c7c8b04 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-e144ff90-19a4-4844-bc21-4bf50c7c8b04 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:35.889819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:35.890770 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:35.891666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:35.891666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:35.891666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:35.891666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:35.891666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:35.891666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:35.891666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:35.891666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:35.891666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:08:35.891666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:35.891666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 511/1035] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:08:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:35.896974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144ff90-19a4-4844-bc21-4bf50c7c8b04 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:35.897868 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84683ac7-8347-419b-9c13-6aff710a5e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:35.898545 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84683ac7-8347-419b-9c13-6aff710a5e29 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:35.898832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84683ac7-8347-419b-9c13-6aff710a5e29 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:35.899111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84683ac7-8347-419b-9c13-6aff710a5e29 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:35.899823 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84683ac7-8347-419b-9c13-6aff710a5e29 None admin] RBAC: Adding query filter params (type=image) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:35.900131 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84683ac7-8347-419b-9c13-6aff710a5e29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:35.901546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84683ac7-8347-419b-9c13-6aff710a5e29 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:35.907768 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 525/1036] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:35.913540 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84683ac7-8347-419b-9c13-6aff710a5e29 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:35.942765 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:36.010641 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144ff90-19a4-4844-bc21-4bf50c7c8b04 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:36.020895 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84683ac7-8347-419b-9c13-6aff710a5e29 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:36.050474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167398a8-593c-4b76-b3a2-43697f446973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:36.051160 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167398a8-593c-4b76-b3a2-43697f446973 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:36.051436 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167398a8-593c-4b76-b3a2-43697f446973 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:36.051706 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167398a8-593c-4b76-b3a2-43697f446973 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:36.052568 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167398a8-593c-4b76-b3a2-43697f446973 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:36.052881 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167398a8-593c-4b76-b3a2-43697f446973 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:36.054390 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167398a8-593c-4b76-b3a2-43697f446973 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:36.061359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeafafe-aac2-47eb-9039-f50f255de2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:36.062028 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeafafe-aac2-47eb-9039-f50f255de2da None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:36.062315 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeafafe-aac2-47eb-9039-f50f255de2da None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:36.062618 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeafafe-aac2-47eb-9039-f50f255de2da None admin] PATH_INFO: `/v3/endpoints` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:36.064074 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeafafe-aac2-47eb-9039-f50f255de2da None admin] RBAC: Adding query filter params (interface=public, service_id=82ae614ff4c94859a2bf9269fcc953c7, region_id=RegionOne) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:36.064074 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeafafe-aac2-47eb-9039-f50f255de2da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:36.065119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeafafe-aac2-47eb-9039-f50f255de2da None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:36.071040 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 526/1037] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:08:35 2022] GET /identity/v3/endpoints?service_id=82ae614ff4c94859a2bf9269fcc953c7&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:36.077557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeafafe-aac2-47eb-9039-f50f255de2da None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:36.080505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 512/1038] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:08:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:36.083124 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167398a8-593c-4b76-b3a2-43697f446973 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:36.136961 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:36.186792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:36.191476 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167398a8-593c-4b76-b3a2-43697f446973 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:36.197403 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeafafe-aac2-47eb-9039-f50f255de2da None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:36.227986 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24c96c-2aa2-4cc2-ac05-bade5ac01032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:36.228667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24c96c-2aa2-4cc2-ac05-bade5ac01032 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:36.228945 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24c96c-2aa2-4cc2-ac05-bade5ac01032 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:36.229217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24c96c-2aa2-4cc2-ac05-bade5ac01032 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:36.233320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b9d49-9263-4fcf-9e8a-b8af66a46998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:36.234010 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b9d49-9263-4fcf-9e8a-b8af66a46998 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:36.234301 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b9d49-9263-4fcf-9e8a-b8af66a46998 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:36.234690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b9d49-9263-4fcf-9e8a-b8af66a46998 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:36.235274 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b9d49-9263-4fcf-9e8a-b8af66a46998 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:36.236694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b9d49-9263-4fcf-9e8a-b8af66a46998 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:36.244686 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 527/1039] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:08:36 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:08:36.305263 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24c96c-2aa2-4cc2-ac05-bade5ac01032 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:36.306750 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24c96c-2aa2-4cc2-ac05-bade5ac01032 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:36.361588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:36.422342 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 513/1040] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:36 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:08:36.431647 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b9d49-9263-4fcf-9e8a-b8af66a46998 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:36.540900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b9d49-9263-4fcf-9e8a-b8af66a46998 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:36.578319 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ceeb8-53c2-42ae-9324-a1922835cfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:36.579526 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ceeb8-53c2-42ae-9324-a1922835cfd3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:36.579999 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ceeb8-53c2-42ae-9324-a1922835cfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:36.580460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ceeb8-53c2-42ae-9324-a1922835cfd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:36.592122 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ceeb8-53c2-42ae-9324-a1922835cfd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:36.593813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ceeb8-53c2-42ae-9324-a1922835cfd3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-aa7ceeb8-53c2-42ae-9324-a1922835cfd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:36.612991 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:36.613883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:36.613883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:36.613883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:36.613883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:36.614134 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 528/1041] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:36.619920 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24c96c-2aa2-4cc2-ac05-bade5ac01032 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:36.725821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24c96c-2aa2-4cc2-ac05-bade5ac01032 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:36.761858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda127-6184-4286-8966-90786b1e154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:36.762558 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda127-6184-4286-8966-90786b1e154c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:36.762850 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda127-6184-4286-8966-90786b1e154c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:36.763174 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cda127-6184-4286-8966-90786b1e154c None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:36.763899 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cda127-6184-4286-8966-90786b1e154c None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:36.764211 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cda127-6184-4286-8966-90786b1e154c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:36.765633 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cda127-6184-4286-8966-90786b1e154c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:36.777501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 514/1042] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:36.783546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ceeb8-53c2-42ae-9324-a1922835cfd3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:36.793952 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:36.887844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ceeb8-53c2-42ae-9324-a1922835cfd3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:36.922867 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a7032-3a79-4326-bbdb-2972e03e99fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:36.923672 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a7032-3a79-4326-bbdb-2972e03e99fb None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:36.923952 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a7032-3a79-4326-bbdb-2972e03e99fb None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:36.924222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874a7032-3a79-4326-bbdb-2972e03e99fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:36.932276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a7032-3a79-4326-bbdb-2972e03e99fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:36.933767 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874a7032-3a79-4326-bbdb-2972e03e99fb None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-874a7032-3a79-4326-bbdb-2972e03e99fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:36.948565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:36.949463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:36.950311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:36.950311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:36.950311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:36.950311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:36.950311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:36.950311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:36.950311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:36.950311 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:36.952786 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 529/1043] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:36.955685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda127-6184-4286-8966-90786b1e154c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:36.971493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:37.063410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cda127-6184-4286-8966-90786b1e154c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:37.108084 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b4e38-93bd-40b3-8ddd-748760a86107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:37.108810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b4e38-93bd-40b3-8ddd-748760a86107 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:37.109098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b4e38-93bd-40b3-8ddd-748760a86107 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:37.109375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b4e38-93bd-40b3-8ddd-748760a86107 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:37.110182 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b4e38-93bd-40b3-8ddd-748760a86107 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:37.110614 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b4e38-93bd-40b3-8ddd-748760a86107 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:37.112069 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b4e38-93bd-40b3-8ddd-748760a86107 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:37.131321 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 515/1044] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:08:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:37.132727 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a7032-3a79-4326-bbdb-2972e03e99fb None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:37.206390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:37.242858 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874a7032-3a79-4326-bbdb-2972e03e99fb None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:37.279841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43495523-67fb-4679-aa5c-3093c4778a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:37.280639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43495523-67fb-4679-aa5c-3093c4778a8e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:37.280931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43495523-67fb-4679-aa5c-3093c4778a8e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:37.281210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43495523-67fb-4679-aa5c-3093c4778a8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:37.282049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43495523-67fb-4679-aa5c-3093c4778a8e None admin] RBAC: Adding query filter params (role.id=07c8382535c74f80a514602a3dbe9a7f, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=f6d44c613f524fe1804d6c9b65fe97ca) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:37.282367 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43495523-67fb-4679-aa5c-3093c4778a8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:37.283807 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43495523-67fb-4679-aa5c-3093c4778a8e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:37.296380 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 530/1045] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:08:37 2022] GET /identity/v3/role_assignments?user.id=f6d44c613f524fe1804d6c9b65fe97ca&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=07c8382535c74f80a514602a3dbe9a7f => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:37.607339 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1240a-db88-47e9-a5fd-41b2df7b94a8 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:37.607730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1240a-db88-47e9-a5fd-41b2df7b94a8 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:37.607915 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1240a-db88-47e9-a5fd-41b2df7b94a8 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:37.609584 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 516/1046] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:37.614836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d01155-3762-46b1-a744-f6c06ec8a438 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:37.615134 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d01155-3762-46b1-a744-f6c06ec8a438 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:37.615413 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d01155-3762-46b1-a744-f6c06ec8a438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:37.632069 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-b4d01155-3762-46b1-a744-f6c06ec8a438 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:37.633289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-b4d01155-3762-46b1-a744-f6c06ec8a438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:37.702914 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 531/1047] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:37.725815 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1240a-db88-47e9-a5fd-41b2df7b94a8 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:37.798631 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:37.833939 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1240a-db88-47e9-a5fd-41b2df7b94a8 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:37.868385 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67f9fe-4278-45c3-8d0e-ed36fbe69722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:37.869102 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67f9fe-4278-45c3-8d0e-ed36fbe69722 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:37.869384 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67f9fe-4278-45c3-8d0e-ed36fbe69722 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:37.869661 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67f9fe-4278-45c3-8d0e-ed36fbe69722 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:37.880837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67f9fe-4278-45c3-8d0e-ed36fbe69722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-4b67f9fe-4278-45c3-8d0e-ed36fbe69722 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:37.889048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:37.889965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:37.890783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:37.892723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 517/1048] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:08:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:37.896552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d01155-3762-46b1-a744-f6c06ec8a438 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:37.957165 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:37.998519 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d01155-3762-46b1-a744-f6c06ec8a438 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:38.034252 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb25c43-9a6d-4626-ad4a-f02dc6dee4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:38.034935 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb25c43-9a6d-4626-ad4a-f02dc6dee4e2 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:38.035220 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb25c43-9a6d-4626-ad4a-f02dc6dee4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:38.035497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb25c43-9a6d-4626-ad4a-f02dc6dee4e2 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:38.036254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb25c43-9a6d-4626-ad4a-f02dc6dee4e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:38.036593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb25c43-9a6d-4626-ad4a-f02dc6dee4e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:38.037996 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb25c43-9a6d-4626-ad4a-f02dc6dee4e2 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:38.048388 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 532/1049] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:08:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:38.054179 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67f9fe-4278-45c3-8d0e-ed36fbe69722 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:38.169009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67f9fe-4278-45c3-8d0e-ed36fbe69722 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:38.197513 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:38.204382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e23d0-204a-4331-87d9-a7117f81b451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:38.205261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e23d0-204a-4331-87d9-a7117f81b451 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:38.205610 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e23d0-204a-4331-87d9-a7117f81b451 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:38.205827 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e23d0-204a-4331-87d9-a7117f81b451 None admin] PATH_INFO: `/v3/users/3a47e58b1ada450eaf778de493c93b62` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:38.220777 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e23d0-204a-4331-87d9-a7117f81b451 None admin] RBAC: Authorizing `identity:get_user(user_id=3a47e58b1ada450eaf778de493c93b62)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:38.222250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5e23d0-204a-4331-87d9-a7117f81b451 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:38.224188 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 518/1050] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:08:38 2022] GET /identity/v3/users/3a47e58b1ada450eaf778de493c93b62 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:38.229807 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb25c43-9a6d-4626-ad4a-f02dc6dee4e2 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:38.335837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb25c43-9a6d-4626-ad4a-f02dc6dee4e2 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:38.361883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:38.373865 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247108f2-7fa1-44fd-b854-41c81532851e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:38.374519 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247108f2-7fa1-44fd-b854-41c81532851e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:38.374801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247108f2-7fa1-44fd-b854-41c81532851e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:38.375075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247108f2-7fa1-44fd-b854-41c81532851e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:38.449688 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247108f2-7fa1-44fd-b854-41c81532851e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:38.451779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247108f2-7fa1-44fd-b854-41c81532851e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:38.512750 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ce7d2-ff00-4895-9c7a-7dcf504758fe None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:38.513087 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ce7d2-ff00-4895-9c7a-7dcf504758fe None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:38.513350 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ce7d2-ff00-4895-9c7a-7dcf504758fe None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:38.515281 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 519/1051] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:38.521764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ce368-6d9a-423a-aa3d-e3842b24d1e8 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:38.522052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ce368-6d9a-423a-aa3d-e3842b24d1e8 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:38.522325 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ce368-6d9a-423a-aa3d-e3842b24d1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:38.541199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-583ce368-6d9a-423a-aa3d-e3842b24d1e8 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:38.542394 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-583ce368-6d9a-423a-aa3d-e3842b24d1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:38.557482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 533/1052] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:38 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:08:38.564188 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247108f2-7fa1-44fd-b854-41c81532851e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:38.588294 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:38.623737 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 520/1053] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:38.647716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ce368-6d9a-423a-aa3d-e3842b24d1e8 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:38.669469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247108f2-7fa1-44fd-b854-41c81532851e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:38.709198 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf68009-d551-46be-aee4-a178e9b61a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:38.710008 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf68009-d551-46be-aee4-a178e9b61a3d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:38.710271 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf68009-d551-46be-aee4-a178e9b61a3d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:38.710553 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf68009-d551-46be-aee4-a178e9b61a3d None admin] PATH_INFO: `/v3/projects/a05605e22abb477fabc16f82893f5d2f` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:38.711746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf68009-d551-46be-aee4-a178e9b61a3d None admin] RBAC: Authorizing `identity:get_project(project_id=a05605e22abb477fabc16f82893f5d2f)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:38.713257 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf68009-d551-46be-aee4-a178e9b61a3d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:38.715785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 534/1054] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:08:38 2022] GET /identity/v3/projects/a05605e22abb477fabc16f82893f5d2f => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:38.721265 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf68009-d551-46be-aee4-a178e9b61a3d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:38.760402 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583ce368-6d9a-423a-aa3d-e3842b24d1e8 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:38.780079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:38.798698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf6997-e6f5-49ee-979e-43bbf95c982a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:38.799484 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf6997-e6f5-49ee-979e-43bbf95c982a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:38.799763 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf6997-e6f5-49ee-979e-43bbf95c982a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:38.800269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf6997-e6f5-49ee-979e-43bbf95c982a None admin] PATH_INFO: `/v3/services/image` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:38.800955 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf6997-e6f5-49ee-979e-43bbf95c982a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:38.802438 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf6997-e6f5-49ee-979e-43bbf95c982a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-67cf6997-e6f5-49ee-979e-43bbf95c982a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:38.816134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:38.817351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:08:38.818200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:38.818913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:38.824382 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 521/1055] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 20:08:38 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:38.826827 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf6997-e6f5-49ee-979e-43bbf95c982a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:38.832735 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf68009-d551-46be-aee4-a178e9b61a3d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:38.873449 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624df41-3702-4c6e-bac7-38e602f0346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:38.874128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624df41-3702-4c6e-bac7-38e602f0346b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:38.874789 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624df41-3702-4c6e-bac7-38e602f0346b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:38.875148 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624df41-3702-4c6e-bac7-38e602f0346b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:38.876299 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624df41-3702-4c6e-bac7-38e602f0346b None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=a05605e22abb477fabc16f82893f5d2f, user.id=3a47e58b1ada450eaf778de493c93b62) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:38.876646 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624df41-3702-4c6e-bac7-38e602f0346b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:38.878259 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624df41-3702-4c6e-bac7-38e602f0346b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:38.892382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 535/1056] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:08:38 2022] GET /identity/v3/role_assignments?user.id=3a47e58b1ada450eaf778de493c93b62&scope.project.id=a05605e22abb477fabc16f82893f5d2f&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:38.949269 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf6997-e6f5-49ee-979e-43bbf95c982a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:38.995219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fbbb0-f941-4f61-8ebf-1b7379db2c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:38.996014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fbbb0-f941-4f61-8ebf-1b7379db2c23 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:38.996372 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fbbb0-f941-4f61-8ebf-1b7379db2c23 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:38.996694 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fbbb0-f941-4f61-8ebf-1b7379db2c23 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:38.997519 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fbbb0-f941-4f61-8ebf-1b7379db2c23 None admin] RBAC: Adding query filter params (name=image) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:38.998415 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fbbb0-f941-4f61-8ebf-1b7379db2c23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:38.999777 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fbbb0-f941-4f61-8ebf-1b7379db2c23 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:39.005585 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 522/1057] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:39.011890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624df41-3702-4c6e-bac7-38e602f0346b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:39.130122 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624df41-3702-4c6e-bac7-38e602f0346b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:39.166295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ff9b3-d912-495b-90b4-1b9451130028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:39.166960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ff9b3-d912-495b-90b4-1b9451130028 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:39.167242 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ff9b3-d912-495b-90b4-1b9451130028 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:39.167515 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ff9b3-d912-495b-90b4-1b9451130028 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:39.168215 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ff9b3-d912-495b-90b4-1b9451130028 None admin] RBAC: Adding query filter params (type=image) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:39.168556 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ff9b3-d912-495b-90b4-1b9451130028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:39.170665 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ff9b3-d912-495b-90b4-1b9451130028 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:39.180827 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 536/1058] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:39.182496 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fbbb0-f941-4f61-8ebf-1b7379db2c23 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:39.239296 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:39.299303 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fbbb0-f941-4f61-8ebf-1b7379db2c23 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:39.334162 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022be731-d59d-42eb-82e9-dd14b05c8252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:39.334824 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be731-d59d-42eb-82e9-dd14b05c8252 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:39.335105 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be731-d59d-42eb-82e9-dd14b05c8252 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:39.335380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be731-d59d-42eb-82e9-dd14b05c8252 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:39.336126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022be731-d59d-42eb-82e9-dd14b05c8252 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:39.337532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022be731-d59d-42eb-82e9-dd14b05c8252 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:39.413070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 523/1059] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:08:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:39.569143 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c371-a3e1-444e-bbb2-a8944bb96410 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:39.569458 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c371-a3e1-444e-bbb2-a8944bb96410 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:39.569718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c371-a3e1-444e-bbb2-a8944bb96410 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:39.571328 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 537/1060] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:39.576963 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb3ced-09bd-4d35-9cb2-46e17917860e None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:39.577252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb3ced-09bd-4d35-9cb2-46e17917860e None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:39.577537 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb3ced-09bd-4d35-9cb2-46e17917860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:39.597873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-bbdb3ced-09bd-4d35-9cb2-46e17917860e None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:39.599085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-bbdb3ced-09bd-4d35-9cb2-46e17917860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:39.686861 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 524/1061] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:39.711450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c371-a3e1-444e-bbb2-a8944bb96410 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:39.731971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:39.819683 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c371-a3e1-444e-bbb2-a8944bb96410 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:39.855274 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b8513-8889-40b5-8cc4-b783224a35f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:39.856114 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b8513-8889-40b5-8cc4-b783224a35f7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:39.856419 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b8513-8889-40b5-8cc4-b783224a35f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:39.856699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b8513-8889-40b5-8cc4-b783224a35f7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:39.867691 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b8513-8889-40b5-8cc4-b783224a35f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-542b8513-8889-40b5-8cc4-b783224a35f7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:39.876434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:39.877435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:39.878218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:39.880426 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 538/1062] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:08:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:39.883581 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb3ced-09bd-4d35-9cb2-46e17917860e None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:39.913488 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:39.990311 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb3ced-09bd-4d35-9cb2-46e17917860e None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:40.029013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35d915-e042-41f8-96fa-1b4962d56ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:40.029730 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35d915-e042-41f8-96fa-1b4962d56ac3 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:40.030019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35d915-e042-41f8-96fa-1b4962d56ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:40.030300 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35d915-e042-41f8-96fa-1b4962d56ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:40.031083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35d915-e042-41f8-96fa-1b4962d56ac3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:40.035076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35d915-e042-41f8-96fa-1b4962d56ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:40.035076 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35d915-e042-41f8-96fa-1b4962d56ac3 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:40.048054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 525/1063] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:08:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:40.049629 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b8513-8889-40b5-8cc4-b783224a35f7 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:40.150183 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:40.174955 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b8513-8889-40b5-8cc4-b783224a35f7 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:40.210040 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75311d95-34ce-43d5-a68a-b8d65b55dbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:40.210821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311d95-34ce-43d5-a68a-b8d65b55dbd6 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:40.211108 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311d95-34ce-43d5-a68a-b8d65b55dbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:40.211387 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75311d95-34ce-43d5-a68a-b8d65b55dbd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:40.223236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75311d95-34ce-43d5-a68a-b8d65b55dbd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:40.224712 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75311d95-34ce-43d5-a68a-b8d65b55dbd6 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-75311d95-34ce-43d5-a68a-b8d65b55dbd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:40.242639 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:40.243517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:40.244382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:40.244382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:40.244382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:40.244382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:40.244382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 539/1064] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:40.251179 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35d915-e042-41f8-96fa-1b4962d56ac3 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:40.348507 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:40.359161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35d915-e042-41f8-96fa-1b4962d56ac3 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:40.393981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbb90f-7ff5-4994-a89b-99adfb436307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:40.394654 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbb90f-7ff5-4994-a89b-99adfb436307 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:40.394943 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbb90f-7ff5-4994-a89b-99adfb436307 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:40.395224 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbb90f-7ff5-4994-a89b-99adfb436307 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:40.395954 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbb90f-7ff5-4994-a89b-99adfb436307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:40.396265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbb90f-7ff5-4994-a89b-99adfb436307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:40.397682 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbb90f-7ff5-4994-a89b-99adfb436307 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:40.411390 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 526/1065] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:40.416989 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75311d95-34ce-43d5-a68a-b8d65b55dbd6 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:40.525656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75311d95-34ce-43d5-a68a-b8d65b55dbd6 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:40.555289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:40.562358 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c9f9c-b307-4405-87ec-36dc2b75098d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:40.563172 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c9f9c-b307-4405-87ec-36dc2b75098d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:40.563456 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c9f9c-b307-4405-87ec-36dc2b75098d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:40.563740 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c9f9c-b307-4405-87ec-36dc2b75098d None admin] PATH_INFO: `/v3/users/nova` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:40.571827 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c9f9c-b307-4405-87ec-36dc2b75098d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:40.573905 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346c9f9c-b307-4405-87ec-36dc2b75098d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-346c9f9c-b307-4405-87ec-36dc2b75098d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:40.588393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:40.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:40.590635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:40.590635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:40.590635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:40.590635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:40.590635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:40.590635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:40.590635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:40.590635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:40.590635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:08:40.590635 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:40.592511 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 540/1066] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:08:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:40.596481 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbb90f-7ff5-4994-a89b-99adfb436307 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:40.703893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbb90f-7ff5-4994-a89b-99adfb436307 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:40.739199 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae0c66-3a35-4ba2-9042-830e23024771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:40.739875 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae0c66-3a35-4ba2-9042-830e23024771 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:40.740155 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae0c66-3a35-4ba2-9042-830e23024771 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:40.740458 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae0c66-3a35-4ba2-9042-830e23024771 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:40.741347 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae0c66-3a35-4ba2-9042-830e23024771 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:40.741659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae0c66-3a35-4ba2-9042-830e23024771 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:40.743090 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae0c66-3a35-4ba2-9042-830e23024771 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:40.766447 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 527/1067] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:08:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:40.767918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c9f9c-b307-4405-87ec-36dc2b75098d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:40.874436 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c9f9c-b307-4405-87ec-36dc2b75098d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:40.908166 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba29e89-96f0-4f85-b90b-6cca284092c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:40.908947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba29e89-96f0-4f85-b90b-6cca284092c2 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:40.909229 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba29e89-96f0-4f85-b90b-6cca284092c2 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:40.909502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba29e89-96f0-4f85-b90b-6cca284092c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:40.921199 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba29e89-96f0-4f85-b90b-6cca284092c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:40.922932 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba29e89-96f0-4f85-b90b-6cca284092c2 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-1ba29e89-96f0-4f85-b90b-6cca284092c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:40.942584 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:40.943566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:40.944473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:40.944473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:40.944473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:40.944473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:40.944473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 541/1068] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:40.950466 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae0c66-3a35-4ba2-9042-830e23024771 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:40.963732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:41.072314 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae0c66-3a35-4ba2-9042-830e23024771 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:41.112915 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dbc3b-2875-46f1-80c4-6f8c2f419da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:41.113594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dbc3b-2875-46f1-80c4-6f8c2f419da4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.113879 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dbc3b-2875-46f1-80c4-6f8c2f419da4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.114157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dbc3b-2875-46f1-80c4-6f8c2f419da4 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.114895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dbc3b-2875-46f1-80c4-6f8c2f419da4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:41.115206 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dbc3b-2875-46f1-80c4-6f8c2f419da4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:41.117019 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dbc3b-2875-46f1-80c4-6f8c2f419da4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:41.134376 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 528/1069] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:41.135546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba29e89-96f0-4f85-b90b-6cca284092c2 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:41.159666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25816a-478c-4417-b0e5-dbeedac6f914 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.159985 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25816a-478c-4417-b0e5-dbeedac6f914 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.160244 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25816a-478c-4417-b0e5-dbeedac6f914 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.161808 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 529/1070] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:41.166648 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369320ea-ae97-4236-9798-45bdc6a38b82 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.166938 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369320ea-ae97-4236-9798-45bdc6a38b82 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.167223 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369320ea-ae97-4236-9798-45bdc6a38b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.184502 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-369320ea-ae97-4236-9798-45bdc6a38b82 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:41.185734 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-369320ea-ae97-4236-9798-45bdc6a38b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:41.187043 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:41.251086 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba29e89-96f0-4f85-b90b-6cca284092c2 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:41.259753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:41.266853 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 530/1071] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:41.290080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369320ea-ae97-4236-9798-45bdc6a38b82 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:41.290774 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20466052-2120-4f55-8429-f75bfd15878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:41.291788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20466052-2120-4f55-8429-f75bfd15878a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.292606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20466052-2120-4f55-8429-f75bfd15878a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.293915 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20466052-2120-4f55-8429-f75bfd15878a None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.303002 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20466052-2120-4f55-8429-f75bfd15878a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:41.305180 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20466052-2120-4f55-8429-f75bfd15878a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-20466052-2120-4f55-8429-f75bfd15878a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:41.324859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:41.326292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:41.327151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:41.327151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:41.327151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:41.327151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:41.327151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:41.327151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:41.327151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:41.327151 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:41.332477 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 542/1072] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:41.334314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20466052-2120-4f55-8429-f75bfd15878a None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:41.401833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369320ea-ae97-4236-9798-45bdc6a38b82 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:41.439558 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20466052-2120-4f55-8429-f75bfd15878a None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:41.441352 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:41.444563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa02d9c-e38b-495d-8078-fee3e710e5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:41.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa02d9c-e38b-495d-8078-fee3e710e5e7 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.445658 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa02d9c-e38b-495d-8078-fee3e710e5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.445940 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa02d9c-e38b-495d-8078-fee3e710e5e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.457683 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa02d9c-e38b-495d-8078-fee3e710e5e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-eaa02d9c-e38b-495d-8078-fee3e710e5e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:41.465551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:41.466531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:41.467401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 531/1073] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:08:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:41.472966 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa02d9c-e38b-495d-8078-fee3e710e5e7 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:41.478837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ace55-ee57-4500-bfd0-495b5c5ac1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:41.479923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ace55-ee57-4500-bfd0-495b5c5ac1d3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.480647 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ace55-ee57-4500-bfd0-495b5c5ac1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.481401 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ace55-ee57-4500-bfd0-495b5c5ac1d3 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.482302 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ace55-ee57-4500-bfd0-495b5c5ac1d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:41.482615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ace55-ee57-4500-bfd0-495b5c5ac1d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:41.484098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ace55-ee57-4500-bfd0-495b5c5ac1d3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:41.501728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 543/1074] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:08:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:41.502723 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ace55-ee57-4500-bfd0-495b5c5ac1d3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:41.540900 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:41.591485 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa02d9c-e38b-495d-8078-fee3e710e5e7 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:41.614896 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ace55-ee57-4500-bfd0-495b5c5ac1d3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:41.628240 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a990-dce9-45dc-9599-ed92d83d5c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:41.628942 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a990-dce9-45dc-9599-ed92d83d5c8a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.629224 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a990-dce9-45dc-9599-ed92d83d5c8a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.629497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333a990-dce9-45dc-9599-ed92d83d5c8a None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.630243 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333a990-dce9-45dc-9599-ed92d83d5c8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:41.630548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333a990-dce9-45dc-9599-ed92d83d5c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:41.631922 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333a990-dce9-45dc-9599-ed92d83d5c8a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:41.641865 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 532/1075] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:08:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:41.647837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a990-dce9-45dc-9599-ed92d83d5c8a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:41.655766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc05d4-322d-46b7-a9cf-71521440b211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:41.656681 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc05d4-322d-46b7-a9cf-71521440b211 None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.656962 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc05d4-322d-46b7-a9cf-71521440b211 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.657228 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc05d4-322d-46b7-a9cf-71521440b211 None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/f6d44c613f524fe1804d6c9b65fe97ca/roles/07c8382535c74f80a514602a3dbe9a7f` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.659429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc05d4-322d-46b7-a9cf-71521440b211 None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=f6d44c613f524fe1804d6c9b65fe97ca, role_id=07c8382535c74f80a514602a3dbe9a7f)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:41.660903 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc05d4-322d-46b7-a9cf-71521440b211 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:41.731936 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 544/1076] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:08:41 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/f6d44c613f524fe1804d6c9b65fe97ca/roles/07c8382535c74f80a514602a3dbe9a7f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:08:41.733738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e7c32-853b-4dfe-a3d9-f8570f3bf718 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.734024 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e7c32-853b-4dfe-a3d9-f8570f3bf718 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.734299 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e7c32-853b-4dfe-a3d9-f8570f3bf718 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.735713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 545/1077] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:41.741591 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64355f-0cfb-4c0d-bea3-e3b36578980a None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.741907 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64355f-0cfb-4c0d-bea3-e3b36578980a None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.742292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64355f-0cfb-4c0d-bea3-e3b36578980a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.761303 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333a990-dce9-45dc-9599-ed92d83d5c8a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:41.761599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-1d64355f-0cfb-4c0d-bea3-e3b36578980a None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:41.762064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-1d64355f-0cfb-4c0d-bea3-e3b36578980a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:41.780940 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:41.786761 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:41.804374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209e778-f683-47c5-8bca-22da981af202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:41.805255 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209e778-f683-47c5-8bca-22da981af202 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.805531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209e778-f683-47c5-8bca-22da981af202 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.805809 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209e778-f683-47c5-8bca-22da981af202 None admin] PATH_INFO: `/v3/users/3a47e58b1ada450eaf778de493c93b62` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.807121 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209e778-f683-47c5-8bca-22da981af202 None admin] RBAC: Authorizing `identity:get_user(user_id=3a47e58b1ada450eaf778de493c93b62)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:41.808643 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209e778-f683-47c5-8bca-22da981af202 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:41.811214 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 533/1078] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:08:41 2022] GET /identity/v3/users/3a47e58b1ada450eaf778de493c93b62 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:41.817235 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209e778-f683-47c5-8bca-22da981af202 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:41.841409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 546/1079] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:41.865575 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64355f-0cfb-4c0d-bea3-e3b36578980a None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:41.938355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209e778-f683-47c5-8bca-22da981af202 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:41.976800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f94a02-81be-44c1-a903-5592778807b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:41.977670 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f94a02-81be-44c1-a903-5592778807b6 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:41.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f94a02-81be-44c1-a903-5592778807b6 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:41.978398 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f94a02-81be-44c1-a903-5592778807b6 None admin] PATH_INFO: `/v3/projects/a05605e22abb477fabc16f82893f5d2f` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:41.979491 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f94a02-81be-44c1-a903-5592778807b6 None admin] RBAC: Authorizing `identity:get_project(project_id=a05605e22abb477fabc16f82893f5d2f)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:41.980976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f94a02-81be-44c1-a903-5592778807b6 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:41.981237 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64355f-0cfb-4c0d-bea3-e3b36578980a None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:41.983600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 534/1080] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:08:41 2022] GET /identity/v3/projects/a05605e22abb477fabc16f82893f5d2f => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:41.990309 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f94a02-81be-44c1-a903-5592778807b6 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:42.024023 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112eada-312d-4015-96ce-619881eb0150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:42.024706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112eada-312d-4015-96ce-619881eb0150 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:42.025343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112eada-312d-4015-96ce-619881eb0150 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:42.025610 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112eada-312d-4015-96ce-619881eb0150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:42.050253 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:42.058364 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:42.101075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112eada-312d-4015-96ce-619881eb0150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:42.102564 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112eada-312d-4015-96ce-619881eb0150 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:42.113027 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f94a02-81be-44c1-a903-5592778807b6 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:42.151951 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87b3cd-286a-40d0-9e89-dc4965d55648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:42.153006 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87b3cd-286a-40d0-9e89-dc4965d55648 None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:42.153286 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87b3cd-286a-40d0-9e89-dc4965d55648 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:42.153570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f87b3cd-286a-40d0-9e89-dc4965d55648 None admin] PATH_INFO: `/v3/projects/a05605e22abb477fabc16f82893f5d2f/users/3a47e58b1ada450eaf778de493c93b62/roles/ab7a63cec3a744848972fcbb89ecd0ca` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:42.156251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87b3cd-286a-40d0-9e89-dc4965d55648 None admin] RBAC: Authorizing `identity:create_grant(project_id=a05605e22abb477fabc16f82893f5d2f, user_id=3a47e58b1ada450eaf778de493c93b62, role_id=ab7a63cec3a744848972fcbb89ecd0ca)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:42.157721 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f87b3cd-286a-40d0-9e89-dc4965d55648 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:42.210423 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 547/1081] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:08:42.218006 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112eada-312d-4015-96ce-619881eb0150 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:42.235634 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 535/1082] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:08:41 2022] PUT /identity/v3/projects/a05605e22abb477fabc16f82893f5d2f/users/3a47e58b1ada450eaf778de493c93b62/roles/ab7a63cec3a744848972fcbb89ecd0ca => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:08:42.277473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:42.333347 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112eada-312d-4015-96ce-619881eb0150 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:42.370338 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40697d5-99e8-4296-ad49-ac59a82629dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:42.371218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40697d5-99e8-4296-ad49-ac59a82629dd None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:42.371509 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40697d5-99e8-4296-ad49-ac59a82629dd None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:42.371790 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40697d5-99e8-4296-ad49-ac59a82629dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:42.384028 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40697d5-99e8-4296-ad49-ac59a82629dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:42.385552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40697d5-99e8-4296-ad49-ac59a82629dd None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-f40697d5-99e8-4296-ad49-ac59a82629dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:42.405366 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:42.406320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:42.407427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:42.407427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:42.407427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:42.407427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:42.407427 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 548/1083] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:42.412866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87b3cd-286a-40d0-9e89-dc4965d55648 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:42.523526 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f87b3cd-286a-40d0-9e89-dc4965d55648 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:42.540778 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:42.561215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5604a5-bfe7-4056-ae9f-eccc9f383ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:42.561938 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5604a5-bfe7-4056-ae9f-eccc9f383ced None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:42.562219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5604a5-bfe7-4056-ae9f-eccc9f383ced None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:42.562496 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5604a5-bfe7-4056-ae9f-eccc9f383ced None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:42.563222 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5604a5-bfe7-4056-ae9f-eccc9f383ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:42.563531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5604a5-bfe7-4056-ae9f-eccc9f383ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:42.564968 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5604a5-bfe7-4056-ae9f-eccc9f383ced None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:42.578242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 536/1084] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:43.980106 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea4c4e-f1e6-46d6-9312-d69ca379fe82 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:43.980719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea4c4e-f1e6-46d6-9312-d69ca379fe82 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:43.980719 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea4c4e-f1e6-46d6-9312-d69ca379fe82 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:43.982270 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 549/1085] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:43.987331 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f5493-5c7f-40fc-9511-56180b829da6 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:43.987735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f5493-5c7f-40fc-9511-56180b829da6 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:43.988005 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f5493-5c7f-40fc-9511-56180b829da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:44.004926 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-c63f5493-5c7f-40fc-9511-56180b829da6 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:44.006112 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-c63f5493-5c7f-40fc-9511-56180b829da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:44.075651 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 537/1086] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:44.099305 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea4c4e-f1e6-46d6-9312-d69ca379fe82 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:44.207695 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea4c4e-f1e6-46d6-9312-d69ca379fe82 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:44.210235 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:44.242660 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faf57f-88cd-454d-900c-020fe2abda80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:44.243396 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faf57f-88cd-454d-900c-020fe2abda80 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:44.243682 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faf57f-88cd-454d-900c-020fe2abda80 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:44.243958 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faf57f-88cd-454d-900c-020fe2abda80 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:44.254592 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1faf57f-88cd-454d-900c-020fe2abda80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-d1faf57f-88cd-454d-900c-020fe2abda80 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:44.263486 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:44.264606 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:44.265960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 550/1087] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:08:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:44.270679 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f5493-5c7f-40fc-9511-56180b829da6 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:44.376856 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f5493-5c7f-40fc-9511-56180b829da6 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:44.383882 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:44.411205 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7049826-a305-4f2b-9696-df4584db4ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:44.411876 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7049826-a305-4f2b-9696-df4584db4ecb None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:44.412164 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7049826-a305-4f2b-9696-df4584db4ecb None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:44.412481 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7049826-a305-4f2b-9696-df4584db4ecb None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:44.413231 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7049826-a305-4f2b-9696-df4584db4ecb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:44.413542 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7049826-a305-4f2b-9696-df4584db4ecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:44.414916 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7049826-a305-4f2b-9696-df4584db4ecb None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:44.424053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 538/1088] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:08:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:44.429708 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faf57f-88cd-454d-900c-020fe2abda80 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:44.505090 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf028db-b04c-497c-bc5b-f364a4daf228 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:44.505308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf028db-b04c-497c-bc5b-f364a4daf228 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:44.505589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf028db-b04c-497c-bc5b-f364a4daf228 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:44.507017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 539/1089] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:08:44.512275 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72358b-061e-4262-865b-320f941c6b01 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:44.512597 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72358b-061e-4262-865b-320f941c6b01 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:44.512875 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72358b-061e-4262-865b-320f941c6b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:44.529758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-0f72358b-061e-4262-865b-320f941c6b01 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:44.530934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-0f72358b-061e-4262-865b-320f941c6b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:44.531150 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faf57f-88cd-454d-900c-020fe2abda80 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:44.566721 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca65c95-aef6-4d74-9020-2fc7db910a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:44.567511 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca65c95-aef6-4d74-9020-2fc7db910a9b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:44.567788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca65c95-aef6-4d74-9020-2fc7db910a9b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:44.568059 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca65c95-aef6-4d74-9020-2fc7db910a9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:44.578720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca65c95-aef6-4d74-9020-2fc7db910a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:44.580278 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca65c95-aef6-4d74-9020-2fc7db910a9b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-aca65c95-aef6-4d74-9020-2fc7db910a9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:44.597804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:44.598901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:44.599864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:44.599864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:44.599864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:44.599864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:44.599864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 551/1090] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:44.602250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 540/1091] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:08:44.605442 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca65c95-aef6-4d74-9020-2fc7db910a9b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:44.623932 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:44.625917 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72358b-061e-4262-865b-320f941c6b01 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:44.706543 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca65c95-aef6-4d74-9020-2fc7db910a9b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:44.726346 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:44.730307 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f72358b-061e-4262-865b-320f941c6b01 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:44.742461 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c184b-251a-472d-9cc7-4cf98a1c369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:44.743183 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c184b-251a-472d-9cc7-4cf98a1c369e None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:44.743474 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c184b-251a-472d-9cc7-4cf98a1c369e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:44.743759 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c184b-251a-472d-9cc7-4cf98a1c369e None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:44.744517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c184b-251a-472d-9cc7-4cf98a1c369e None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:44.744837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c184b-251a-472d-9cc7-4cf98a1c369e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:44.746229 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c184b-251a-472d-9cc7-4cf98a1c369e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:44.758120 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 552/1092] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:44.764233 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c184b-251a-472d-9cc7-4cf98a1c369e None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:44.767882 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5ee66-4e7f-4603-9c1a-f5815ca0238b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:44.768711 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ee66-4e7f-4603-9c1a-f5815ca0238b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:44.768993 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ee66-4e7f-4603-9c1a-f5815ca0238b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:44.769268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5ee66-4e7f-4603-9c1a-f5815ca0238b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:44.780765 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b5ee66-4e7f-4603-9c1a-f5815ca0238b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-e8b5ee66-4e7f-4603-9c1a-f5815ca0238b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:08:44.788366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:44.789444 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:44.795369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 541/1093] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:08:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:44.796422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5ee66-4e7f-4603-9c1a-f5815ca0238b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:44.850847 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:44.875056 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c184b-251a-472d-9cc7-4cf98a1c369e None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:44.903048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5ee66-4e7f-4603-9c1a-f5815ca0238b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:44.911844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdbd98-4a8a-44cb-8d07-905aa4363cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:44.912701 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdbd98-4a8a-44cb-8d07-905aa4363cd0 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:44.912985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdbd98-4a8a-44cb-8d07-905aa4363cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:44.913266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdbd98-4a8a-44cb-8d07-905aa4363cd0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:44.920872 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bdbd98-4a8a-44cb-8d07-905aa4363cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:44.922321 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bdbd98-4a8a-44cb-8d07-905aa4363cd0 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-11bdbd98-4a8a-44cb-8d07-905aa4363cd0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:08:44.936747 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:44.937900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:44.939019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:44.939019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:44.939019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:08:44.939019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:08:44.939019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:08:44.939019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:08:44.939019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:08:44.939019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:08:44.939019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:08:44.939019 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:44.939527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa30a0-a28d-4b65-aefc-ca25e6c484a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:44.939527 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa30a0-a28d-4b65-aefc-ca25e6c484a4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:44.939688 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 553/1094] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:08:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:44.940568 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa30a0-a28d-4b65-aefc-ca25e6c484a4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:44.940864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa30a0-a28d-4b65-aefc-ca25e6c484a4 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:44.941682 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa30a0-a28d-4b65-aefc-ca25e6c484a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:44.941993 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa30a0-a28d-4b65-aefc-ca25e6c484a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:44.943649 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa30a0-a28d-4b65-aefc-ca25e6c484a4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:44.943775 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdbd98-4a8a-44cb-8d07-905aa4363cd0 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:44.952905 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 542/1095] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:08:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:44.958838 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa30a0-a28d-4b65-aefc-ca25e6c484a4 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:44.984483 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:45.048399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdbd98-4a8a-44cb-8d07-905aa4363cd0 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:45.067592 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa30a0-a28d-4b65-aefc-ca25e6c484a4 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:45.086382 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d973-17e5-49ec-8618-32b25481edfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:45.087065 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d973-17e5-49ec-8618-32b25481edfb None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:45.087347 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d973-17e5-49ec-8618-32b25481edfb None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:45.087627 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c70d973-17e5-49ec-8618-32b25481edfb None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:45.088503 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d973-17e5-49ec-8618-32b25481edfb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:45.088816 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d973-17e5-49ec-8618-32b25481edfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:45.090269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c70d973-17e5-49ec-8618-32b25481edfb None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:45.094260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:45.107657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14483674-a8ea-4f67-afe0-4ad082e47073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:45.108570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14483674-a8ea-4f67-afe0-4ad082e47073 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:45.108853 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14483674-a8ea-4f67-afe0-4ad082e47073 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:45.109132 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14483674-a8ea-4f67-afe0-4ad082e47073 None admin] PATH_INFO: `/v3/users/3a47e58b1ada450eaf778de493c93b62` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:45.109737 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 554/1096] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:08:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:45.110375 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14483674-a8ea-4f67-afe0-4ad082e47073 None admin] RBAC: Authorizing `identity:get_user(user_id=3a47e58b1ada450eaf778de493c93b62)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:45.111829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14483674-a8ea-4f67-afe0-4ad082e47073 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:45.113585 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 543/1097] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:08:44 2022] GET /identity/v3/users/3a47e58b1ada450eaf778de493c93b62 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:45.115396 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d973-17e5-49ec-8618-32b25481edfb None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:45.119313 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14483674-a8ea-4f67-afe0-4ad082e47073 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:45.214213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:45.226543 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14483674-a8ea-4f67-afe0-4ad082e47073 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:45.231409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c70d973-17e5-49ec-8618-32b25481edfb None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:45.264018 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac69ed0-a5e2-44b3-b614-47050d17b7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:45.264692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac69ed0-a5e2-44b3-b614-47050d17b7aa None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:45.264976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac69ed0-a5e2-44b3-b614-47050d17b7aa None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:45.265250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac69ed0-a5e2-44b3-b614-47050d17b7aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:45.268843 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d72d54-e29a-4470-9685-4b6a15fe3179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:45.269492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d72d54-e29a-4470-9685-4b6a15fe3179 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:45.269775 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d72d54-e29a-4470-9685-4b6a15fe3179 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:45.270073 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d72d54-e29a-4470-9685-4b6a15fe3179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:45.332857 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:45.334470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac69ed0-a5e2-44b3-b614-47050d17b7aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:45.335918 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac69ed0-a5e2-44b3-b614-47050d17b7aa None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:45.339954 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d72d54-e29a-4470-9685-4b6a15fe3179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:45.341469 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d72d54-e29a-4470-9685-4b6a15fe3179 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:45.423037 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:45.435984 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 544/1098] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:08:45.442070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac69ed0-a5e2-44b3-b614-47050d17b7aa None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:45.448100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 555/1099] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:08:45.449182 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d72d54-e29a-4470-9685-4b6a15fe3179 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:45.546550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:45.547005 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac69ed0-a5e2-44b3-b614-47050d17b7aa None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:45.554080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d72d54-e29a-4470-9685-4b6a15fe3179 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:45.583303 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ffe64-3f4b-4cf2-be5f-6aa65cbca923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:45.584082 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ffe64-3f4b-4cf2-be5f-6aa65cbca923 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:45.584391 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ffe64-3f4b-4cf2-be5f-6aa65cbca923 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:45.584672 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ffe64-3f4b-4cf2-be5f-6aa65cbca923 None admin] PATH_INFO: `/v3/projects/a05605e22abb477fabc16f82893f5d2f` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:45.585865 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ffe64-3f4b-4cf2-be5f-6aa65cbca923 None admin] RBAC: Authorizing `identity:get_project(project_id=a05605e22abb477fabc16f82893f5d2f)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:45.587301 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ffe64-3f4b-4cf2-be5f-6aa65cbca923 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:45.589054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da6dad-48e8-4e6b-8d62-4b0e8ab99a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:45.589221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 545/1100] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:08:45 2022] GET /identity/v3/projects/a05605e22abb477fabc16f82893f5d2f => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:45.589840 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da6dad-48e8-4e6b-8d62-4b0e8ab99a79 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:45.590121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da6dad-48e8-4e6b-8d62-4b0e8ab99a79 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:45.590395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da6dad-48e8-4e6b-8d62-4b0e8ab99a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:45.595701 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ffe64-3f4b-4cf2-be5f-6aa65cbca923 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:45.601424 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da6dad-48e8-4e6b-8d62-4b0e8ab99a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:45.602878 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da6dad-48e8-4e6b-8d62-4b0e8ab99a79 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-d1da6dad-48e8-4e6b-8d62-4b0e8ab99a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:08:45.620985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:45.621928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:45.623009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:08:45.623009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:08:45.623009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:08:45.623009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:08:45.623009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 556/1101] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:08:45.627752 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da6dad-48e8-4e6b-8d62-4b0e8ab99a79 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:45.653203 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:45.697546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ffe64-3f4b-4cf2-be5f-6aa65cbca923 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:45.727761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da6dad-48e8-4e6b-8d62-4b0e8ab99a79 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:45.733432 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71449ec-8b6f-4384-aafd-25904f35745d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:45.734088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71449ec-8b6f-4384-aafd-25904f35745d None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:45.734373 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71449ec-8b6f-4384-aafd-25904f35745d None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:45.734642 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71449ec-8b6f-4384-aafd-25904f35745d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:45.735477 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71449ec-8b6f-4384-aafd-25904f35745d None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=a05605e22abb477fabc16f82893f5d2f, user.id=3a47e58b1ada450eaf778de493c93b62) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:45.735788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71449ec-8b6f-4384-aafd-25904f35745d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:45.737251 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71449ec-8b6f-4384-aafd-25904f35745d None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:45.743485 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 546/1102] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:08:45 2022] GET /identity/v3/role_assignments?user.id=3a47e58b1ada450eaf778de493c93b62&scope.project.id=a05605e22abb477fabc16f82893f5d2f&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:45.762384 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0760e-a207-47f6-a40a-8ccb48630b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:45.763103 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0760e-a207-47f6-a40a-8ccb48630b19 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:45.763931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0760e-a207-47f6-a40a-8ccb48630b19 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:45.764208 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0760e-a207-47f6-a40a-8ccb48630b19 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:45.764996 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0760e-a207-47f6-a40a-8ccb48630b19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:45.765309 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0760e-a207-47f6-a40a-8ccb48630b19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:45.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0760e-a207-47f6-a40a-8ccb48630b19 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:45.780170 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 557/1103] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:45.786073 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71449ec-8b6f-4384-aafd-25904f35745d None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:45.895510 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71449ec-8b6f-4384-aafd-25904f35745d None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:45.917980 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:45.932767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb273f-eb50-4553-93ee-15d5592e9fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:45.937114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb273f-eb50-4553-93ee-15d5592e9fc6 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:45.937114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb273f-eb50-4553-93ee-15d5592e9fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:45.937114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb273f-eb50-4553-93ee-15d5592e9fc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:45.942795 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb273f-eb50-4553-93ee-15d5592e9fc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:45.944285 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb273f-eb50-4553-93ee-15d5592e9fc6 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-10fb273f-eb50-4553-93ee-15d5592e9fc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:08:45.960075 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:08:45.961232 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:08:45.962168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:08:45.962168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:08:45.962168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:08:45.962168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:08:45.962168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:08:45.962168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:08:45.962168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:08:45.962168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:08:45.962168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 547/1104] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:08:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:08:45.967557 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0760e-a207-47f6-a40a-8ccb48630b19 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:45.976921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:46.068671 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0760e-a207-47f6-a40a-8ccb48630b19 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:46.103038 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde26dc-dd2d-4504-9d9e-b41b77b8bd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:46.103708 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde26dc-dd2d-4504-9d9e-b41b77b8bd10 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:46.103990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde26dc-dd2d-4504-9d9e-b41b77b8bd10 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:46.104263 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde26dc-dd2d-4504-9d9e-b41b77b8bd10 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:46.105087 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde26dc-dd2d-4504-9d9e-b41b77b8bd10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:46.105397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde26dc-dd2d-4504-9d9e-b41b77b8bd10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:46.106847 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde26dc-dd2d-4504-9d9e-b41b77b8bd10 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:46.119259 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 558/1105] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:08:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:46.125719 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb273f-eb50-4553-93ee-15d5592e9fc6 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:46.228068 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb273f-eb50-4553-93ee-15d5592e9fc6 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:46.262628 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4c08e-a1a7-4889-a89e-9485ecc9a913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:46.263325 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4c08e-a1a7-4889-a89e-9485ecc9a913 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:46.264110 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4c08e-a1a7-4889-a89e-9485ecc9a913 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:46.264409 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4c08e-a1a7-4889-a89e-9485ecc9a913 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:46.265235 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4c08e-a1a7-4889-a89e-9485ecc9a913 None admin] RBAC: Adding query filter params (role.id=07c8382535c74f80a514602a3dbe9a7f, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=f6d44c613f524fe1804d6c9b65fe97ca) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:46.265548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4c08e-a1a7-4889-a89e-9485ecc9a913 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:46.266970 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4c08e-a1a7-4889-a89e-9485ecc9a913 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:46.272867 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 548/1106] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:08:46 2022] GET /identity/v3/role_assignments?user.id=f6d44c613f524fe1804d6c9b65fe97ca&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=07c8382535c74f80a514602a3dbe9a7f => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:48.029848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7699f0-5729-467f-ad6f-3e6450ccd5e1 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:48.030378 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7699f0-5729-467f-ad6f-3e6450ccd5e1 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:48.030378 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7699f0-5729-467f-ad6f-3e6450ccd5e1 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:48.031770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 559/1107] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:48.037215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7404e4-3537-42a9-84b5-0c451b98f7ee None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:48.037632 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7404e4-3537-42a9-84b5-0c451b98f7ee None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:48.037766 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7404e4-3537-42a9-84b5-0c451b98f7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:48.055438 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-8e7404e4-3537-42a9-84b5-0c451b98f7ee None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:48.056655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-8e7404e4-3537-42a9-84b5-0c451b98f7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:48.097946 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:48.124467 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 549/1108] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:48.149413 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7699f0-5729-467f-ad6f-3e6450ccd5e1 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:48.217464 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:48.271540 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7699f0-5729-467f-ad6f-3e6450ccd5e1 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:48.305609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69d322-a206-4ddf-93fc-e66d8de93734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:48.306325 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69d322-a206-4ddf-93fc-e66d8de93734 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:48.306601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69d322-a206-4ddf-93fc-e66d8de93734 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:48.306871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69d322-a206-4ddf-93fc-e66d8de93734 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:48.308206 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69d322-a206-4ddf-93fc-e66d8de93734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:48.309678 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69d322-a206-4ddf-93fc-e66d8de93734 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:48.311829 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 560/1109] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:08:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:48.317549 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7404e4-3537-42a9-84b5-0c451b98f7ee None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:48.415079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7404e4-3537-42a9-84b5-0c451b98f7ee None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:48.448602 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f6615-e4a4-4463-ae36-028ee96f164e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:48.449258 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6615-e4a4-4463-ae36-028ee96f164e None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:48.449538 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6615-e4a4-4463-ae36-028ee96f164e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:48.449811 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f6615-e4a4-4463-ae36-028ee96f164e None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:48.450525 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f6615-e4a4-4463-ae36-028ee96f164e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:48.452562 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f6615-e4a4-4463-ae36-028ee96f164e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:48.567378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 550/1110] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:08:48 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:08:50.849936 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2b77-8fd0-49ed-a6dc-fd1a0472e616 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:50.850544 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2b77-8fd0-49ed-a6dc-fd1a0472e616 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:50.850544 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2b77-8fd0-49ed-a6dc-fd1a0472e616 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:50.851894 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 561/1111] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:50.857658 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1f1cc-984b-4b9f-b367-2a708fe5e5d9 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:50.858058 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1f1cc-984b-4b9f-b367-2a708fe5e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:50.858217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1f1cc-984b-4b9f-b367-2a708fe5e5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:50.875624 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-16e1f1cc-984b-4b9f-b367-2a708fe5e5d9 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:50.876845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-16e1f1cc-984b-4b9f-b367-2a708fe5e5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:50.900126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:50.944700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 551/1112] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:50.968287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef2b77-8fd0-49ed-a6dc-fd1a0472e616 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:51.070436 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef2b77-8fd0-49ed-a6dc-fd1a0472e616 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:51.104605 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecab696-de9b-472f-9832-89954cd7bbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:51.105393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecab696-de9b-472f-9832-89954cd7bbb1 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:51.105673 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecab696-de9b-472f-9832-89954cd7bbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:51.105943 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecab696-de9b-472f-9832-89954cd7bbb1 None admin] PATH_INFO: `/v3/roles/c9fbfe89c48d458c94f2a8a11652fcf2` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:51.108007 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecab696-de9b-472f-9832-89954cd7bbb1 None admin] RBAC: Authorizing `identity:get_role(role_id=c9fbfe89c48d458c94f2a8a11652fcf2)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:51.109428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecab696-de9b-472f-9832-89954cd7bbb1 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:51.110797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 562/1113] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:08:50 2022] GET /identity/v3/roles/c9fbfe89c48d458c94f2a8a11652fcf2 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:51.116566 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1f1cc-984b-4b9f-b367-2a708fe5e5d9 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:51.218995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1f1cc-984b-4b9f-b367-2a708fe5e5d9 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:51.253692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a7f98-f6ba-442e-a091-dbce14c12fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:51.254554 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a7f98-f6ba-442e-a091-dbce14c12fad None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:51.254837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a7f98-f6ba-442e-a091-dbce14c12fad None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:51.255113 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a7f98-f6ba-442e-a091-dbce14c12fad None admin] PATH_INFO: `/v3/users/40891b71c9b543d1967e0cdb2ba23c7c` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:51.268578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a7f98-f6ba-442e-a091-dbce14c12fad None admin] RBAC: Authorizing `identity:get_user(user_id=40891b71c9b543d1967e0cdb2ba23c7c)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:51.270065 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a7f98-f6ba-442e-a091-dbce14c12fad None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:51.271740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 552/1114] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:08:51 2022] GET /identity/v3/users/40891b71c9b543d1967e0cdb2ba23c7c => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:51.277527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecab696-de9b-472f-9832-89954cd7bbb1 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:51.280699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:51.378424 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecab696-de9b-472f-9832-89954cd7bbb1 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:51.412326 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9faf863-a428-46c1-8ec0-0c35fa9200ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:51.412993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faf863-a428-46c1-8ec0-0c35fa9200ff None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:51.413277 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faf863-a428-46c1-8ec0-0c35fa9200ff None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:51.413553 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faf863-a428-46c1-8ec0-0c35fa9200ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:51.480188 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9faf863-a428-46c1-8ec0-0c35fa9200ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:51.481679 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9faf863-a428-46c1-8ec0-0c35fa9200ff None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:51.484079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:51.579103 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 563/1115] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:08:51.585149 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a7f98-f6ba-442e-a091-dbce14c12fad None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:51.604422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:51.684832 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a7f98-f6ba-442e-a091-dbce14c12fad None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:51.718426 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81e3e-e8e6-4281-a640-2a36b9dbac2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:51.719273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81e3e-e8e6-4281-a640-2a36b9dbac2e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:51.719555 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81e3e-e8e6-4281-a640-2a36b9dbac2e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:51.719830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81e3e-e8e6-4281-a640-2a36b9dbac2e None admin] PATH_INFO: `/v3/projects/a05605e22abb477fabc16f82893f5d2f` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:51.721290 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81e3e-e8e6-4281-a640-2a36b9dbac2e None admin] RBAC: Authorizing `identity:get_project(project_id=a05605e22abb477fabc16f82893f5d2f)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:51.722720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81e3e-e8e6-4281-a640-2a36b9dbac2e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:51.724431 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 553/1116] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:08:51 2022] GET /identity/v3/projects/a05605e22abb477fabc16f82893f5d2f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:51.730641 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9faf863-a428-46c1-8ec0-0c35fa9200ff None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:51.829881 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9faf863-a428-46c1-8ec0-0c35fa9200ff None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:51.831436 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:51.863742 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ea886-2f08-424a-898b-3ca767d26684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:51.864417 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ea886-2f08-424a-898b-3ca767d26684 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:51.864705 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ea886-2f08-424a-898b-3ca767d26684 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:51.864983 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ea886-2f08-424a-898b-3ca767d26684 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:51.865810 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ea886-2f08-424a-898b-3ca767d26684 None admin] RBAC: Adding query filter params (role.id=c9fbfe89c48d458c94f2a8a11652fcf2, scope.project.id=a05605e22abb477fabc16f82893f5d2f, user.id=40891b71c9b543d1967e0cdb2ba23c7c) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:51.866127 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ea886-2f08-424a-898b-3ca767d26684 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:51.867587 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ea886-2f08-424a-898b-3ca767d26684 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:51.873916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 564/1117] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:08:51 2022] GET /identity/v3/role_assignments?user.id=40891b71c9b543d1967e0cdb2ba23c7c&scope.project.id=a05605e22abb477fabc16f82893f5d2f&role.id=c9fbfe89c48d458c94f2a8a11652fcf2 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:54.067456 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8033c-1e4e-42c6-97ca-23652b4f2986 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:54.067992 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8033c-1e4e-42c6-97ca-23652b4f2986 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:54.067992 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8033c-1e4e-42c6-97ca-23652b4f2986 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:54.069477 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 554/1118] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:54.074925 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba55b8d-d9a2-4155-87f9-758210825c05 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:54.075346 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba55b8d-d9a2-4155-87f9-758210825c05 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:54.075560 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba55b8d-d9a2-4155-87f9-758210825c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:54.092360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-1ba55b8d-d9a2-4155-87f9-758210825c05 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:54.093549 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-1ba55b8d-d9a2-4155-87f9-758210825c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:54.160754 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 565/1119] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:54.184027 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8033c-1e4e-42c6-97ca-23652b4f2986 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:54.268829 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:54.285128 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8033c-1e4e-42c6-97ca-23652b4f2986 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:54.319309 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1789abc-539f-4f3e-8d03-af60c2326f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:54.320085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1789abc-539f-4f3e-8d03-af60c2326f1b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:54.320388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1789abc-539f-4f3e-8d03-af60c2326f1b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:54.320665 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1789abc-539f-4f3e-8d03-af60c2326f1b None admin] PATH_INFO: `/v3/roles/c9fbfe89c48d458c94f2a8a11652fcf2` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:54.322126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1789abc-539f-4f3e-8d03-af60c2326f1b None admin] RBAC: Authorizing `identity:get_role(role_id=c9fbfe89c48d458c94f2a8a11652fcf2)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:54.323502 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1789abc-539f-4f3e-8d03-af60c2326f1b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:54.325022 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 555/1120] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:08:54 2022] GET /identity/v3/roles/c9fbfe89c48d458c94f2a8a11652fcf2 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:54.330918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba55b8d-d9a2-4155-87f9-758210825c05 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:54.428016 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:54.433650 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba55b8d-d9a2-4155-87f9-758210825c05 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:54.467048 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881d86f-337b-4600-90cf-7ea9c12293e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:54.468221 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881d86f-337b-4600-90cf-7ea9c12293e3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:54.468527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881d86f-337b-4600-90cf-7ea9c12293e3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:54.468805 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881d86f-337b-4600-90cf-7ea9c12293e3 None admin] PATH_INFO: `/v3/users/40891b71c9b543d1967e0cdb2ba23c7c` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:54.469999 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881d86f-337b-4600-90cf-7ea9c12293e3 None admin] RBAC: Authorizing `identity:get_user(user_id=40891b71c9b543d1967e0cdb2ba23c7c)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:54.471435 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881d86f-337b-4600-90cf-7ea9c12293e3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:54.473104 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 566/1121] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:08:54 2022] GET /identity/v3/users/40891b71c9b543d1967e0cdb2ba23c7c => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:54.479032 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1789abc-539f-4f3e-8d03-af60c2326f1b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:54.577694 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1789abc-539f-4f3e-8d03-af60c2326f1b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:54.610659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34d899-3091-4326-9224-5cddd72d2695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:54.611505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34d899-3091-4326-9224-5cddd72d2695 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:54.611782 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34d899-3091-4326-9224-5cddd72d2695 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:54.612053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34d899-3091-4326-9224-5cddd72d2695 None admin] PATH_INFO: `/v3/projects/a05605e22abb477fabc16f82893f5d2f` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:54.613245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34d899-3091-4326-9224-5cddd72d2695 None admin] RBAC: Authorizing `identity:get_project(project_id=a05605e22abb477fabc16f82893f5d2f)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:54.614684 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34d899-3091-4326-9224-5cddd72d2695 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:54.616396 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 556/1122] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:08:54 2022] GET /identity/v3/projects/a05605e22abb477fabc16f82893f5d2f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:08:54.622329 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881d86f-337b-4600-90cf-7ea9c12293e3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:54.723870 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881d86f-337b-4600-90cf-7ea9c12293e3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:54.757809 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f2719e-10dc-4897-becb-413158cba824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:54.758754 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2719e-10dc-4897-becb-413158cba824 None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:54.759041 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2719e-10dc-4897-becb-413158cba824 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:54.759320 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f2719e-10dc-4897-becb-413158cba824 None admin] PATH_INFO: `/v3/projects/a05605e22abb477fabc16f82893f5d2f/users/40891b71c9b543d1967e0cdb2ba23c7c/roles/c9fbfe89c48d458c94f2a8a11652fcf2` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:54.761561 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2719e-10dc-4897-becb-413158cba824 None admin] RBAC: Authorizing `identity:create_grant(project_id=a05605e22abb477fabc16f82893f5d2f, user_id=40891b71c9b543d1967e0cdb2ba23c7c, role_id=c9fbfe89c48d458c94f2a8a11652fcf2)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:54.763603 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f2719e-10dc-4897-becb-413158cba824 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:54.862996 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 567/1123] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:08:54 2022] PUT /identity/v3/projects/a05605e22abb477fabc16f82893f5d2f/users/40891b71c9b543d1967e0cdb2ba23c7c/roles/c9fbfe89c48d458c94f2a8a11652fcf2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:08:57.108976 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430726a-b5e7-4275-a5c8-d11ed3d5f999 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:57.109496 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430726a-b5e7-4275-a5c8-d11ed3d5f999 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:57.109587 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430726a-b5e7-4275-a5c8-d11ed3d5f999 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:57.110902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 557/1124] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:08:57.116577 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6ca1e-984f-437a-8092-64790abee998 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:57.117002 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6ca1e-984f-437a-8092-64790abee998 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:57.117140 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6ca1e-984f-437a-8092-64790abee998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:57.134488 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-86b6ca1e-984f-437a-8092-64790abee998 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:08:57.135870 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-86b6ca1e-984f-437a-8092-64790abee998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:08:57.153239 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:57.205857 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 568/1125] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:08:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:08:57.230365 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430726a-b5e7-4275-a5c8-d11ed3d5f999 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:57.255785 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:57.334460 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430726a-b5e7-4275-a5c8-d11ed3d5f999 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:57.366862 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c0f61-1c5f-4df6-965f-f694b2ea70a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:57.367711 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c0f61-1c5f-4df6-965f-f694b2ea70a4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:57.367992 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c0f61-1c5f-4df6-965f-f694b2ea70a4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:57.368268 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c0f61-1c5f-4df6-965f-f694b2ea70a4 None admin] PATH_INFO: `/v3/roles/c9fbfe89c48d458c94f2a8a11652fcf2` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:57.369748 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c0f61-1c5f-4df6-965f-f694b2ea70a4 None admin] RBAC: Authorizing `identity:get_role(role_id=c9fbfe89c48d458c94f2a8a11652fcf2)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:57.371135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c0f61-1c5f-4df6-965f-f694b2ea70a4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:57.373050 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 558/1126] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:08:57 2022] GET /identity/v3/roles/c9fbfe89c48d458c94f2a8a11652fcf2 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:57.378799 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6ca1e-984f-437a-8092-64790abee998 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:57.476224 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6ca1e-984f-437a-8092-64790abee998 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:57.509460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97466280-56b4-4411-aafc-89b0bb38be08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:57.510317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97466280-56b4-4411-aafc-89b0bb38be08 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:57.510593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97466280-56b4-4411-aafc-89b0bb38be08 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:57.510864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97466280-56b4-4411-aafc-89b0bb38be08 None admin] PATH_INFO: `/v3/users/40891b71c9b543d1967e0cdb2ba23c7c` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:57.512038 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97466280-56b4-4411-aafc-89b0bb38be08 None admin] RBAC: Authorizing `identity:get_user(user_id=40891b71c9b543d1967e0cdb2ba23c7c)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:57.514079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97466280-56b4-4411-aafc-89b0bb38be08 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:57.515690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 569/1127] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:08:57 2022] GET /identity/v3/users/40891b71c9b543d1967e0cdb2ba23c7c => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:57.521648 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c0f61-1c5f-4df6-965f-f694b2ea70a4 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:57.611490 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:57.620736 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c0f61-1c5f-4df6-965f-f694b2ea70a4 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:57.654014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622be7a7-93cb-4acf-82f9-ade9f3d49218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:57.654655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622be7a7-93cb-4acf-82f9-ade9f3d49218 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:57.654934 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622be7a7-93cb-4acf-82f9-ade9f3d49218 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:57.655243 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622be7a7-93cb-4acf-82f9-ade9f3d49218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:57.720836 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622be7a7-93cb-4acf-82f9-ade9f3d49218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:57.722270 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622be7a7-93cb-4acf-82f9-ade9f3d49218 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:57.809151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:57.819785 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 559/1128] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:08:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:08:57.825850 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97466280-56b4-4411-aafc-89b0bb38be08 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:57.850269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:08:57.926926 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97466280-56b4-4411-aafc-89b0bb38be08 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:57.960865 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcfc71-b02d-4d77-87d1-0677916cd016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:57.961722 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcfc71-b02d-4d77-87d1-0677916cd016 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:57.962009 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcfc71-b02d-4d77-87d1-0677916cd016 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:57.962289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcfc71-b02d-4d77-87d1-0677916cd016 None admin] PATH_INFO: `/v3/projects/a05605e22abb477fabc16f82893f5d2f` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:57.963475 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcfc71-b02d-4d77-87d1-0677916cd016 None admin] RBAC: Authorizing `identity:get_project(project_id=a05605e22abb477fabc16f82893f5d2f)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:57.964971 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bcfc71-b02d-4d77-87d1-0677916cd016 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:57.967232 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 570/1129] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:08:57 2022] GET /identity/v3/projects/a05605e22abb477fabc16f82893f5d2f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:08:57.973340 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622be7a7-93cb-4acf-82f9-ade9f3d49218 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:08:58.070745 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622be7a7-93cb-4acf-82f9-ade9f3d49218 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:08:58.104698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b0ec53-a1e3-441b-bb59-bf9b9426a5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:08:58.105351 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0ec53-a1e3-441b-bb59-bf9b9426a5d1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:08:58.105628 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0ec53-a1e3-441b-bb59-bf9b9426a5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:08:58.105896 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0ec53-a1e3-441b-bb59-bf9b9426a5d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:08:58.107370 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b0ec53-a1e3-441b-bb59-bf9b9426a5d1 None admin] RBAC: Adding query filter params (role.id=c9fbfe89c48d458c94f2a8a11652fcf2, scope.project.id=a05605e22abb477fabc16f82893f5d2f, user.id=40891b71c9b543d1967e0cdb2ba23c7c) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:08:58.107682 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b0ec53-a1e3-441b-bb59-bf9b9426a5d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:08:58.109192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b0ec53-a1e3-441b-bb59-bf9b9426a5d1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:08:58.115230 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 560/1130] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:08:57 2022] GET /identity/v3/role_assignments?user.id=40891b71c9b543d1967e0cdb2ba23c7c&scope.project.id=a05605e22abb477fabc16f82893f5d2f&role.id=c9fbfe89c48d458c94f2a8a11652fcf2 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:00.348944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea829048-3024-4b24-968d-d94564d8d1ae None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:00.349465 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea829048-3024-4b24-968d-d94564d8d1ae None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:00.349465 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea829048-3024-4b24-968d-d94564d8d1ae None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:00.350842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 571/1131] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:00.356200 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba2807-8d5b-4ed8-abfb-f5851391b112 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:00.356635 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba2807-8d5b-4ed8-abfb-f5851391b112 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:00.357388 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba2807-8d5b-4ed8-abfb-f5851391b112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:00.373945 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-40ba2807-8d5b-4ed8-abfb-f5851391b112 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:00.375126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-40ba2807-8d5b-4ed8-abfb-f5851391b112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:00.424621 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:00.441638 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 561/1132] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:00.465203 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea829048-3024-4b24-968d-d94564d8d1ae None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:00.558421 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:00.568896 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea829048-3024-4b24-968d-d94564d8d1ae None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:00.604253 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41004a-6f69-4a39-891f-319ebda82e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:00.605024 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41004a-6f69-4a39-891f-319ebda82e74 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:00.605305 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41004a-6f69-4a39-891f-319ebda82e74 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:00.605578 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41004a-6f69-4a39-891f-319ebda82e74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:00.606921 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41004a-6f69-4a39-891f-319ebda82e74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:00.608417 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41004a-6f69-4a39-891f-319ebda82e74 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:00.610062 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 572/1133] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:09:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:00.616164 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba2807-8d5b-4ed8-abfb-f5851391b112 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:00.734815 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba2807-8d5b-4ed8-abfb-f5851391b112 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:00.768419 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ae692-3f49-419c-92f8-9a3e0b8914de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:00.769086 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ae692-3f49-419c-92f8-9a3e0b8914de None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:00.769369 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ae692-3f49-419c-92f8-9a3e0b8914de None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:00.769645 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ae692-3f49-419c-92f8-9a3e0b8914de None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:00.770368 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ae692-3f49-419c-92f8-9a3e0b8914de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:00.771807 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ae692-3f49-419c-92f8-9a3e0b8914de None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:00.864569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 562/1134] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:09:00 2022] POST /identity/v3/projects => generated 311 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:09:03.122976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2544dde-8204-4957-b175-9744b52cb5ed None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:03.123529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2544dde-8204-4957-b175-9744b52cb5ed None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:03.123529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2544dde-8204-4957-b175-9744b52cb5ed None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:03.124928 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 573/1135] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:03.130115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c26a-a332-4b9b-80ac-c75fe577c193 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:03.130530 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c26a-a332-4b9b-80ac-c75fe577c193 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:03.130682 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c26a-a332-4b9b-80ac-c75fe577c193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:03.148026 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-c741c26a-a332-4b9b-80ac-c75fe577c193 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:03.149246 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-c741c26a-a332-4b9b-80ac-c75fe577c193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:03.178689 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:03.220262 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 563/1136] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:03.245970 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2544dde-8204-4957-b175-9744b52cb5ed None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:03.355302 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2544dde-8204-4957-b175-9744b52cb5ed None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:03.388160 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f351dc-1b4c-4dfa-b80a-56402e6a75d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:03.388967 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f351dc-1b4c-4dfa-b80a-56402e6a75d7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:03.389248 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f351dc-1b4c-4dfa-b80a-56402e6a75d7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:03.389526 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f351dc-1b4c-4dfa-b80a-56402e6a75d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:03.390695 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f351dc-1b4c-4dfa-b80a-56402e6a75d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:03.392142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f351dc-1b4c-4dfa-b80a-56402e6a75d7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:03.393791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 574/1137] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:09:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:03.399559 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741c26a-a332-4b9b-80ac-c75fe577c193 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:03.498038 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741c26a-a332-4b9b-80ac-c75fe577c193 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:03.532136 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a4968-f830-4876-a62f-b18cecba1237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:03.532828 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a4968-f830-4876-a62f-b18cecba1237 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:03.533109 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a4968-f830-4876-a62f-b18cecba1237 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:03.533386 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a4968-f830-4876-a62f-b18cecba1237 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:03.534079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a4968-f830-4876-a62f-b18cecba1237 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:03.535500 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a4968-f830-4876-a62f-b18cecba1237 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:03.651365 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 564/1138] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:09:03 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:09:05.937364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec6c0c-d6dc-468d-b40f-c134699ef550 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:05.937918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec6c0c-d6dc-468d-b40f-c134699ef550 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:05.937918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec6c0c-d6dc-468d-b40f-c134699ef550 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:05.939297 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 575/1139] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:05.944801 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0aa9b-5168-44e7-91ab-8f795c045ec5 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:05.945175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0aa9b-5168-44e7-91ab-8f795c045ec5 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:05.945354 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0aa9b-5168-44e7-91ab-8f795c045ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:05.962782 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-6da0aa9b-5168-44e7-91ab-8f795c045ec5 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:05.964010 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-6da0aa9b-5168-44e7-91ab-8f795c045ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:05.980868 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:06.031347 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 565/1140] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:06.054790 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec6c0c-d6dc-468d-b40f-c134699ef550 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:06.082866 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:06.158694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec6c0c-d6dc-468d-b40f-c134699ef550 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:06.192498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67781e6-9094-4918-80a0-ced8552183de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:06.193210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67781e6-9094-4918-80a0-ced8552183de None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:06.193492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67781e6-9094-4918-80a0-ced8552183de None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:06.193766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67781e6-9094-4918-80a0-ced8552183de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:06.204384 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67781e6-9094-4918-80a0-ced8552183de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-a67781e6-9094-4918-80a0-ced8552183de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:09:06.212100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:06.213399 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:09:06.214997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 576/1141] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:09:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:09:06.219552 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0aa9b-5168-44e7-91ab-8f795c045ec5 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:06.317856 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da0aa9b-5168-44e7-91ab-8f795c045ec5 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:06.351305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b837ac-740f-4cb1-863a-d8838a3e5bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:06.351962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b837ac-740f-4cb1-863a-d8838a3e5bc0 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:06.352241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b837ac-740f-4cb1-863a-d8838a3e5bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:06.352567 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b837ac-740f-4cb1-863a-d8838a3e5bc0 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:06.353299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b837ac-740f-4cb1-863a-d8838a3e5bc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:06.353601 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b837ac-740f-4cb1-863a-d8838a3e5bc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:06.354964 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b837ac-740f-4cb1-863a-d8838a3e5bc0 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:06.363005 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 566/1142] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:09:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:06.368998 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67781e6-9094-4918-80a0-ced8552183de None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:06.455443 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:06.469876 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67781e6-9094-4918-80a0-ced8552183de None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:06.504104 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5ceaa-2024-4783-b3e9-75ee45130ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:06.505000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5ceaa-2024-4783-b3e9-75ee45130ad8 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:06.505290 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5ceaa-2024-4783-b3e9-75ee45130ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:06.505571 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5ceaa-2024-4783-b3e9-75ee45130ad8 None admin] PATH_INFO: `/v3/users/2630ca8a93174d679b2ee0f7b6c60715` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:06.519352 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5ceaa-2024-4783-b3e9-75ee45130ad8 None admin] RBAC: Authorizing `identity:get_user(user_id=2630ca8a93174d679b2ee0f7b6c60715)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:06.520842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5ceaa-2024-4783-b3e9-75ee45130ad8 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:06.522495 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 577/1143] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:09:06 2022] GET /identity/v3/users/2630ca8a93174d679b2ee0f7b6c60715 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:06.528296 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b837ac-740f-4cb1-863a-d8838a3e5bc0 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:06.545194 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:06.628159 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b837ac-740f-4cb1-863a-d8838a3e5bc0 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:06.660600 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac645b-9951-4f46-8769-06fc642fdf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:06.661256 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac645b-9951-4f46-8769-06fc642fdf56 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:06.661538 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac645b-9951-4f46-8769-06fc642fdf56 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:06.661809 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac645b-9951-4f46-8769-06fc642fdf56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:06.728135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac645b-9951-4f46-8769-06fc642fdf56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:06.729572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac645b-9951-4f46-8769-06fc642fdf56 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:06.743929 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:06.827103 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 567/1144] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:09:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:09:06.833495 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5ceaa-2024-4783-b3e9-75ee45130ad8 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:06.933704 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5ceaa-2024-4783-b3e9-75ee45130ad8 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:06.967109 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7493c88-b8a7-4608-b69e-6d0dbc207b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:06.967960 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7493c88-b8a7-4608-b69e-6d0dbc207b30 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:06.968247 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7493c88-b8a7-4608-b69e-6d0dbc207b30 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:06.968554 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7493c88-b8a7-4608-b69e-6d0dbc207b30 None admin] PATH_INFO: `/v3/projects/7d85dc24ac444aa19d3c8b39c10541e4` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:06.969727 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7493c88-b8a7-4608-b69e-6d0dbc207b30 None admin] RBAC: Authorizing `identity:get_project(project_id=7d85dc24ac444aa19d3c8b39c10541e4)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:06.971177 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7493c88-b8a7-4608-b69e-6d0dbc207b30 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:06.972911 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 578/1145] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:09:06 2022] GET /identity/v3/projects/7d85dc24ac444aa19d3c8b39c10541e4 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:06.979115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac645b-9951-4f46-8769-06fc642fdf56 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:07.078710 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac645b-9951-4f46-8769-06fc642fdf56 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:07.093621 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:07.113806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33ae6b-ece1-42cd-8059-58044cebd108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:07.114459 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33ae6b-ece1-42cd-8059-58044cebd108 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:07.119740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33ae6b-ece1-42cd-8059-58044cebd108 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:07.119740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33ae6b-ece1-42cd-8059-58044cebd108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:07.119740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33ae6b-ece1-42cd-8059-58044cebd108 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=7d85dc24ac444aa19d3c8b39c10541e4, user.id=2630ca8a93174d679b2ee0f7b6c60715) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:07.119740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33ae6b-ece1-42cd-8059-58044cebd108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:07.119740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33ae6b-ece1-42cd-8059-58044cebd108 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:07.123092 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 568/1146] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:09:06 2022] GET /identity/v3/role_assignments?user.id=2630ca8a93174d679b2ee0f7b6c60715&scope.project.id=7d85dc24ac444aa19d3c8b39c10541e4&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:09.338182 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a853d-5080-42b7-bcde-43c4ecbdc198 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:09.338717 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a853d-5080-42b7-bcde-43c4ecbdc198 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:09.338717 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a853d-5080-42b7-bcde-43c4ecbdc198 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:09.340018 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 579/1147] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:09.345760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea916b43-7c3d-465b-8372-c2dc1a65b611 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:09.346206 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea916b43-7c3d-465b-8372-c2dc1a65b611 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:09.346302 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea916b43-7c3d-465b-8372-c2dc1a65b611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:09.366421 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-ea916b43-7c3d-465b-8372-c2dc1a65b611 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:09.366421 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-ea916b43-7c3d-465b-8372-c2dc1a65b611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:09.432460 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 569/1148] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:09.454748 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a853d-5080-42b7-bcde-43c4ecbdc198 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:09.470392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:09.557897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a853d-5080-42b7-bcde-43c4ecbdc198 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:09.592766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d71dcb-b8c3-4b72-93e4-eb405c7241b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:09.593439 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d71dcb-b8c3-4b72-93e4-eb405c7241b2 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:09.593718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d71dcb-b8c3-4b72-93e4-eb405c7241b2 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:09.593994 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d71dcb-b8c3-4b72-93e4-eb405c7241b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:09.604561 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d71dcb-b8c3-4b72-93e4-eb405c7241b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-91d71dcb-b8c3-4b72-93e4-eb405c7241b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:09:09.612266 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:09.613460 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:09:09.619314 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 580/1149] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:09:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:09:09.620216 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea916b43-7c3d-465b-8372-c2dc1a65b611 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:09.718014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea916b43-7c3d-465b-8372-c2dc1a65b611 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:09.727247 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:09.752521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43e63c-f61c-4246-8372-d77f19374518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:09.753430 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43e63c-f61c-4246-8372-d77f19374518 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:09.753799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43e63c-f61c-4246-8372-d77f19374518 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:09.754245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43e63c-f61c-4246-8372-d77f19374518 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:09.755264 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43e63c-f61c-4246-8372-d77f19374518 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:09.755572 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43e63c-f61c-4246-8372-d77f19374518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:09.756978 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43e63c-f61c-4246-8372-d77f19374518 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:09.765482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 570/1150] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:09:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:09.771539 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d71dcb-b8c3-4b72-93e4-eb405c7241b2 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:09.842316 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:09.871392 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d71dcb-b8c3-4b72-93e4-eb405c7241b2 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:09.905386 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c19ef6-2ebf-4083-83f3-32f246fcd5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:09.906247 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c19ef6-2ebf-4083-83f3-32f246fcd5dc None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:09.906531 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c19ef6-2ebf-4083-83f3-32f246fcd5dc None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:09.906808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c19ef6-2ebf-4083-83f3-32f246fcd5dc None admin] PATH_INFO: `/v3/users/2630ca8a93174d679b2ee0f7b6c60715` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:09.908021 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c19ef6-2ebf-4083-83f3-32f246fcd5dc None admin] RBAC: Authorizing `identity:get_user(user_id=2630ca8a93174d679b2ee0f7b6c60715)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:09.909497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c19ef6-2ebf-4083-83f3-32f246fcd5dc None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:09.911121 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 581/1151] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:09:09 2022] GET /identity/v3/users/2630ca8a93174d679b2ee0f7b6c60715 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:09.917151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43e63c-f61c-4246-8372-d77f19374518 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:10.017414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43e63c-f61c-4246-8372-d77f19374518 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:10.050896 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e54f9e-4150-48a2-aa3e-8b93a42bfbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:10.051761 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e54f9e-4150-48a2-aa3e-8b93a42bfbbb None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:10.052047 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e54f9e-4150-48a2-aa3e-8b93a42bfbbb None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:10.052328 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e54f9e-4150-48a2-aa3e-8b93a42bfbbb None admin] PATH_INFO: `/v3/projects/7d85dc24ac444aa19d3c8b39c10541e4` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:10.053539 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e54f9e-4150-48a2-aa3e-8b93a42bfbbb None admin] RBAC: Authorizing `identity:get_project(project_id=7d85dc24ac444aa19d3c8b39c10541e4)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:10.054972 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e54f9e-4150-48a2-aa3e-8b93a42bfbbb None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:10.056715 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 571/1152] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:09:09 2022] GET /identity/v3/projects/7d85dc24ac444aa19d3c8b39c10541e4 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:10.063297 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c19ef6-2ebf-4083-83f3-32f246fcd5dc None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:10.167119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c19ef6-2ebf-4083-83f3-32f246fcd5dc None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:10.200316 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8067e91-e656-44d4-bd10-8f3ed9ff1013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:10.201208 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8067e91-e656-44d4-bd10-8f3ed9ff1013 None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:10.201488 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8067e91-e656-44d4-bd10-8f3ed9ff1013 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:10.201762 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8067e91-e656-44d4-bd10-8f3ed9ff1013 None admin] PATH_INFO: `/v3/projects/7d85dc24ac444aa19d3c8b39c10541e4/users/2630ca8a93174d679b2ee0f7b6c60715/roles/ab7a63cec3a744848972fcbb89ecd0ca` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:10.203818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8067e91-e656-44d4-bd10-8f3ed9ff1013 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d85dc24ac444aa19d3c8b39c10541e4, user_id=2630ca8a93174d679b2ee0f7b6c60715, role_id=ab7a63cec3a744848972fcbb89ecd0ca)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:10.205294 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8067e91-e656-44d4-bd10-8f3ed9ff1013 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:10.286913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:10.291621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 582/1153] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:09:10 2022] PUT /identity/v3/projects/7d85dc24ac444aa19d3c8b39c10541e4/users/2630ca8a93174d679b2ee0f7b6c60715/roles/ab7a63cec3a744848972fcbb89ecd0ca => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:09:12.505812 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5553aba-4145-46d1-a382-a1c67a374acc None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:12.506335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5553aba-4145-46d1-a382-a1c67a374acc None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:12.506335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5553aba-4145-46d1-a382-a1c67a374acc None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:12.507666 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 572/1154] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:12.513476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200c7c1-ea3a-4792-bb6d-b12166fa00df None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:12.513899 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200c7c1-ea3a-4792-bb6d-b12166fa00df None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:12.514026 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200c7c1-ea3a-4792-bb6d-b12166fa00df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:12.532466 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-c200c7c1-ea3a-4792-bb6d-b12166fa00df None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:12.533646 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-c200c7c1-ea3a-4792-bb6d-b12166fa00df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:12.599067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 583/1155] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:12.622101 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5553aba-4145-46d1-a382-a1c67a374acc None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:12.663794 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:12.725525 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5553aba-4145-46d1-a382-a1c67a374acc None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:12.758048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6023091-5400-4f10-acad-5165af026b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:12.758764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6023091-5400-4f10-acad-5165af026b52 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:12.759048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6023091-5400-4f10-acad-5165af026b52 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:12.759323 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6023091-5400-4f10-acad-5165af026b52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:12.770190 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6023091-5400-4f10-acad-5165af026b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-e6023091-5400-4f10-acad-5165af026b52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:09:12.777735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:12.779335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:09:12.780134 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 573/1156] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:09:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:09:12.785642 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200c7c1-ea3a-4792-bb6d-b12166fa00df None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:12.886246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200c7c1-ea3a-4792-bb6d-b12166fa00df None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:12.915585 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:12.922017 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8915300-b4b7-4e8b-80ac-524758b2b884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:12.922683 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8915300-b4b7-4e8b-80ac-524758b2b884 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:12.923219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8915300-b4b7-4e8b-80ac-524758b2b884 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:12.923501 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8915300-b4b7-4e8b-80ac-524758b2b884 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:12.924244 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8915300-b4b7-4e8b-80ac-524758b2b884 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:12.924588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8915300-b4b7-4e8b-80ac-524758b2b884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:12.925975 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8915300-b4b7-4e8b-80ac-524758b2b884 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:12.934671 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 584/1157] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:09:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:12.939782 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6023091-5400-4f10-acad-5165af026b52 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:13.036739 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:13.040573 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6023091-5400-4f10-acad-5165af026b52 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:13.074783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702529f-99e8-405a-bfce-c39ececac8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:13.075692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702529f-99e8-405a-bfce-c39ececac8d4 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:13.075979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702529f-99e8-405a-bfce-c39ececac8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:13.076258 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702529f-99e8-405a-bfce-c39ececac8d4 None admin] PATH_INFO: `/v3/users/2630ca8a93174d679b2ee0f7b6c60715` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:13.077526 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702529f-99e8-405a-bfce-c39ececac8d4 None admin] RBAC: Authorizing `identity:get_user(user_id=2630ca8a93174d679b2ee0f7b6c60715)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:13.078978 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702529f-99e8-405a-bfce-c39ececac8d4 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:13.080680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 574/1158] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:09:12 2022] GET /identity/v3/users/2630ca8a93174d679b2ee0f7b6c60715 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:13.086404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8915300-b4b7-4e8b-80ac-524758b2b884 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:13.186196 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8915300-b4b7-4e8b-80ac-524758b2b884 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:13.220312 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad1a30-37d5-4400-992f-8c0dc581e69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:13.221355 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad1a30-37d5-4400-992f-8c0dc581e69a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:13.221499 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad1a30-37d5-4400-992f-8c0dc581e69a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:13.221948 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad1a30-37d5-4400-992f-8c0dc581e69a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:13.280914 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:13.290260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad1a30-37d5-4400-992f-8c0dc581e69a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:13.291710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad1a30-37d5-4400-992f-8c0dc581e69a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:13.388192 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 585/1159] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:09:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:09:13.394201 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702529f-99e8-405a-bfce-c39ececac8d4 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:13.506491 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702529f-99e8-405a-bfce-c39ececac8d4 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:13.540241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ecc60d-068d-49a7-918f-9b31e214da00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:13.541166 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecc60d-068d-49a7-918f-9b31e214da00 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:13.541393 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecc60d-068d-49a7-918f-9b31e214da00 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:13.541664 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ecc60d-068d-49a7-918f-9b31e214da00 None admin] PATH_INFO: `/v3/projects/7d85dc24ac444aa19d3c8b39c10541e4` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:13.542858 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecc60d-068d-49a7-918f-9b31e214da00 None admin] RBAC: Authorizing `identity:get_project(project_id=7d85dc24ac444aa19d3c8b39c10541e4)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:13.544655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ecc60d-068d-49a7-918f-9b31e214da00 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:13.546578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 575/1160] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:09:13 2022] GET /identity/v3/projects/7d85dc24ac444aa19d3c8b39c10541e4 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:13.552964 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad1a30-37d5-4400-992f-8c0dc581e69a None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:13.641465 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:13.649864 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad1a30-37d5-4400-992f-8c0dc581e69a None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:13.684118 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a926061-b8f7-4d55-9acd-7511e0c7f984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:13.685338 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a926061-b8f7-4d55-9acd-7511e0c7f984 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:13.685622 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a926061-b8f7-4d55-9acd-7511e0c7f984 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:13.685900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a926061-b8f7-4d55-9acd-7511e0c7f984 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:13.686720 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a926061-b8f7-4d55-9acd-7511e0c7f984 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=7d85dc24ac444aa19d3c8b39c10541e4, user.id=2630ca8a93174d679b2ee0f7b6c60715) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:13.687040 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a926061-b8f7-4d55-9acd-7511e0c7f984 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:13.688559 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a926061-b8f7-4d55-9acd-7511e0c7f984 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:13.694287 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 586/1161] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:09:13 2022] GET /identity/v3/role_assignments?user.id=2630ca8a93174d679b2ee0f7b6c60715&scope.project.id=7d85dc24ac444aa19d3c8b39c10541e4&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:15.940394 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d09a8-827c-4ae8-8bff-d0d7000ed831 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:15.940947 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d09a8-827c-4ae8-8bff-d0d7000ed831 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:15.940947 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d09a8-827c-4ae8-8bff-d0d7000ed831 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:15.942392 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 576/1162] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:15.947715 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ce363-157a-4bfb-b4cc-a9655045cd38 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:15.948124 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ce363-157a-4bfb-b4cc-a9655045cd38 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:15.948272 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ce363-157a-4bfb-b4cc-a9655045cd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:15.966310 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-e93ce363-157a-4bfb-b4cc-a9655045cd38 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:15.967488 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-e93ce363-157a-4bfb-b4cc-a9655045cd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:16.032761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 587/1163] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:16.055646 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d09a8-827c-4ae8-8bff-d0d7000ed831 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:16.103868 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:16.186935 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d09a8-827c-4ae8-8bff-d0d7000ed831 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:16.229432 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667be6c8-2cf9-4cce-b7e2-e293b09e996c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:16.230003 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667be6c8-2cf9-4cce-b7e2-e293b09e996c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:16.230278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667be6c8-2cf9-4cce-b7e2-e293b09e996c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:16.230556 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667be6c8-2cf9-4cce-b7e2-e293b09e996c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:16.298422 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667be6c8-2cf9-4cce-b7e2-e293b09e996c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:16.299852 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667be6c8-2cf9-4cce-b7e2-e293b09e996c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:16.331799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:16.397177 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 577/1164] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:09:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:09:16.403333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ce363-157a-4bfb-b4cc-a9655045cd38 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:16.489357 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:16.504481 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ce363-157a-4bfb-b4cc-a9655045cd38 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:16.543454 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05859a-d021-441f-95eb-51cbcb8c42a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:16.544840 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05859a-d021-441f-95eb-51cbcb8c42a7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:16.545129 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05859a-d021-441f-95eb-51cbcb8c42a7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:16.545391 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05859a-d021-441f-95eb-51cbcb8c42a7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:16.552360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05859a-d021-441f-95eb-51cbcb8c42a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:16.553840 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae05859a-d021-441f-95eb-51cbcb8c42a7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-ae05859a-d021-441f-95eb-51cbcb8c42a7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:16.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:16.570353 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:09:16.571283 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 588/1165] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:09:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:09:16.576589 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667be6c8-2cf9-4cce-b7e2-e293b09e996c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:16.676631 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667be6c8-2cf9-4cce-b7e2-e293b09e996c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:16.711955 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0232f4-65f6-4e5e-b974-3f08ae719a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:16.712652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0232f4-65f6-4e5e-b974-3f08ae719a86 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:16.712941 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0232f4-65f6-4e5e-b974-3f08ae719a86 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:16.713223 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0232f4-65f6-4e5e-b974-3f08ae719a86 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:16.713955 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0232f4-65f6-4e5e-b974-3f08ae719a86 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:16.714272 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0232f4-65f6-4e5e-b974-3f08ae719a86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:16.716168 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0232f4-65f6-4e5e-b974-3f08ae719a86 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:16.717680 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:16.723982 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 578/1166] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:09:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:18.936791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990a55c-a266-4cf7-9241-d766d2ea09e9 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:18.937321 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990a55c-a266-4cf7-9241-d766d2ea09e9 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:18.937425 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990a55c-a266-4cf7-9241-d766d2ea09e9 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:18.938730 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 589/1167] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:18.944122 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb37fb3-f0a1-42cf-97f4-5cec5ee6170a None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:18.944611 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb37fb3-f0a1-42cf-97f4-5cec5ee6170a None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:18.944707 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb37fb3-f0a1-42cf-97f4-5cec5ee6170a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:18.961244 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-bcb37fb3-f0a1-42cf-97f4-5cec5ee6170a None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:18.962410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-bcb37fb3-f0a1-42cf-97f4-5cec5ee6170a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:19.028693 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 579/1168] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:19.052130 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990a55c-a266-4cf7-9241-d766d2ea09e9 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:19.155398 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990a55c-a266-4cf7-9241-d766d2ea09e9 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:19.189731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51169c99-c30d-4983-bdbb-3256521d33c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:19.190334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51169c99-c30d-4983-bdbb-3256521d33c3 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:19.190619 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51169c99-c30d-4983-bdbb-3256521d33c3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:19.190897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51169c99-c30d-4983-bdbb-3256521d33c3 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:19.191634 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51169c99-c30d-4983-bdbb-3256521d33c3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:19.193133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51169c99-c30d-4983-bdbb-3256521d33c3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:19.304527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 590/1169] 10.222.0.88 () {60 vars in 1138 bytes} [Thu May 19 20:09:19 2022] POST /identity/v3/domains => generated 267 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:09:21.566569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df649112-c238-405a-9296-2ba8c4d93a55 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:21.567125 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df649112-c238-405a-9296-2ba8c4d93a55 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:21.567125 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df649112-c238-405a-9296-2ba8c4d93a55 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:21.568531 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 580/1170] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:09:21.573913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e66025-d445-4d1e-9500-0daedd712a36 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:21.574343 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e66025-d445-4d1e-9500-0daedd712a36 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:21.574473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e66025-d445-4d1e-9500-0daedd712a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:21.586030 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:21.591575 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-99e66025-d445-4d1e-9500-0daedd712a36 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:21.592786 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-99e66025-d445-4d1e-9500-0daedd712a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:21.658902 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 591/1171] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:21.682948 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df649112-c238-405a-9296-2ba8c4d93a55 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:21.783217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df649112-c238-405a-9296-2ba8c4d93a55 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:21.810983 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:21.819785 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8027e1-bfd0-416b-b6d8-9777ca0d3d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:21.819785 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8027e1-bfd0-416b-b6d8-9777ca0d3d67 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:21.820028 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8027e1-bfd0-416b-b6d8-9777ca0d3d67 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:21.820150 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8027e1-bfd0-416b-b6d8-9777ca0d3d67 None admin] PATH_INFO: `/v3/domains/6176ecb2a041431a9813fbd00b62eee7` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:21.830540 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8027e1-bfd0-416b-b6d8-9777ca0d3d67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6176ecb2a041431a9813fbd00b62eee7)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:21.831986 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8027e1-bfd0-416b-b6d8-9777ca0d3d67 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:21.833785 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 581/1172] 10.222.0.88 () {56 vars in 1183 bytes} [Thu May 19 20:09:21 2022] GET /identity/v3/domains/6176ecb2a041431a9813fbd00b62eee7 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:21.839700 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e66025-d445-4d1e-9500-0daedd712a36 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:21.938911 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e66025-d445-4d1e-9500-0daedd712a36 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:21.972509 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0ec366-4fb2-44ab-910d-4932c35bfc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:21.973161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ec366-4fb2-44ab-910d-4932c35bfc02 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:21.973449 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ec366-4fb2-44ab-910d-4932c35bfc02 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:21.973729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ec366-4fb2-44ab-910d-4932c35bfc02 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:21.974447 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ec366-4fb2-44ab-910d-4932c35bfc02 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:21.975897 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0ec366-4fb2-44ab-910d-4932c35bfc02 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:21.979929 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:22.132324 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 592/1173] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:09:21 2022] POST /identity/v3/projects => generated 361 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:09:24.379365 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed946b8-4d44-4657-856a-75457639d8fe None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:24.379877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed946b8-4d44-4657-856a-75457639d8fe None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:24.379877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed946b8-4d44-4657-856a-75457639d8fe None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:24.381259 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 582/1174] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:24.386496 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c815-bf3a-4238-ae32-01b3a8039d31 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:24.386904 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c815-bf3a-4238-ae32-01b3a8039d31 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:24.387049 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940c815-bf3a-4238-ae32-01b3a8039d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:24.404281 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-f940c815-bf3a-4238-ae32-01b3a8039d31 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:24.405484 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-f940c815-bf3a-4238-ae32-01b3a8039d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:24.470678 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 593/1175] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:24.495579 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed946b8-4d44-4657-856a-75457639d8fe None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:24.597497 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed946b8-4d44-4657-856a-75457639d8fe None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:24.633284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a3732-92d1-4b5d-8dbd-d3c6926ca016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:24.634059 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a3732-92d1-4b5d-8dbd-d3c6926ca016 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:24.634340 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a3732-92d1-4b5d-8dbd-d3c6926ca016 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:24.634616 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a3732-92d1-4b5d-8dbd-d3c6926ca016 None admin] PATH_INFO: `/v3/domains/6176ecb2a041431a9813fbd00b62eee7` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:24.635808 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a3732-92d1-4b5d-8dbd-d3c6926ca016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6176ecb2a041431a9813fbd00b62eee7)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:24.637316 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a3732-92d1-4b5d-8dbd-d3c6926ca016 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:24.639485 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 583/1176] 10.222.0.88 () {56 vars in 1183 bytes} [Thu May 19 20:09:24 2022] GET /identity/v3/domains/6176ecb2a041431a9813fbd00b62eee7 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:24.645334 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940c815-bf3a-4238-ae32-01b3a8039d31 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:24.743301 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940c815-bf3a-4238-ae32-01b3a8039d31 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:24.760870 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:24.777101 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d181158-992a-45c5-81dd-8c8242d021bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:24.787913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d181158-992a-45c5-81dd-8c8242d021bd None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:24.787913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d181158-992a-45c5-81dd-8c8242d021bd None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:24.787913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d181158-992a-45c5-81dd-8c8242d021bd None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:24.787913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d181158-992a-45c5-81dd-8c8242d021bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:24.787913 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d181158-992a-45c5-81dd-8c8242d021bd None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:24.965931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 594/1177] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:09:24 2022] POST /identity/v3/users => generated 311 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:09:27.247827 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071be9d-6d05-4a7e-bdae-25ea29212526 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:27.248401 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071be9d-6d05-4a7e-bdae-25ea29212526 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:27.248508 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071be9d-6d05-4a7e-bdae-25ea29212526 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:27.249819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 584/1178] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:27.255011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded37b20-e7fc-447e-9330-67edb13a84ed None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:27.255450 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded37b20-e7fc-447e-9330-67edb13a84ed None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:27.255575 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded37b20-e7fc-447e-9330-67edb13a84ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:27.272619 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-ded37b20-e7fc-447e-9330-67edb13a84ed None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:27.273804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-ded37b20-e7fc-447e-9330-67edb13a84ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:27.341770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 595/1179] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:27.365454 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071be9d-6d05-4a7e-bdae-25ea29212526 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:27.423571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:27.467613 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071be9d-6d05-4a7e-bdae-25ea29212526 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:27.500814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851323a0-14ef-4962-9b13-90edf9ae8d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:27.501604 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851323a0-14ef-4962-9b13-90edf9ae8d42 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:27.501884 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851323a0-14ef-4962-9b13-90edf9ae8d42 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:27.502159 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851323a0-14ef-4962-9b13-90edf9ae8d42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:27.512175 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851323a0-14ef-4962-9b13-90edf9ae8d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-851323a0-14ef-4962-9b13-90edf9ae8d42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:09:27.520213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:27.521284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:09:27.522282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 585/1180] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:09:27.527487 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded37b20-e7fc-447e-9330-67edb13a84ed None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:27.635792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded37b20-e7fc-447e-9330-67edb13a84ed None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:27.638061 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:27.670011 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7b0c8-01a6-4f8f-8913-a5f47d49596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:27.670669 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7b0c8-01a6-4f8f-8913-a5f47d49596b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:27.670951 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7b0c8-01a6-4f8f-8913-a5f47d49596b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:27.671227 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7b0c8-01a6-4f8f-8913-a5f47d49596b None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:27.671961 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7b0c8-01a6-4f8f-8913-a5f47d49596b None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:27.672271 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7b0c8-01a6-4f8f-8913-a5f47d49596b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:27.673969 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7b0c8-01a6-4f8f-8913-a5f47d49596b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:27.682333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 596/1181] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:09:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:27.688949 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851323a0-14ef-4962-9b13-90edf9ae8d42 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:27.786557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851323a0-14ef-4962-9b13-90edf9ae8d42 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:27.803339 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:27.821833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7359a-13e2-4186-91a1-5462d883ae3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:27.824599 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7359a-13e2-4186-91a1-5462d883ae3f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:27.824599 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7359a-13e2-4186-91a1-5462d883ae3f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:27.824599 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7359a-13e2-4186-91a1-5462d883ae3f None admin] PATH_INFO: `/v3/users/a8cb301f81404a119648858909206beb` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:27.837119 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7359a-13e2-4186-91a1-5462d883ae3f None admin] RBAC: Authorizing `identity:get_user(user_id=a8cb301f81404a119648858909206beb)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:27.838575 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7359a-13e2-4186-91a1-5462d883ae3f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:27.840235 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 586/1182] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:09:27 2022] GET /identity/v3/users/a8cb301f81404a119648858909206beb => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:27.847006 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7b0c8-01a6-4f8f-8913-a5f47d49596b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:27.947915 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7b0c8-01a6-4f8f-8913-a5f47d49596b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:27.981176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272ef3b-ef6b-4b8c-b0fc-eb7eadd121cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:27.981823 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272ef3b-ef6b-4b8c-b0fc-eb7eadd121cb None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:27.982107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272ef3b-ef6b-4b8c-b0fc-eb7eadd121cb None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:27.982386 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272ef3b-ef6b-4b8c-b0fc-eb7eadd121cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:28.015994 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:28.048778 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272ef3b-ef6b-4b8c-b0fc-eb7eadd121cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:28.050338 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272ef3b-ef6b-4b8c-b0fc-eb7eadd121cb None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:28.150107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 597/1183] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:09:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:09:28.156494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7359a-13e2-4186-91a1-5462d883ae3f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:28.255448 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7359a-13e2-4186-91a1-5462d883ae3f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:28.289423 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647e328-da8c-490d-88c7-6944d6d3e0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:28.290275 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647e328-da8c-490d-88c7-6944d6d3e0a1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:28.290562 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647e328-da8c-490d-88c7-6944d6d3e0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:28.290837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647e328-da8c-490d-88c7-6944d6d3e0a1 None admin] PATH_INFO: `/v3/projects/f921e89a5ae74e5d9a643d4163e72612` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:28.292014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647e328-da8c-490d-88c7-6944d6d3e0a1 None admin] RBAC: Authorizing `identity:get_project(project_id=f921e89a5ae74e5d9a643d4163e72612)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:28.293507 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4647e328-da8c-490d-88c7-6944d6d3e0a1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:28.295210 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 587/1184] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:09:28 2022] GET /identity/v3/projects/f921e89a5ae74e5d9a643d4163e72612 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:28.301675 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272ef3b-ef6b-4b8c-b0fc-eb7eadd121cb None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:28.366068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:28.399679 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272ef3b-ef6b-4b8c-b0fc-eb7eadd121cb None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:28.434175 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0369c-1910-4f45-a5df-e9bad4692d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:28.434819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0369c-1910-4f45-a5df-e9bad4692d3d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:28.435101 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0369c-1910-4f45-a5df-e9bad4692d3d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:28.435377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0369c-1910-4f45-a5df-e9bad4692d3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:28.436193 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0369c-1910-4f45-a5df-e9bad4692d3d None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=f921e89a5ae74e5d9a643d4163e72612, user.id=a8cb301f81404a119648858909206beb) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:28.436608 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0369c-1910-4f45-a5df-e9bad4692d3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:28.438059 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0369c-1910-4f45-a5df-e9bad4692d3d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:28.443452 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 598/1185] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:09:28 2022] GET /identity/v3/role_assignments?user.id=a8cb301f81404a119648858909206beb&scope.project.id=f921e89a5ae74e5d9a643d4163e72612&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:30.624218 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125239d-668a-442e-8733-9510725f310a None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:30.624803 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125239d-668a-442e-8733-9510725f310a None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:30.624803 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125239d-668a-442e-8733-9510725f310a None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:30.626183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 588/1186] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:30.631513 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c1519-f0d4-48f9-bfab-64d26c0c63e4 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:30.631936 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c1519-f0d4-48f9-bfab-64d26c0c63e4 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:30.632072 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c1519-f0d4-48f9-bfab-64d26c0c63e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:30.649393 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-458c1519-f0d4-48f9-bfab-64d26c0c63e4 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:30.650575 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-458c1519-f0d4-48f9-bfab-64d26c0c63e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:30.717529 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 599/1187] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:30.739336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125239d-668a-442e-8733-9510725f310a None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:30.783927 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:30.841329 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125239d-668a-442e-8733-9510725f310a None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:30.874548 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69b0b3-6ce8-4056-99b9-f15b39d4e3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:30.875273 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69b0b3-6ce8-4056-99b9-f15b39d4e3fb None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:30.875564 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69b0b3-6ce8-4056-99b9-f15b39d4e3fb None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:30.875836 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69b0b3-6ce8-4056-99b9-f15b39d4e3fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:30.885918 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69b0b3-6ce8-4056-99b9-f15b39d4e3fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-cd69b0b3-6ce8-4056-99b9-f15b39d4e3fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:09:30.894080 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:30.895133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:09:30.896196 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 589/1188] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:09:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:09:30.901440 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c1519-f0d4-48f9-bfab-64d26c0c63e4 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:30.959993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:31.000025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c1519-f0d4-48f9-bfab-64d26c0c63e4 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:31.033657 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae06121-c9d6-488a-a6ef-4011747d099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:31.034324 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae06121-c9d6-488a-a6ef-4011747d099b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:31.034611 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae06121-c9d6-488a-a6ef-4011747d099b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:31.034892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae06121-c9d6-488a-a6ef-4011747d099b None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:31.035636 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae06121-c9d6-488a-a6ef-4011747d099b None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:31.035947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae06121-c9d6-488a-a6ef-4011747d099b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:31.037364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae06121-c9d6-488a-a6ef-4011747d099b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:31.045463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 600/1189] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:09:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:09:31.051804 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69b0b3-6ce8-4056-99b9-f15b39d4e3fb None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:31.152775 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69b0b3-6ce8-4056-99b9-f15b39d4e3fb None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:31.154783 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:31.187150 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e955753-a194-48f3-99f3-703ceba24db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:31.188253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e955753-a194-48f3-99f3-703ceba24db3 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:31.188784 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e955753-a194-48f3-99f3-703ceba24db3 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:31.189359 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e955753-a194-48f3-99f3-703ceba24db3 None admin] PATH_INFO: `/v3/users/a8cb301f81404a119648858909206beb` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:31.190584 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e955753-a194-48f3-99f3-703ceba24db3 None admin] RBAC: Authorizing `identity:get_user(user_id=a8cb301f81404a119648858909206beb)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:31.192017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e955753-a194-48f3-99f3-703ceba24db3 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:31.193803 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 590/1190] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:09:31 2022] GET /identity/v3/users/a8cb301f81404a119648858909206beb => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:31.200286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae06121-c9d6-488a-a6ef-4011747d099b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:31.301431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae06121-c9d6-488a-a6ef-4011747d099b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:31.323844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:31.335105 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47542a57-a9a2-4b19-9167-75e9dbb01b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:31.343977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47542a57-a9a2-4b19-9167-75e9dbb01b1c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:31.343977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47542a57-a9a2-4b19-9167-75e9dbb01b1c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:31.343977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47542a57-a9a2-4b19-9167-75e9dbb01b1c None admin] PATH_INFO: `/v3/projects/f921e89a5ae74e5d9a643d4163e72612` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:31.343977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47542a57-a9a2-4b19-9167-75e9dbb01b1c None admin] RBAC: Authorizing `identity:get_project(project_id=f921e89a5ae74e5d9a643d4163e72612)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:31.343977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47542a57-a9a2-4b19-9167-75e9dbb01b1c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:31.343977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 601/1191] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:09:31 2022] GET /identity/v3/projects/f921e89a5ae74e5d9a643d4163e72612 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:31.346844 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e955753-a194-48f3-99f3-703ceba24db3 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:31.447868 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e955753-a194-48f3-99f3-703ceba24db3 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:31.480470 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafed03-3768-43a5-ae8d-2c9b94c8f14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:31.481879 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafed03-3768-43a5-ae8d-2c9b94c8f14a None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:31.482161 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafed03-3768-43a5-ae8d-2c9b94c8f14a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:31.482435 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafed03-3768-43a5-ae8d-2c9b94c8f14a None admin] PATH_INFO: `/v3/projects/f921e89a5ae74e5d9a643d4163e72612/users/a8cb301f81404a119648858909206beb/roles/ab7a63cec3a744848972fcbb89ecd0ca` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:31.484410 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafed03-3768-43a5-ae8d-2c9b94c8f14a None admin] RBAC: Authorizing `identity:create_grant(project_id=f921e89a5ae74e5d9a643d4163e72612, user_id=a8cb301f81404a119648858909206beb, role_id=ab7a63cec3a744848972fcbb89ecd0ca)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:31.485843 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafed03-3768-43a5-ae8d-2c9b94c8f14a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:31.600477 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 591/1192] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:09:31 2022] PUT /identity/v3/projects/f921e89a5ae74e5d9a643d4163e72612/users/a8cb301f81404a119648858909206beb/roles/ab7a63cec3a744848972fcbb89ecd0ca => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:09:33.847040 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4401158-6877-4899-82b1-692244bd423f None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:33.847602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4401158-6877-4899-82b1-692244bd423f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:33.847602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4401158-6877-4899-82b1-692244bd423f None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:33.848994 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 602/1193] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:09:33.854287 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877835b9-42ce-4668-839d-38d535c2fe38 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:33.854679 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877835b9-42ce-4668-839d-38d535c2fe38 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:33.854845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877835b9-42ce-4668-839d-38d535c2fe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:33.872563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-877835b9-42ce-4668-839d-38d535c2fe38 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:09:33.873741 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-877835b9-42ce-4668-839d-38d535c2fe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:09:33.896648 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:33.939400 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 592/1194] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:09:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:09:33.962592 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4401158-6877-4899-82b1-692244bd423f None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:34.066415 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4401158-6877-4899-82b1-692244bd423f None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:34.100122 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc2fa-e535-40b3-aa0a-1cec43189938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:34.100910 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc2fa-e535-40b3-aa0a-1cec43189938 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:34.101247 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc2fa-e535-40b3-aa0a-1cec43189938 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:34.101531 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ddc2fa-e535-40b3-aa0a-1cec43189938 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:34.112728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ddc2fa-e535-40b3-aa0a-1cec43189938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-50ddc2fa-e535-40b3-aa0a-1cec43189938 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:09:34.120445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:09:34.122111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:09:34.123076 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 603/1195] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:09:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:09:34.128277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877835b9-42ce-4668-839d-38d535c2fe38 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:34.227271 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877835b9-42ce-4668-839d-38d535c2fe38 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:34.261332 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf547e-41f4-4699-ad73-76c09ea04327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:34.262038 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf547e-41f4-4699-ad73-76c09ea04327 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:34.262895 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf547e-41f4-4699-ad73-76c09ea04327 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:34.263173 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf547e-41f4-4699-ad73-76c09ea04327 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:34.263909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf547e-41f4-4699-ad73-76c09ea04327 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:34.264214 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf547e-41f4-4699-ad73-76c09ea04327 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:34.265610 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf547e-41f4-4699-ad73-76c09ea04327 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:34.273570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 593/1196] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:09:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:34.279480 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc2fa-e535-40b3-aa0a-1cec43189938 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:34.329662 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:34.379918 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ddc2fa-e535-40b3-aa0a-1cec43189938 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:34.412693 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41e130-2883-44ab-bd2f-b99ee637b02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:34.413551 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41e130-2883-44ab-bd2f-b99ee637b02c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:34.413830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41e130-2883-44ab-bd2f-b99ee637b02c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:34.414104 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41e130-2883-44ab-bd2f-b99ee637b02c None admin] PATH_INFO: `/v3/users/a8cb301f81404a119648858909206beb` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:34.415324 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41e130-2883-44ab-bd2f-b99ee637b02c None admin] RBAC: Authorizing `identity:get_user(user_id=a8cb301f81404a119648858909206beb)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:34.416794 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41e130-2883-44ab-bd2f-b99ee637b02c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:34.418972 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 604/1197] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:09:34 2022] GET /identity/v3/users/a8cb301f81404a119648858909206beb => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:34.425006 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf547e-41f4-4699-ad73-76c09ea04327 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:34.451003 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:34.526673 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf547e-41f4-4699-ad73-76c09ea04327 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:34.561735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a749-ca64-405e-a400-6910b8620d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:34.562370 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a749-ca64-405e-a400-6910b8620d2a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:34.562653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a749-ca64-405e-a400-6910b8620d2a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:34.562926 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a749-ca64-405e-a400-6910b8620d2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:34.628607 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927a749-ca64-405e-a400-6910b8620d2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:34.630092 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927a749-ca64-405e-a400-6910b8620d2a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:34.652564 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:34.729355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 594/1198] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:09:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:09:34.737699 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41e130-2883-44ab-bd2f-b99ee637b02c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:34.841038 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41e130-2883-44ab-bd2f-b99ee637b02c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:34.858142 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:34.875281 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd43033-f4b7-409c-aacc-063e865147f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:34.876133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd43033-f4b7-409c-aacc-063e865147f7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:34.876458 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd43033-f4b7-409c-aacc-063e865147f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:34.876786 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd43033-f4b7-409c-aacc-063e865147f7 None admin] PATH_INFO: `/v3/projects/f921e89a5ae74e5d9a643d4163e72612` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:34.877973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd43033-f4b7-409c-aacc-063e865147f7 None admin] RBAC: Authorizing `identity:get_project(project_id=f921e89a5ae74e5d9a643d4163e72612)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:34.879619 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd43033-f4b7-409c-aacc-063e865147f7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:34.881785 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 605/1199] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:09:34 2022] GET /identity/v3/projects/f921e89a5ae74e5d9a643d4163e72612 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:34.888606 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a749-ca64-405e-a400-6910b8620d2a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:09:34.987819 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a749-ca64-405e-a400-6910b8620d2a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:09:35.011139 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:35.023148 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9edaad-768c-430c-8448-7fe8afdc7938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:09:35.036692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9edaad-768c-430c-8448-7fe8afdc7938 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:09:35.036692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9edaad-768c-430c-8448-7fe8afdc7938 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:09:35.036692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9edaad-768c-430c-8448-7fe8afdc7938 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:09:35.036692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9edaad-768c-430c-8448-7fe8afdc7938 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=f921e89a5ae74e5d9a643d4163e72612, user.id=a8cb301f81404a119648858909206beb) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:09:35.036692 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9edaad-768c-430c-8448-7fe8afdc7938 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:09:35.037147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9edaad-768c-430c-8448-7fe8afdc7938 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:09:35.037147 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 595/1200] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:09:34 2022] GET /identity/v3/role_assignments?user.id=a8cb301f81404a119648858909206beb&scope.project.id=f921e89a5ae74e5d9a643d4163e72612&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:09:44.882389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=7 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:09:45.056756 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=7 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:56.964856 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f6e20-cc1b-4b64-b927-b1dfddfd93dd None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:56.965501 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f6e20-cc1b-4b64-b927-b1dfddfd93dd None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:56.965501 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f6e20-cc1b-4b64-b927-b1dfddfd93dd None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:56.966781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 606/1201] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:10:56.973056 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9079b0-5f63-4d54-b73c-41f69015292e None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:56.977181 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9079b0-5f63-4d54-b73c-41f69015292e None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:56.977181 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9079b0-5f63-4d54-b73c-41f69015292e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:56.991908 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-9d9079b0-5f63-4d54-b73c-41f69015292e None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:56.993143 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-9d9079b0-5f63-4d54-b73c-41f69015292e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:10:57.068379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 596/1202] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:10:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:10:57.091885 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f6e20-cc1b-4b64-b927-b1dfddfd93dd None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:57.197514 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f6e20-cc1b-4b64-b927-b1dfddfd93dd None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:57.232711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff0815-396a-4c6e-86d4-d5dee49ab506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:57.233472 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff0815-396a-4c6e-86d4-d5dee49ab506 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:57.233762 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff0815-396a-4c6e-86d4-d5dee49ab506 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:57.234036 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff0815-396a-4c6e-86d4-d5dee49ab506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:57.244484 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff0815-396a-4c6e-86d4-d5dee49ab506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:57.245936 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff0815-396a-4c6e-86d4-d5dee49ab506 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-56ff0815-396a-4c6e-86d4-d5dee49ab506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:10:57.263064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:10:57.264068 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:10:57.265219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:10:57.265219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:10:57.265219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:10:57.265219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:10:57.265219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 607/1203] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:10:57.270944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9079b0-5f63-4d54-b73c-41f69015292e None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:57.371780 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9079b0-5f63-4d54-b73c-41f69015292e None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:57.405217 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:57.411510 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e393652-74e8-4c13-9514-e39b9bb7127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:57.416277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e393652-74e8-4c13-9514-e39b9bb7127e None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:57.416277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e393652-74e8-4c13-9514-e39b9bb7127e None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:57.416277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e393652-74e8-4c13-9514-e39b9bb7127e None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:57.416277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e393652-74e8-4c13-9514-e39b9bb7127e None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:10:57.416277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e393652-74e8-4c13-9514-e39b9bb7127e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:57.416277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e393652-74e8-4c13-9514-e39b9bb7127e None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:57.430070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 597/1204] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:10:57.436023 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff0815-396a-4c6e-86d4-d5dee49ab506 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:10:57.496766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:10:57.538288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff0815-396a-4c6e-86d4-d5dee49ab506 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:10:57.573181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc9c59-1c24-421d-a43d-c3490f51e9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:10:57.573823 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc9c59-1c24-421d-a43d-c3490f51e9e2 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:57.574158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc9c59-1c24-421d-a43d-c3490f51e9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:57.574489 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc9c59-1c24-421d-a43d-c3490f51e9e2 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:57.575209 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc9c59-1c24-421d-a43d-c3490f51e9e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:10:57.576666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc9c59-1c24-421d-a43d-c3490f51e9e2 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:10:57.679892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 608/1205] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:10:57 2022] POST /identity/v3/users => generated 251 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:10:59.929850 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e27f7-c886-4781-af18-af8ec9109696 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.930447 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e27f7-c886-4781-af18-af8ec9109696 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.930447 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688e27f7-c886-4781-af18-af8ec9109696 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.931782 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 598/1206] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:10:59.937044 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f0d92-643a-4e22-9135-7fd4c49ee7df None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:10:59.937433 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f0d92-643a-4e22-9135-7fd4c49ee7df None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:10:59.937599 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f0d92-643a-4e22-9135-7fd4c49ee7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:10:59.954087 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-9e4f0d92-643a-4e22-9135-7fd4c49ee7df None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:10:59.955274 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-9e4f0d92-643a-4e22-9135-7fd4c49ee7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:00.021659 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 609/1207] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:10:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:00.045214 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e27f7-c886-4781-af18-af8ec9109696 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:00.151352 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688e27f7-c886-4781-af18-af8ec9109696 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.185989 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43cd40-bc73-44c6-864a-93f9af4ac180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.186757 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43cd40-bc73-44c6-864a-93f9af4ac180 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.187028 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43cd40-bc73-44c6-864a-93f9af4ac180 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.187308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43cd40-bc73-44c6-864a-93f9af4ac180 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.198818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43cd40-bc73-44c6-864a-93f9af4ac180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-ff43cd40-bc73-44c6-864a-93f9af4ac180 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:00.207085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:00.208221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:00.209341 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 599/1208] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:11:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:00.214424 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f0d92-643a-4e22-9135-7fd4c49ee7df None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:00.253861 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:00.312735 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f0d92-643a-4e22-9135-7fd4c49ee7df None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.347148 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41c457-229a-4ec2-bc5b-1d3f4a17264d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.347846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41c457-229a-4ec2-bc5b-1d3f4a17264d None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.348134 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41c457-229a-4ec2-bc5b-1d3f4a17264d None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.348456 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d41c457-229a-4ec2-bc5b-1d3f4a17264d None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.349232 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41c457-229a-4ec2-bc5b-1d3f4a17264d None admin] RBAC: Adding query filter params (name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:00.349544 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41c457-229a-4ec2-bc5b-1d3f4a17264d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:00.350993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d41c457-229a-4ec2-bc5b-1d3f4a17264d None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:00.359023 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 610/1209] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:11:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:00.364904 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43cd40-bc73-44c6-864a-93f9af4ac180 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:00.415747 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:00.464279 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43cd40-bc73-44c6-864a-93f9af4ac180 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.497703 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57649f48-f82d-4287-9919-cbc90a4f6f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.498482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57649f48-f82d-4287-9919-cbc90a4f6f7a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.498761 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57649f48-f82d-4287-9919-cbc90a4f6f7a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.499033 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57649f48-f82d-4287-9919-cbc90a4f6f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.509445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57649f48-f82d-4287-9919-cbc90a4f6f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:00.511449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57649f48-f82d-4287-9919-cbc90a4f6f7a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-57649f48-f82d-4287-9919-cbc90a4f6f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:00.530157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:00.531239 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:00.532297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:00.532297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:00.532297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:00.532297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:00.532297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 600/1210] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:00.537424 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41c457-229a-4ec2-bc5b-1d3f4a17264d None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:00.636412 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:00.639401 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d41c457-229a-4ec2-bc5b-1d3f4a17264d None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.672682 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39217cf5-a0b9-4e7b-853d-ba73105217c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.673438 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39217cf5-a0b9-4e7b-853d-ba73105217c9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.673706 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39217cf5-a0b9-4e7b-853d-ba73105217c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.673979 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39217cf5-a0b9-4e7b-853d-ba73105217c9 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.674951 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39217cf5-a0b9-4e7b-853d-ba73105217c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:00.675256 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39217cf5-a0b9-4e7b-853d-ba73105217c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:00.676729 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39217cf5-a0b9-4e7b-853d-ba73105217c9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:00.688951 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 611/1211] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:00.694793 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57649f48-f82d-4287-9919-cbc90a4f6f7a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:00.796648 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57649f48-f82d-4287-9919-cbc90a4f6f7a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.798685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:00.831588 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f115c-fe43-45db-b30e-fac16d1143de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.832439 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f115c-fe43-45db-b30e-fac16d1143de None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.832720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f115c-fe43-45db-b30e-fac16d1143de None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.832991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f115c-fe43-45db-b30e-fac16d1143de None admin] PATH_INFO: `/v3/users/placement` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.839930 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f115c-fe43-45db-b30e-fac16d1143de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:00.841406 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f115c-fe43-45db-b30e-fac16d1143de None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-504f115c-fe43-45db-b30e-fac16d1143de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:00.856278 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:00.857282 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:00.858367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:00.858367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:00.858367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:00.858367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:00.858367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:00.858367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:00.858367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:00.858367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:00.858367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:00.858367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:00.858367 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 601/1212] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:00.864081 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39217cf5-a0b9-4e7b-853d-ba73105217c9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:00.964268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39217cf5-a0b9-4e7b-853d-ba73105217c9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:00.997829 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8900e-7e10-41eb-b454-f843aa5164ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:00.998505 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8900e-7e10-41eb-b454-f843aa5164ec None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:00.998796 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8900e-7e10-41eb-b454-f843aa5164ec None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:00.999054 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8900e-7e10-41eb-b454-f843aa5164ec None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:00.999892 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8900e-7e10-41eb-b454-f843aa5164ec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:01.000187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8900e-7e10-41eb-b454-f843aa5164ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:01.002198 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8900e-7e10-41eb-b454-f843aa5164ec None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:01.019155 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 612/1213] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:11:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:01.024875 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f115c-fe43-45db-b30e-fac16d1143de None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:01.127796 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f115c-fe43-45db-b30e-fac16d1143de None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:01.162477 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da926659-08ac-4106-990c-1cce7b2fb86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:01.163114 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da926659-08ac-4106-990c-1cce7b2fb86c None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:01.163396 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da926659-08ac-4106-990c-1cce7b2fb86c None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:01.163674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da926659-08ac-4106-990c-1cce7b2fb86c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:01.193755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:01.231329 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da926659-08ac-4106-990c-1cce7b2fb86c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:01.232810 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da926659-08ac-4106-990c-1cce7b2fb86c None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:01.332047 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 602/1214] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:11:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:11:01.337942 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8900e-7e10-41eb-b454-f843aa5164ec None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:01.356547 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:01.434549 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8900e-7e10-41eb-b454-f843aa5164ec None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:01.469289 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29af31-8e70-48b0-b654-8ed6d3c8ea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:01.470231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29af31-8e70-48b0-b654-8ed6d3c8ea83 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:01.470541 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29af31-8e70-48b0-b654-8ed6d3c8ea83 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:01.470816 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29af31-8e70-48b0-b654-8ed6d3c8ea83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:01.480998 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29af31-8e70-48b0-b654-8ed6d3c8ea83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:01.482482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b29af31-8e70-48b0-b654-8ed6d3c8ea83 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-1b29af31-8e70-48b0-b654-8ed6d3c8ea83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:01.500119 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:01.501593 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:01.502732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:01.502732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:01.502732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:01.502732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:01.502732 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 613/1215] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:01.507711 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da926659-08ac-4106-990c-1cce7b2fb86c None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:01.569500 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:01.606261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da926659-08ac-4106-990c-1cce7b2fb86c None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:01.639788 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05831b47-5a27-426c-8ce7-c130a1221af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:01.640473 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05831b47-5a27-426c-8ce7-c130a1221af8 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:01.640784 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05831b47-5a27-426c-8ce7-c130a1221af8 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:01.641060 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05831b47-5a27-426c-8ce7-c130a1221af8 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:01.641776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05831b47-5a27-426c-8ce7-c130a1221af8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:01.642083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05831b47-5a27-426c-8ce7-c130a1221af8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:01.643551 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05831b47-5a27-426c-8ce7-c130a1221af8 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:01.655493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 603/1216] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:01.661212 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29af31-8e70-48b0-b654-8ed6d3c8ea83 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:01.730725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:01.761860 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29af31-8e70-48b0-b654-8ed6d3c8ea83 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:01.795384 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e0903-f06c-4a79-be64-ef079b00053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:01.796154 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e0903-f06c-4a79-be64-ef079b00053c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:01.796502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e0903-f06c-4a79-be64-ef079b00053c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:01.796789 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e0903-f06c-4a79-be64-ef079b00053c None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:01.804101 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e0903-f06c-4a79-be64-ef079b00053c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:01.805586 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e0903-f06c-4a79-be64-ef079b00053c None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-368e0903-f06c-4a79-be64-ef079b00053c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:01.819781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:01.820817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:01.821674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:01.821674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:01.821674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:01.821674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:01.821674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:01.821674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:01.821674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:01.821674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:01.821674 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 614/1217] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:11:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:01.826866 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05831b47-5a27-426c-8ce7-c130a1221af8 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:01.929732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05831b47-5a27-426c-8ce7-c130a1221af8 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:01.951852 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:01.963219 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9562ed5-d1ea-4710-aad6-16bbfae45c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:01.963894 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562ed5-d1ea-4710-aad6-16bbfae45c90 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:01.964173 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562ed5-d1ea-4710-aad6-16bbfae45c90 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:01.964488 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9562ed5-d1ea-4710-aad6-16bbfae45c90 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:01.965280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562ed5-d1ea-4710-aad6-16bbfae45c90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:01.965595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562ed5-d1ea-4710-aad6-16bbfae45c90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:01.967027 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9562ed5-d1ea-4710-aad6-16bbfae45c90 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:01.980081 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 604/1218] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:11:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:01.986884 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e0903-f06c-4a79-be64-ef079b00053c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:02.093367 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e0903-f06c-4a79-be64-ef079b00053c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:02.123317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:02.129573 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e334bc2-8e03-464d-b26b-0769af7f30c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:02.130232 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e334bc2-8e03-464d-b26b-0769af7f30c1 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:02.130517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e334bc2-8e03-464d-b26b-0769af7f30c1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:02.130791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e334bc2-8e03-464d-b26b-0769af7f30c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:02.131621 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e334bc2-8e03-464d-b26b-0769af7f30c1 None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=34483f3f0a934c50805c31b4af99176d) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:02.131932 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e334bc2-8e03-464d-b26b-0769af7f30c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:02.133425 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e334bc2-8e03-464d-b26b-0769af7f30c1 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:02.139242 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 615/1219] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:11:01 2022] GET /identity/v3/role_assignments?user.id=34483f3f0a934c50805c31b4af99176d&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:04.338058 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc99fc-96eb-45ac-ba46-7a0945c2d832 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.338568 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc99fc-96eb-45ac-ba46-7a0945c2d832 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.338568 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc99fc-96eb-45ac-ba46-7a0945c2d832 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.340031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 605/1220] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:04.345900 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db105e53-fc2f-4104-8b94-0e545834b012 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.346333 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db105e53-fc2f-4104-8b94-0e545834b012 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.346468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db105e53-fc2f-4104-8b94-0e545834b012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.364374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-db105e53-fc2f-4104-8b94-0e545834b012 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:04.365562 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-db105e53-fc2f-4104-8b94-0e545834b012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:04.431683 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 616/1221] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:11:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:04.454546 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc99fc-96eb-45ac-ba46-7a0945c2d832 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.559053 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc99fc-96eb-45ac-ba46-7a0945c2d832 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.598056 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752403d-5658-4bb7-8d9b-5f6e887dd60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.598779 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752403d-5658-4bb7-8d9b-5f6e887dd60b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.599056 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752403d-5658-4bb7-8d9b-5f6e887dd60b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.599319 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752403d-5658-4bb7-8d9b-5f6e887dd60b None admin] PATH_INFO: `/v3/roles/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.610211 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752403d-5658-4bb7-8d9b-5f6e887dd60b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-a752403d-5658-4bb7-8d9b-5f6e887dd60b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:04.618083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:04.619151 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:04.625070 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 606/1222] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:04.625819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db105e53-fc2f-4104-8b94-0e545834b012 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.726161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db105e53-fc2f-4104-8b94-0e545834b012 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.749313 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:04.759826 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8dd7f-2c68-473c-ae63-8ee1ffa6b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.760524 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8dd7f-2c68-473c-ae63-8ee1ffa6b8eb None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.760811 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8dd7f-2c68-473c-ae63-8ee1ffa6b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.761088 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b8dd7f-2c68-473c-ae63-8ee1ffa6b8eb None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.761830 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8dd7f-2c68-473c-ae63-8ee1ffa6b8eb None admin] RBAC: Adding query filter params (name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:04.762133 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8dd7f-2c68-473c-ae63-8ee1ffa6b8eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:04.763565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b8dd7f-2c68-473c-ae63-8ee1ffa6b8eb None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:04.772525 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 617/1223] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:04.778491 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752403d-5658-4bb7-8d9b-5f6e887dd60b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:04.836786 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:04.895655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752403d-5658-4bb7-8d9b-5f6e887dd60b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:04.929863 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052cde0-f10e-4f93-96a7-c44109218568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:04.930661 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052cde0-f10e-4f93-96a7-c44109218568 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:04.930946 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052cde0-f10e-4f93-96a7-c44109218568 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:04.931223 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052cde0-f10e-4f93-96a7-c44109218568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:04.944189 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052cde0-f10e-4f93-96a7-c44109218568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:04.945756 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052cde0-f10e-4f93-96a7-c44109218568 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-1052cde0-f10e-4f93-96a7-c44109218568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:04.968720 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:04.969578 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:04.970498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:04.970498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:04.970498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:04.970498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:04.970498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 607/1224] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:04.970911 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8dd7f-2c68-473c-ae63-8ee1ffa6b8eb None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.071652 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b8dd7f-2c68-473c-ae63-8ee1ffa6b8eb None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.111795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcda2b-17cb-4392-975c-957c9d948e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.112504 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcda2b-17cb-4392-975c-957c9d948e19 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.112791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcda2b-17cb-4392-975c-957c9d948e19 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.113069 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcda2b-17cb-4392-975c-957c9d948e19 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.113789 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcda2b-17cb-4392-975c-957c9d948e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:05.114097 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcda2b-17cb-4392-975c-957c9d948e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.115482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcda2b-17cb-4392-975c-957c9d948e19 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.128029 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 618/1225] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:05.133864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052cde0-f10e-4f93-96a7-c44109218568 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.237465 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.240498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052cde0-f10e-4f93-96a7-c44109218568 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.275475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abd5e1-5b4a-4839-8dc4-66f3cdb1f568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.276325 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abd5e1-5b4a-4839-8dc4-66f3cdb1f568 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.276633 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abd5e1-5b4a-4839-8dc4-66f3cdb1f568 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.276904 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abd5e1-5b4a-4839-8dc4-66f3cdb1f568 None admin] PATH_INFO: `/v3/users/placement` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.284452 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abd5e1-5b4a-4839-8dc4-66f3cdb1f568 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.285962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abd5e1-5b4a-4839-8dc4-66f3cdb1f568 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-47abd5e1-5b4a-4839-8dc4-66f3cdb1f568 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:05.301013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:05.303764 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:05.304728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:05.304728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:05.304728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:05.304728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:05.304728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:05.304728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:05.304728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:05.304728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:05.304728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:05.304728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:05.304728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 608/1226] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:05.308652 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcda2b-17cb-4392-975c-957c9d948e19 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.366420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.417761 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcda2b-17cb-4392-975c-957c9d948e19 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.451992 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a071cb-2c08-4ae2-9bae-cd21a9d7546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.452690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a071cb-2c08-4ae2-9bae-cd21a9d7546b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.452976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a071cb-2c08-4ae2-9bae-cd21a9d7546b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.453254 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a071cb-2c08-4ae2-9bae-cd21a9d7546b None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.454164 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a071cb-2c08-4ae2-9bae-cd21a9d7546b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:05.454471 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a071cb-2c08-4ae2-9bae-cd21a9d7546b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.455915 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a071cb-2c08-4ae2-9bae-cd21a9d7546b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.475980 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 619/1227] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:05.481699 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abd5e1-5b4a-4839-8dc4-66f3cdb1f568 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.586863 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abd5e1-5b4a-4839-8dc4-66f3cdb1f568 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.620630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec803-f546-4241-8759-f5ad3deedc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.621418 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec803-f546-4241-8759-f5ad3deedc57 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.621700 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec803-f546-4241-8759-f5ad3deedc57 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.621972 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec803-f546-4241-8759-f5ad3deedc57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.634185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ec803-f546-4241-8759-f5ad3deedc57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.635640 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ec803-f546-4241-8759-f5ad3deedc57 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-381ec803-f546-4241-8759-f5ad3deedc57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:05.652727 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:05.653873 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:05.654747 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:05.654747 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:05.654747 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:05.654747 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:05.654747 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 609/1228] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:05.660167 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a071cb-2c08-4ae2-9bae-cd21a9d7546b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.770856 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a071cb-2c08-4ae2-9bae-cd21a9d7546b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.773225 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.806449 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d31db6-43e8-45ba-8ab8-1ab15b4b32b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.807111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d31db6-43e8-45ba-8ab8-1ab15b4b32b8 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.807391 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d31db6-43e8-45ba-8ab8-1ab15b4b32b8 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.807661 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d31db6-43e8-45ba-8ab8-1ab15b4b32b8 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.808397 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d31db6-43e8-45ba-8ab8-1ab15b4b32b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:05.808749 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d31db6-43e8-45ba-8ab8-1ab15b4b32b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.810118 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d31db6-43e8-45ba-8ab8-1ab15b4b32b8 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.821623 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 620/1229] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:05.827314 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec803-f546-4241-8759-f5ad3deedc57 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:05.837094 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:05.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec803-f546-4241-8759-f5ad3deedc57 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:05.964354 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b28fa3-c262-4ab6-964a-62617acc2d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:05.965164 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b28fa3-c262-4ab6-964a-62617acc2d01 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:05.965449 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b28fa3-c262-4ab6-964a-62617acc2d01 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:05.965723 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b28fa3-c262-4ab6-964a-62617acc2d01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:05.972452 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b28fa3-c262-4ab6-964a-62617acc2d01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:05.973908 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b28fa3-c262-4ab6-964a-62617acc2d01 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-a8b28fa3-c262-4ab6-964a-62617acc2d01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:05.988178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:05.994698 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:05.995767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:05.995767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:05.995767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:05.995767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:05.995767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:05.995767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:05.995767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:05.995767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:05.995767 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 610/1230] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:05.996666 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d31db6-43e8-45ba-8ab8-1ab15b4b32b8 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.104691 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d31db6-43e8-45ba-8ab8-1ab15b4b32b8 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:06.140316 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102f877-834e-4044-9144-cbc427439d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:06.141680 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102f877-834e-4044-9144-cbc427439d99 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:06.141965 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102f877-834e-4044-9144-cbc427439d99 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:06.142241 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102f877-834e-4044-9144-cbc427439d99 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:06.143025 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102f877-834e-4044-9144-cbc427439d99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:06.143359 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102f877-834e-4044-9144-cbc427439d99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:06.144874 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102f877-834e-4044-9144-cbc427439d99 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:06.157977 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 621/1231] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:11:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:06.163809 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b28fa3-c262-4ab6-964a-62617acc2d01 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:06.231635 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:06.265911 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b28fa3-c262-4ab6-964a-62617acc2d01 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:06.301935 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24479ecb-16ee-4b8e-9820-1b2d1289d3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:06.302823 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24479ecb-16ee-4b8e-9820-1b2d1289d3c9 None admin] REQUEST_METHOD: `PUT` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:06.303105 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24479ecb-16ee-4b8e-9820-1b2d1289d3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:06.303381 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24479ecb-16ee-4b8e-9820-1b2d1289d3c9 None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/34483f3f0a934c50805c31b4af99176d/roles/afaf4273358148d3b0b9cac8dbeeba97` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:06.318315 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24479ecb-16ee-4b8e-9820-1b2d1289d3c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=34483f3f0a934c50805c31b4af99176d, role_id=afaf4273358148d3b0b9cac8dbeeba97)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:06.319757 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24479ecb-16ee-4b8e-9820-1b2d1289d3c9 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:06.411334 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 611/1232] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:11:06 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/34483f3f0a934c50805c31b4af99176d/roles/afaf4273358148d3b0b9cac8dbeeba97 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:11:08.648602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e56de-04d1-4101-b8ed-101223b05426 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:08.649139 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e56de-04d1-4101-b8ed-101223b05426 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:08.649139 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e56de-04d1-4101-b8ed-101223b05426 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:08.650592 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 622/1233] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:08.655993 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b453f-69fa-4031-846e-2c9f828208db None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:08.656451 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b453f-69fa-4031-846e-2c9f828208db None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:08.657126 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b453f-69fa-4031-846e-2c9f828208db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:08.674055 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-ec9b453f-69fa-4031-846e-2c9f828208db None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:08.675498 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-ec9b453f-69fa-4031-846e-2c9f828208db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:08.744655 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 612/1234] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:11:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:08.768214 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e56de-04d1-4101-b8ed-101223b05426 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:08.794615 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:08.879024 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e56de-04d1-4101-b8ed-101223b05426 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:08.913232 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22408c59-bba1-4c4c-b1e2-7e0160d558c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:08.913949 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22408c59-bba1-4c4c-b1e2-7e0160d558c1 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:08.914219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22408c59-bba1-4c4c-b1e2-7e0160d558c1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:08.914485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22408c59-bba1-4c4c-b1e2-7e0160d558c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:08.925947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22408c59-bba1-4c4c-b1e2-7e0160d558c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-22408c59-bba1-4c4c-b1e2-7e0160d558c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:08.933891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:08.934917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:08.935738 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 623/1235] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:11:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:08.941325 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b453f-69fa-4031-846e-2c9f828208db None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:08.989241 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:09.044352 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b453f-69fa-4031-846e-2c9f828208db None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:09.080827 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e59550-3ec9-4338-b8b2-cb94653e5322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:09.081521 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e59550-3ec9-4338-b8b2-cb94653e5322 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:09.081806 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e59550-3ec9-4338-b8b2-cb94653e5322 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:09.082086 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e59550-3ec9-4338-b8b2-cb94653e5322 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:09.082849 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e59550-3ec9-4338-b8b2-cb94653e5322 None admin] RBAC: Adding query filter params (name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:09.083164 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e59550-3ec9-4338-b8b2-cb94653e5322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:09.084607 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e59550-3ec9-4338-b8b2-cb94653e5322 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:09.093089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 613/1236] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:11:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:09.099362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22408c59-bba1-4c4c-b1e2-7e0160d558c1 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:09.183867 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:09.199771 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22408c59-bba1-4c4c-b1e2-7e0160d558c1 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:09.234023 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed1a8b-9d23-43c5-a6a0-250f41729877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:09.234814 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed1a8b-9d23-43c5-a6a0-250f41729877 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:09.235096 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed1a8b-9d23-43c5-a6a0-250f41729877 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:09.235371 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed1a8b-9d23-43c5-a6a0-250f41729877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:09.245923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed1a8b-9d23-43c5-a6a0-250f41729877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:09.247381 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed1a8b-9d23-43c5-a6a0-250f41729877 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-29ed1a8b-9d23-43c5-a6a0-250f41729877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:09.264050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:09.265161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:09.266642 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:09.266642 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:09.266642 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:09.266642 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:09.266642 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 624/1237] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:09.271594 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e59550-3ec9-4338-b8b2-cb94653e5322 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:09.371667 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e59550-3ec9-4338-b8b2-cb94653e5322 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:09.380601 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:09.405252 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f99576-3f87-4728-a5bf-b14c47d01bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:09.405918 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f99576-3f87-4728-a5bf-b14c47d01bfa None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:09.406203 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f99576-3f87-4728-a5bf-b14c47d01bfa None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:09.406482 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f99576-3f87-4728-a5bf-b14c47d01bfa None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:09.407771 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f99576-3f87-4728-a5bf-b14c47d01bfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:09.408083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f99576-3f87-4728-a5bf-b14c47d01bfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:09.409541 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f99576-3f87-4728-a5bf-b14c47d01bfa None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:09.422085 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 614/1238] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:09.428022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed1a8b-9d23-43c5-a6a0-250f41729877 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:09.530434 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed1a8b-9d23-43c5-a6a0-250f41729877 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:09.566138 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d86314-b601-471f-8d7b-ae34fa302615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:09.566938 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d86314-b601-471f-8d7b-ae34fa302615 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:09.567223 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d86314-b601-471f-8d7b-ae34fa302615 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:09.567502 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d86314-b601-471f-8d7b-ae34fa302615 None admin] PATH_INFO: `/v3/users/placement` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:09.570319 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:09.575360 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d86314-b601-471f-8d7b-ae34fa302615 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:09.576917 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d86314-b601-471f-8d7b-ae34fa302615 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-72d86314-b601-471f-8d7b-ae34fa302615 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:09.591797 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:09.593231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:09.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:09.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:09.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:09.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:09.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:09.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:09.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:09.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:09.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:09.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:09.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 625/1239] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:09.599385 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f99576-3f87-4728-a5bf-b14c47d01bfa None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:09.703490 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f99576-3f87-4728-a5bf-b14c47d01bfa None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:09.739178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bd290-3092-4d32-a688-81f8c3d08477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:09.739855 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd290-3092-4d32-a688-81f8c3d08477 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:09.740135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd290-3092-4d32-a688-81f8c3d08477 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:09.740458 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bd290-3092-4d32-a688-81f8c3d08477 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:09.741299 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bd290-3092-4d32-a688-81f8c3d08477 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:09.741613 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bd290-3092-4d32-a688-81f8c3d08477 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:09.743040 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bd290-3092-4d32-a688-81f8c3d08477 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:09.761586 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 615/1240] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:11:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:09.767823 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d86314-b601-471f-8d7b-ae34fa302615 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:09.875219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d86314-b601-471f-8d7b-ae34fa302615 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:09.908878 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e323e3-7437-4afb-b7df-26d02812f253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:09.909526 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e323e3-7437-4afb-b7df-26d02812f253 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:09.909811 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e323e3-7437-4afb-b7df-26d02812f253 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:09.910088 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e323e3-7437-4afb-b7df-26d02812f253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:09.971140 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:09.977768 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e323e3-7437-4afb-b7df-26d02812f253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:09.979219 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e323e3-7437-4afb-b7df-26d02812f253 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:10.077398 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 626/1241] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:11:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:11:10.083378 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bd290-3092-4d32-a688-81f8c3d08477 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:10.111563 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:10.186089 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bd290-3092-4d32-a688-81f8c3d08477 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:10.220990 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd3282-d89a-4e94-a1c1-515e9e91a831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:10.221787 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd3282-d89a-4e94-a1c1-515e9e91a831 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.222068 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd3282-d89a-4e94-a1c1-515e9e91a831 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.222344 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd3282-d89a-4e94-a1c1-515e9e91a831 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.232503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd3282-d89a-4e94-a1c1-515e9e91a831 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:10.233945 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd3282-d89a-4e94-a1c1-515e9e91a831 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-6cdd3282-d89a-4e94-a1c1-515e9e91a831 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:10.251681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:10.252740 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:10.253837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:10.253837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:10.253837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:10.253837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:10.253837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 616/1242] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:10.258688 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e323e3-7437-4afb-b7df-26d02812f253 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:10.360112 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:10.365804 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e323e3-7437-4afb-b7df-26d02812f253 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:10.399372 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e48e8-c637-41c8-ab58-27064a825575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:10.400021 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e48e8-c637-41c8-ab58-27064a825575 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.400303 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e48e8-c637-41c8-ab58-27064a825575 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.400608 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e48e8-c637-41c8-ab58-27064a825575 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.401317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e48e8-c637-41c8-ab58-27064a825575 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:10.401664 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e48e8-c637-41c8-ab58-27064a825575 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:10.403037 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e48e8-c637-41c8-ab58-27064a825575 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:10.414210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 627/1243] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:10.420145 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd3282-d89a-4e94-a1c1-515e9e91a831 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:10.498988 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:10.521336 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd3282-d89a-4e94-a1c1-515e9e91a831 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:10.555178 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c265c56-db92-4283-a782-f53f9428a0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:10.555950 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c265c56-db92-4283-a782-f53f9428a0d1 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.556231 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c265c56-db92-4283-a782-f53f9428a0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.556545 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c265c56-db92-4283-a782-f53f9428a0d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.563681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c265c56-db92-4283-a782-f53f9428a0d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:10.565254 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c265c56-db92-4283-a782-f53f9428a0d1 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-4c265c56-db92-4283-a782-f53f9428a0d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:10.578887 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:10.585054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:10.586242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:10.586242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:10.586242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:10.586242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:10.586242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:10.586242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:10.586242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:10.586242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:10.586242 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 617/1244] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:11:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:10.586926 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e48e8-c637-41c8-ab58-27064a825575 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:10.689301 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e48e8-c637-41c8-ab58-27064a825575 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:10.723078 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a5734-d2d9-4f10-91f0-a5f3390fa988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:10.723744 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a5734-d2d9-4f10-91f0-a5f3390fa988 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.724022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a5734-d2d9-4f10-91f0-a5f3390fa988 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.724297 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a5734-d2d9-4f10-91f0-a5f3390fa988 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.725110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a5734-d2d9-4f10-91f0-a5f3390fa988 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:10.725420 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a5734-d2d9-4f10-91f0-a5f3390fa988 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:10.726933 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a5734-d2d9-4f10-91f0-a5f3390fa988 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:10.739680 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 628/1245] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:11:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:10.745846 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c265c56-db92-4283-a782-f53f9428a0d1 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:10.849335 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c265c56-db92-4283-a782-f53f9428a0d1 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:10.894474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90bd29-04b4-4d3c-a9af-06575387f730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:10.895141 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90bd29-04b4-4d3c-a9af-06575387f730 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:10.895427 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90bd29-04b4-4d3c-a9af-06575387f730 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:10.895704 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90bd29-04b4-4d3c-a9af-06575387f730 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:10.896647 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90bd29-04b4-4d3c-a9af-06575387f730 None admin] RBAC: Adding query filter params (role.id=afaf4273358148d3b0b9cac8dbeeba97, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=34483f3f0a934c50805c31b4af99176d) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:10.896967 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90bd29-04b4-4d3c-a9af-06575387f730 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:10.898413 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90bd29-04b4-4d3c-a9af-06575387f730 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:10.904670 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 618/1246] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:11:10 2022] GET /identity/v3/role_assignments?user.id=34483f3f0a934c50805c31b4af99176d&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=afaf4273358148d3b0b9cac8dbeeba97 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:13.208090 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8997c-9f98-4849-9113-2e1f88f3b296 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.208709 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8997c-9f98-4849-9113-2e1f88f3b296 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.208709 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8997c-9f98-4849-9113-2e1f88f3b296 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.210085 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 629/1247] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:13.215694 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bdf3b-a614-4467-8775-b3fffcd86910 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.216179 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bdf3b-a614-4467-8775-b3fffcd86910 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.216304 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bdf3b-a614-4467-8775-b3fffcd86910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.222040 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:13.233968 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-359bdf3b-a614-4467-8775-b3fffcd86910 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:13.235779 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-359bdf3b-a614-4467-8775-b3fffcd86910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:13.303221 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 619/1248] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:11:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:13.326829 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8997c-9f98-4849-9113-2e1f88f3b296 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.345428 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:13.427733 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8997c-9f98-4849-9113-2e1f88f3b296 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.461836 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab6200-147f-4202-8f2c-3b4bbe0c71b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.462627 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab6200-147f-4202-8f2c-3b4bbe0c71b1 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.462914 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab6200-147f-4202-8f2c-3b4bbe0c71b1 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.463197 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab6200-147f-4202-8f2c-3b4bbe0c71b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.473172 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab6200-147f-4202-8f2c-3b4bbe0c71b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-37ab6200-147f-4202-8f2c-3b4bbe0c71b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:13.480901 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:13.482000 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:13.483020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 630/1249] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:13.488296 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bdf3b-a614-4467-8775-b3fffcd86910 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.589007 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bdf3b-a614-4467-8775-b3fffcd86910 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.618642 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:13.623447 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f26c9-d3ed-4e38-9401-5ed180234c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.624118 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f26c9-d3ed-4e38-9401-5ed180234c17 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.624432 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f26c9-d3ed-4e38-9401-5ed180234c17 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.624715 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f26c9-d3ed-4e38-9401-5ed180234c17 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.625492 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f26c9-d3ed-4e38-9401-5ed180234c17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:13.625800 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f26c9-d3ed-4e38-9401-5ed180234c17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.627183 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f26c9-d3ed-4e38-9401-5ed180234c17 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.635207 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 620/1250] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:13.641472 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab6200-147f-4202-8f2c-3b4bbe0c71b1 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.719644 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:13.741684 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab6200-147f-4202-8f2c-3b4bbe0c71b1 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.775685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80c269-afae-4723-b3b1-a85d56451e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.776508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80c269-afae-4723-b3b1-a85d56451e0b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.776795 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80c269-afae-4723-b3b1-a85d56451e0b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.777074 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80c269-afae-4723-b3b1-a85d56451e0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.788269 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80c269-afae-4723-b3b1-a85d56451e0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.789770 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80c269-afae-4723-b3b1-a85d56451e0b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-7c80c269-afae-4723-b3b1-a85d56451e0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:13.806801 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:13.807842 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:13.808844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:13.808844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:13.808844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:13.808844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:13.808844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 631/1251] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:13.814010 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f26c9-d3ed-4e38-9401-5ed180234c17 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:13.913864 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f26c9-d3ed-4e38-9401-5ed180234c17 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:13.947975 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa6d83-f8d6-4540-87b8-5a84ff19f67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:13.948665 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa6d83-f8d6-4540-87b8-5a84ff19f67f None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:13.948947 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa6d83-f8d6-4540-87b8-5a84ff19f67f None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:13.949222 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa6d83-f8d6-4540-87b8-5a84ff19f67f None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:13.949974 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa6d83-f8d6-4540-87b8-5a84ff19f67f None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:13.950281 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa6d83-f8d6-4540-87b8-5a84ff19f67f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:13.951648 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa6d83-f8d6-4540-87b8-5a84ff19f67f None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:13.964490 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 621/1252] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:13.970215 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80c269-afae-4723-b3b1-a85d56451e0b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.072791 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80c269-afae-4723-b3b1-a85d56451e0b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.099779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.108404 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa26e7c-ecc1-49cf-9062-a3908f300b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.109236 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa26e7c-ecc1-49cf-9062-a3908f300b3a None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.109548 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa26e7c-ecc1-49cf-9062-a3908f300b3a None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.109826 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa26e7c-ecc1-49cf-9062-a3908f300b3a None admin] PATH_INFO: `/v3/users/placement` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.118050 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa26e7c-ecc1-49cf-9062-a3908f300b3a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.119498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa26e7c-ecc1-49cf-9062-a3908f300b3a None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-2aa26e7c-ecc1-49cf-9062-a3908f300b3a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:14.133939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:14.134973 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:14.136181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:14.136181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:14.136181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:14.136181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:14.136181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:14.136181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:14.136181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:14.136181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:14.136181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:14.136181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:14.136181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 632/1253] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:14.141827 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa6d83-f8d6-4540-87b8-5a84ff19f67f None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.198998 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.244464 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa6d83-f8d6-4540-87b8-5a84ff19f67f None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.278478 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976a053-756d-4098-911b-dbde9fab9766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.279140 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976a053-756d-4098-911b-dbde9fab9766 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.279430 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976a053-756d-4098-911b-dbde9fab9766 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.279717 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976a053-756d-4098-911b-dbde9fab9766 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.280567 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976a053-756d-4098-911b-dbde9fab9766 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:14.280878 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976a053-756d-4098-911b-dbde9fab9766 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.282925 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976a053-756d-4098-911b-dbde9fab9766 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.299914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 622/1254] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:14.305858 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa26e7c-ecc1-49cf-9062-a3908f300b3a None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.404135 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa26e7c-ecc1-49cf-9062-a3908f300b3a None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.437963 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80833f9c-b7fa-4b9f-a31b-0ea370132a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.438603 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80833f9c-b7fa-4b9f-a31b-0ea370132a21 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.438890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80833f9c-b7fa-4b9f-a31b-0ea370132a21 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.439167 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80833f9c-b7fa-4b9f-a31b-0ea370132a21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.497362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.510575 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80833f9c-b7fa-4b9f-a31b-0ea370132a21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.512074 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80833f9c-b7fa-4b9f-a31b-0ea370132a21 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.610173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 633/1255] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:11:14.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976a053-756d-4098-911b-dbde9fab9766 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.717861 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976a053-756d-4098-911b-dbde9fab9766 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.731245 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.752171 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe02b72-86d8-4487-abc3-949117e2175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.752933 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe02b72-86d8-4487-abc3-949117e2175a None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.753215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe02b72-86d8-4487-abc3-949117e2175a None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.753489 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe02b72-86d8-4487-abc3-949117e2175a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.763358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe02b72-86d8-4487-abc3-949117e2175a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.764839 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe02b72-86d8-4487-abc3-949117e2175a None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-afe02b72-86d8-4487-abc3-949117e2175a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:14.782404 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:14.787133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:14.788111 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:14.788111 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:14.788111 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:14.788111 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:14.788111 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 623/1256] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:14.789482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80833f9c-b7fa-4b9f-a31b-0ea370132a21 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:14.888820 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:14.898217 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80833f9c-b7fa-4b9f-a31b-0ea370132a21 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:14.933256 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d045a5-a4c9-4055-97c0-c9be093efc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:14.933912 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d045a5-a4c9-4055-97c0-c9be093efc7b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:14.934197 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d045a5-a4c9-4055-97c0-c9be093efc7b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:14.934472 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d045a5-a4c9-4055-97c0-c9be093efc7b None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:14.935188 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d045a5-a4c9-4055-97c0-c9be093efc7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:14.935497 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d045a5-a4c9-4055-97c0-c9be093efc7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:14.937128 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d045a5-a4c9-4055-97c0-c9be093efc7b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:14.948725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 634/1257] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:14.954493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe02b72-86d8-4487-abc3-949117e2175a None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:15.056137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe02b72-86d8-4487-abc3-949117e2175a None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:15.091091 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d34b64-02da-4b1e-acc1-03ef93663418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:15.093445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d34b64-02da-4b1e-acc1-03ef93663418 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.093445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d34b64-02da-4b1e-acc1-03ef93663418 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.093445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d34b64-02da-4b1e-acc1-03ef93663418 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.100013 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d34b64-02da-4b1e-acc1-03ef93663418 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:15.101508 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d34b64-02da-4b1e-acc1-03ef93663418 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-31d34b64-02da-4b1e-acc1-03ef93663418 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:15.115547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:15.116608 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:15.117814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:15.117814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:15.117814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:15.117814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:15.117814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:15.117814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:15.117814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:15.117814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:15.117814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 624/1258] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:11:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:15.122572 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d045a5-a4c9-4055-97c0-c9be093efc7b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:15.227364 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d045a5-a4c9-4055-97c0-c9be093efc7b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:15.260787 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91856ee-6df5-422d-b6ce-93ca1d7d50f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:15.261440 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91856ee-6df5-422d-b6ce-93ca1d7d50f7 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.261724 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91856ee-6df5-422d-b6ce-93ca1d7d50f7 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.262005 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91856ee-6df5-422d-b6ce-93ca1d7d50f7 None admin] PATH_INFO: `/v3/projects` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.262783 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91856ee-6df5-422d-b6ce-93ca1d7d50f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:15.263091 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91856ee-6df5-422d-b6ce-93ca1d7d50f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:15.264553 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91856ee-6df5-422d-b6ce-93ca1d7d50f7 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:15.276963 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 635/1259] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:11:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:15.283779 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d34b64-02da-4b1e-acc1-03ef93663418 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:15.309986 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:15.384472 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d34b64-02da-4b1e-acc1-03ef93663418 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:15.418234 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6082f6-40d7-46b4-a295-4d5df581aa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:15.418879 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6082f6-40d7-46b4-a295-4d5df581aa01 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:15.419158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6082f6-40d7-46b4-a295-4d5df581aa01 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:15.419432 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6082f6-40d7-46b4-a295-4d5df581aa01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:15.420263 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6082f6-40d7-46b4-a295-4d5df581aa01 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=34483f3f0a934c50805c31b4af99176d) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:15.420644 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6082f6-40d7-46b4-a295-4d5df581aa01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:15.422644 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6082f6-40d7-46b4-a295-4d5df581aa01 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:15.428285 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 625/1260] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:11:15 2022] GET /identity/v3/role_assignments?user.id=34483f3f0a934c50805c31b4af99176d&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:17.628916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39053a6f-017e-43c8-b183-183ff255f160 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:17.629481 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39053a6f-017e-43c8-b183-183ff255f160 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:17.629481 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39053a6f-017e-43c8-b183-183ff255f160 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:17.630845 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 636/1261] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:17.635996 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6ce0c-c52a-4556-a1dd-a997e920ba19 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:17.636396 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6ce0c-c52a-4556-a1dd-a997e920ba19 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:17.636597 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6ce0c-c52a-4556-a1dd-a997e920ba19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:17.653872 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-cfe6ce0c-c52a-4556-a1dd-a997e920ba19 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:17.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-cfe6ce0c-c52a-4556-a1dd-a997e920ba19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:17.722650 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 626/1262] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:11:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:17.744849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39053a6f-017e-43c8-b183-183ff255f160 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:17.756498 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:17.847726 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39053a6f-017e-43c8-b183-183ff255f160 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:17.882890 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5680b-b492-4e46-81d8-ff1745598c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:17.883556 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5680b-b492-4e46-81d8-ff1745598c9c None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:17.883844 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5680b-b492-4e46-81d8-ff1745598c9c None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:17.884116 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5680b-b492-4e46-81d8-ff1745598c9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:17.894823 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5680b-b492-4e46-81d8-ff1745598c9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-12a5680b-b492-4e46-81d8-ff1745598c9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:17.902431 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:17.903702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:17.904588 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 637/1263] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:11:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:17.909687 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6ce0c-c52a-4556-a1dd-a997e920ba19 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:17.928259 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:18.009297 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6ce0c-c52a-4556-a1dd-a997e920ba19 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.043683 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cbd0d-edac-4b62-9f68-58f09a5f9de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.044368 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cbd0d-edac-4b62-9f68-58f09a5f9de8 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.044658 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cbd0d-edac-4b62-9f68-58f09a5f9de8 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.044936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8cbd0d-edac-4b62-9f68-58f09a5f9de8 None admin] PATH_INFO: `/v3/roles` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.045679 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cbd0d-edac-4b62-9f68-58f09a5f9de8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:18.045987 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cbd0d-edac-4b62-9f68-58f09a5f9de8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.047894 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8cbd0d-edac-4b62-9f68-58f09a5f9de8 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.056280 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 627/1264] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:11:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:18.062288 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5680b-b492-4e46-81d8-ff1745598c9c None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.131997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:18.164849 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5680b-b492-4e46-81d8-ff1745598c9c None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.198034 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aaac31-7656-4a91-8e4c-3bdfaafb8b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.198821 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaac31-7656-4a91-8e4c-3bdfaafb8b30 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.199100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaac31-7656-4a91-8e4c-3bdfaafb8b30 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.199375 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aaac31-7656-4a91-8e4c-3bdfaafb8b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.209880 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aaac31-7656-4a91-8e4c-3bdfaafb8b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.211358 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aaac31-7656-4a91-8e4c-3bdfaafb8b30 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-b6aaac31-7656-4a91-8e4c-3bdfaafb8b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:18.229690 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:18.230696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:18.231758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:18.231758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:18.231758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:18.231758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:18.231758 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 638/1265] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:18.237338 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cbd0d-edac-4b62-9f68-58f09a5f9de8 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.319804 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:18.338334 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8cbd0d-edac-4b62-9f68-58f09a5f9de8 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.372115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c543886-6192-4dbd-9253-f934da5bf3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.376358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c543886-6192-4dbd-9253-f934da5bf3bb None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.376358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c543886-6192-4dbd-9253-f934da5bf3bb None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.376358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c543886-6192-4dbd-9253-f934da5bf3bb None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.376358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c543886-6192-4dbd-9253-f934da5bf3bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:18.376358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c543886-6192-4dbd-9253-f934da5bf3bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.376358 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c543886-6192-4dbd-9253-f934da5bf3bb None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.388149 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 628/1266] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:18.394095 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aaac31-7656-4a91-8e4c-3bdfaafb8b30 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.496066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aaac31-7656-4a91-8e4c-3bdfaafb8b30 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.515292 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:18.528811 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ceb0a-0d75-40ee-985b-223042d3c35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.529601 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ceb0a-0d75-40ee-985b-223042d3c35b None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.529883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ceb0a-0d75-40ee-985b-223042d3c35b None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.530158 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ceb0a-0d75-40ee-985b-223042d3c35b None admin] PATH_INFO: `/v3/users/placement` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.537984 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ceb0a-0d75-40ee-985b-223042d3c35b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.539548 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ceb0a-0d75-40ee-985b-223042d3c35b None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-372ceb0a-0d75-40ee-985b-223042d3c35b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:18.554527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:18.555633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:18.556648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:18.556648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:18.556648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:18.556648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:18.556648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:18.556648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:18.556648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:18.556648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:18.556648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:18.556648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:18.556648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 639/1267] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:18.561718 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c543886-6192-4dbd-9253-f934da5bf3bb None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.661952 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c543886-6192-4dbd-9253-f934da5bf3bb None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.697327 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1181c7f-dc1c-4752-83a6-603fb7fe1a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.697987 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1181c7f-dc1c-4752-83a6-603fb7fe1a86 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.698266 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1181c7f-dc1c-4752-83a6-603fb7fe1a86 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.698541 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1181c7f-dc1c-4752-83a6-603fb7fe1a86 None admin] PATH_INFO: `/v3/users` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.699408 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1181c7f-dc1c-4752-83a6-603fb7fe1a86 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:18.699776 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1181c7f-dc1c-4752-83a6-603fb7fe1a86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.701253 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1181c7f-dc1c-4752-83a6-603fb7fe1a86 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.710096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:18.718614 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 629/1268] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:18.725123 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ceb0a-0d75-40ee-985b-223042d3c35b None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:18.836309 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ceb0a-0d75-40ee-985b-223042d3c35b None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:18.872552 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026ea3c-0d5a-4fc5-93ce-1b9026e01ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:18.873373 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026ea3c-0d5a-4fc5-93ce-1b9026e01ff5 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:18.873653 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026ea3c-0d5a-4fc5-93ce-1b9026e01ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:18.873926 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026ea3c-0d5a-4fc5-93ce-1b9026e01ff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:18.886056 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026ea3c-0d5a-4fc5-93ce-1b9026e01ff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:18.892653 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026ea3c-0d5a-4fc5-93ce-1b9026e01ff5 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-3026ea3c-0d5a-4fc5-93ce-1b9026e01ff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:18.906429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:18.908051 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:18.909035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:18.909035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:18.909035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:18.909035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:18.909035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 640/1269] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:18.914385 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1181c7f-dc1c-4752-83a6-603fb7fe1a86 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.026010 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1181c7f-dc1c-4752-83a6-603fb7fe1a86 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.063435 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec844b1d-665e-4e76-9aad-229eaf595d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.064116 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec844b1d-665e-4e76-9aad-229eaf595d71 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.064424 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec844b1d-665e-4e76-9aad-229eaf595d71 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.065458 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec844b1d-665e-4e76-9aad-229eaf595d71 None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.065458 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec844b1d-665e-4e76-9aad-229eaf595d71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:19.065732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec844b1d-665e-4e76-9aad-229eaf595d71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:19.067136 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec844b1d-665e-4e76-9aad-229eaf595d71 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:19.080012 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 630/1270] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:19.085903 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026ea3c-0d5a-4fc5-93ce-1b9026e01ff5 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.120870 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:19.194499 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026ea3c-0d5a-4fc5-93ce-1b9026e01ff5 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.229367 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510eb50f-4c2a-40d4-9c29-827aa3c467e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.230169 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb50f-4c2a-40d4-9c29-827aa3c467e4 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.230454 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb50f-4c2a-40d4-9c29-827aa3c467e4 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.230730 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510eb50f-4c2a-40d4-9c29-827aa3c467e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.239754 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510eb50f-4c2a-40d4-9c29-827aa3c467e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:19.240043 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510eb50f-4c2a-40d4-9c29-827aa3c467e4 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-510eb50f-4c2a-40d4-9c29-827aa3c467e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:19.255711 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:19.256819 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:19.257710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:19.257710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:19.257710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:19.257710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:19.257710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:19.257710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:19.257710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:19.257710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:19.257710 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 641/1271] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:11:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:19.263018 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec844b1d-665e-4e76-9aad-229eaf595d71 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.326919 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:19.372261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec844b1d-665e-4e76-9aad-229eaf595d71 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.407426 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28071d79-644a-43ca-892c-04749e0007e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.408090 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28071d79-644a-43ca-892c-04749e0007e3 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.408398 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28071d79-644a-43ca-892c-04749e0007e3 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.408786 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28071d79-644a-43ca-892c-04749e0007e3 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.409582 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28071d79-644a-43ca-892c-04749e0007e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:19.409891 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28071d79-644a-43ca-892c-04749e0007e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:19.411324 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28071d79-644a-43ca-892c-04749e0007e3 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:19.425201 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 631/1272] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:11:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:19.431610 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510eb50f-4c2a-40d4-9c29-827aa3c467e4 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:19.529015 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:19.537636 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510eb50f-4c2a-40d4-9c29-827aa3c467e4 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:19.574914 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b10bec-2916-44f1-9ee1-824d0225b72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:19.576161 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b10bec-2916-44f1-9ee1-824d0225b72e None admin] REQUEST_METHOD: `PUT` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:19.576667 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b10bec-2916-44f1-9ee1-824d0225b72e None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:19.577573 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b10bec-2916-44f1-9ee1-824d0225b72e None admin] PATH_INFO: `/v3/projects/adcff914910e46c181b074c0f7497994/users/34483f3f0a934c50805c31b4af99176d/roles/ab7a63cec3a744848972fcbb89ecd0ca` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:19.579760 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b10bec-2916-44f1-9ee1-824d0225b72e None admin] RBAC: Authorizing `identity:create_grant(project_id=adcff914910e46c181b074c0f7497994, user_id=34483f3f0a934c50805c31b4af99176d, role_id=ab7a63cec3a744848972fcbb89ecd0ca)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:19.581262 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b10bec-2916-44f1-9ee1-824d0225b72e None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:19.662355 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 642/1273] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:11:19 2022] PUT /identity/v3/projects/adcff914910e46c181b074c0f7497994/users/34483f3f0a934c50805c31b4af99176d/roles/ab7a63cec3a744848972fcbb89ecd0ca => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:11:21.887697 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c6c19-1a13-415c-bbf4-7cf476a8aa81 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:21.888362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c6c19-1a13-415c-bbf4-7cf476a8aa81 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:21.888362 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c6c19-1a13-415c-bbf4-7cf476a8aa81 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:21.889736 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 632/1274] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:21.895017 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd0728-dcf0-445f-b306-795b6b364feb None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:21.895426 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd0728-dcf0-445f-b306-795b6b364feb None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:21.895579 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd0728-dcf0-445f-b306-795b6b364feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:21.911927 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-50dd0728-dcf0-445f-b306-795b6b364feb None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:21.913166 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-50dd0728-dcf0-445f-b306-795b6b364feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:21.985064 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 643/1275] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:11:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:22.010154 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c6c19-1a13-415c-bbf4-7cf476a8aa81 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.115083 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c6c19-1a13-415c-bbf4-7cf476a8aa81 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.133158 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:22.150208 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1ee43-4b1e-4a0e-8ee8-0c2205a12898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.150936 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1ee43-4b1e-4a0e-8ee8-0c2205a12898 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.151266 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1ee43-4b1e-4a0e-8ee8-0c2205a12898 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.151545 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1ee43-4b1e-4a0e-8ee8-0c2205a12898 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.162148 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1ee43-4b1e-4a0e-8ee8-0c2205a12898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-e4e1ee43-4b1e-4a0e-8ee8-0c2205a12898 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:11:22.169760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:22.170880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:22.171746 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 633/1276] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:22.177832 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd0728-dcf0-445f-b306-795b6b364feb None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.260482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:22.287581 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd0728-dcf0-445f-b306-795b6b364feb None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.323953 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24223d-13a9-48a5-a972-a50563d19748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.324656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24223d-13a9-48a5-a972-a50563d19748 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.324939 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24223d-13a9-48a5-a972-a50563d19748 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24223d-13a9-48a5-a972-a50563d19748 None admin] PATH_INFO: `/v3/roles` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.325958 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24223d-13a9-48a5-a972-a50563d19748 None admin] RBAC: Adding query filter params (name=admin) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:22.326267 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24223d-13a9-48a5-a972-a50563d19748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.327656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24223d-13a9-48a5-a972-a50563d19748 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.336280 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 644/1277] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:22.342017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1ee43-4b1e-4a0e-8ee8-0c2205a12898 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.443186 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1ee43-4b1e-4a0e-8ee8-0c2205a12898 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.476512 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e977038-0f97-459d-865a-20f034967c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.477305 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e977038-0f97-459d-865a-20f034967c6b None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.477593 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e977038-0f97-459d-865a-20f034967c6b None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.477890 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e977038-0f97-459d-865a-20f034967c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.489475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e977038-0f97-459d-865a-20f034967c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.490954 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e977038-0f97-459d-865a-20f034967c6b None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-5e977038-0f97-459d-865a-20f034967c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:22.508014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:22.509098 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:22.510024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:22.510024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:22.510024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:22.510024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:22.510024 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 634/1278] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:22.515110 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24223d-13a9-48a5-a972-a50563d19748 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.617138 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24223d-13a9-48a5-a972-a50563d19748 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.650473 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3616566-1176-428f-bfd1-f6bdd7ad8f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.651129 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3616566-1176-428f-bfd1-f6bdd7ad8f14 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.651409 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3616566-1176-428f-bfd1-f6bdd7ad8f14 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.651683 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3616566-1176-428f-bfd1-f6bdd7ad8f14 None admin] PATH_INFO: `/v3/domains` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.652418 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3616566-1176-428f-bfd1-f6bdd7ad8f14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:22.652725 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3616566-1176-428f-bfd1-f6bdd7ad8f14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.654106 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3616566-1176-428f-bfd1-f6bdd7ad8f14 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.666381 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 645/1279] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:22.672270 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e977038-0f97-459d-865a-20f034967c6b None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.709417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:22.778896 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e977038-0f97-459d-865a-20f034967c6b None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.812762 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbc817-fa0b-4cb0-8296-27e1f0ccc958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.813559 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbc817-fa0b-4cb0-8296-27e1f0ccc958 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.813842 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbc817-fa0b-4cb0-8296-27e1f0ccc958 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.814115 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbc817-fa0b-4cb0-8296-27e1f0ccc958 None admin] PATH_INFO: `/v3/users/placement` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.821580 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbc817-fa0b-4cb0-8296-27e1f0ccc958 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.823043 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adbc817-fa0b-4cb0-8296-27e1f0ccc958 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-7adbc817-fa0b-4cb0-8296-27e1f0ccc958 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:11:22.837595 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:22.838652 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:22.839688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:22.839688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:22.839688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:11:22.839688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:11:22.839688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:11:22.839688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:11:22.839688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:11:22.839688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:11:22.839688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:11:22.839688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:22.839688 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 635/1280] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:22.845395 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3616566-1176-428f-bfd1-f6bdd7ad8f14 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:22.849251 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:22.944578 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3616566-1176-428f-bfd1-f6bdd7ad8f14 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:22.979230 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0806c00-e31b-461d-a876-e50ad2bc6488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:22.979405 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0806c00-e31b-461d-a876-e50ad2bc6488 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:22.979597 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0806c00-e31b-461d-a876-e50ad2bc6488 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:22.979871 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0806c00-e31b-461d-a876-e50ad2bc6488 None admin] PATH_INFO: `/v3/users` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:22.980773 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0806c00-e31b-461d-a876-e50ad2bc6488 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:22.981131 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0806c00-e31b-461d-a876-e50ad2bc6488 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:22.982535 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0806c00-e31b-461d-a876-e50ad2bc6488 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:22.999430 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 646/1281] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:11:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:23.005918 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbc817-fa0b-4cb0-8296-27e1f0ccc958 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.099970 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.109758 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adbc817-fa0b-4cb0-8296-27e1f0ccc958 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.143862 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8a105-1a3e-4971-9e00-fc123fe2e6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.144538 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8a105-1a3e-4971-9e00-fc123fe2e6ed None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.144825 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8a105-1a3e-4971-9e00-fc123fe2e6ed None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.145104 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8a105-1a3e-4971-9e00-fc123fe2e6ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.213164 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8a105-1a3e-4971-9e00-fc123fe2e6ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.214670 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8a105-1a3e-4971-9e00-fc123fe2e6ed None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.302452 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.313284 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 636/1282] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:11:23.324456 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0806c00-e31b-461d-a876-e50ad2bc6488 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.386564 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.423503 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0806c00-e31b-461d-a876-e50ad2bc6488 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.456768 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71384fab-ae6b-4f47-ba7b-0f776f1794a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.466608 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71384fab-ae6b-4f47-ba7b-0f776f1794a3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.466608 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71384fab-ae6b-4f47-ba7b-0f776f1794a3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.466608 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71384fab-ae6b-4f47-ba7b-0f776f1794a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.467788 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71384fab-ae6b-4f47-ba7b-0f776f1794a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.469327 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71384fab-ae6b-4f47-ba7b-0f776f1794a3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-71384fab-ae6b-4f47-ba7b-0f776f1794a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:11:23.486841 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:23.487685 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:23.488658 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:11:23.488658 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:11:23.488658 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:11:23.488658 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:23.488658 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 647/1283] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:23.493792 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8a105-1a3e-4971-9e00-fc123fe2e6ed None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.594096 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8a105-1a3e-4971-9e00-fc123fe2e6ed None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.628716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afc123-e9ca-468b-8a0c-f0c0ee0a87da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.629385 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afc123-e9ca-468b-8a0c-f0c0ee0a87da None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.629670 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afc123-e9ca-468b-8a0c-f0c0ee0a87da None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.629948 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afc123-e9ca-468b-8a0c-f0c0ee0a87da None admin] PATH_INFO: `/v3/domains` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.630681 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afc123-e9ca-468b-8a0c-f0c0ee0a87da None admin] RBAC: Adding query filter params (name=Default) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:23.630991 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afc123-e9ca-468b-8a0c-f0c0ee0a87da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.632403 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afc123-e9ca-468b-8a0c-f0c0ee0a87da None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.645111 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 637/1284] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:23.651286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71384fab-ae6b-4f47-ba7b-0f776f1794a3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.750534 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71384fab-ae6b-4f47-ba7b-0f776f1794a3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.764153 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.784072 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51966b6-d85d-4d81-8da1-aec92309a4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.784891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51966b6-d85d-4d81-8da1-aec92309a4c9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.785173 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51966b6-d85d-4d81-8da1-aec92309a4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.785448 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51966b6-d85d-4d81-8da1-aec92309a4c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.791985 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51966b6-d85d-4d81-8da1-aec92309a4c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.793454 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51966b6-d85d-4d81-8da1-aec92309a4c9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application [None req-b51966b6-d85d-4d81-8da1-aec92309a4c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:11:23.807993 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application with Lock( May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:23.809022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:23.821550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:11:23.821550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:11:23.821550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:11:23.821550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:11:23.821550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:11:23.821550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:11:23.821550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:11:23.821550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: ERROR keystone.server.flask.application May 19 20:11:23.821550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 648/1285] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:23.822164 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afc123-e9ca-468b-8a0c-f0c0ee0a87da None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:23.866323 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:23.915454 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afc123-e9ca-468b-8a0c-f0c0ee0a87da None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:23.949446 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404924d-347c-4c2e-bcea-90446b0c58e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:23.950103 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404924d-347c-4c2e-bcea-90446b0c58e5 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:23.950385 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404924d-347c-4c2e-bcea-90446b0c58e5 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:23.950661 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404924d-347c-4c2e-bcea-90446b0c58e5 None admin] PATH_INFO: `/v3/projects` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:23.951562 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404924d-347c-4c2e-bcea-90446b0c58e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:23.951877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404924d-347c-4c2e-bcea-90446b0c58e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:23.953355 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404924d-347c-4c2e-bcea-90446b0c58e5 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:23.965979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 638/1286] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:23.972515 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51966b6-d85d-4d81-8da1-aec92309a4c9 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:24.075817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51966b6-d85d-4d81-8da1-aec92309a4c9 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:24.117053 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844be97-9f5f-46c7-b906-a8b0ccf7dc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:24.117688 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844be97-9f5f-46c7-b906-a8b0ccf7dc15 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:24.117974 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844be97-9f5f-46c7-b906-a8b0ccf7dc15 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:24.118249 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844be97-9f5f-46c7-b906-a8b0ccf7dc15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:24.119075 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844be97-9f5f-46c7-b906-a8b0ccf7dc15 None admin] RBAC: Adding query filter params (role.id=ab7a63cec3a744848972fcbb89ecd0ca, scope.project.id=adcff914910e46c181b074c0f7497994, user.id=34483f3f0a934c50805c31b4af99176d) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:24.119385 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844be97-9f5f-46c7-b906-a8b0ccf7dc15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:24.120859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844be97-9f5f-46c7-b906-a8b0ccf7dc15 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:24.126721 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 649/1287] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:11:23 2022] GET /identity/v3/role_assignments?user.id=34483f3f0a934c50805c31b4af99176d&scope.project.id=adcff914910e46c181b074c0f7497994&role.id=ab7a63cec3a744848972fcbb89ecd0ca => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:26.389833 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767310d5-b175-4306-bc60-0be1914ea17f None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.390472 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767310d5-b175-4306-bc60-0be1914ea17f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.390472 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767310d5-b175-4306-bc60-0be1914ea17f None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.391733 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 639/1288] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:11:26.397210 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b359d-e876-4137-93e3-fd5b9581d8e0 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.397633 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b359d-e876-4137-93e3-fd5b9581d8e0 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.397773 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b359d-e876-4137-93e3-fd5b9581d8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.414243 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-c21b359d-e876-4137-93e3-fd5b9581d8e0 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:26.415421 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-c21b359d-e876-4137-93e3-fd5b9581d8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:26.446032 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:26.485035 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 650/1289] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:11:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:26.507363 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767310d5-b175-4306-bc60-0be1914ea17f None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:26.609017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767310d5-b175-4306-bc60-0be1914ea17f None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:26.620494 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:26.644172 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8df0ae-023f-42cf-be54-a80aebe5c648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:26.644935 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8df0ae-023f-42cf-be54-a80aebe5c648 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.645215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8df0ae-023f-42cf-be54-a80aebe5c648 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.645489 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8df0ae-023f-42cf-be54-a80aebe5c648 None admin] PATH_INFO: `/v3/services/placement` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.646135 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8df0ae-023f-42cf-be54-a80aebe5c648 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:26.647511 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8df0ae-023f-42cf-be54-a80aebe5c648 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-bb8df0ae-023f-42cf-be54-a80aebe5c648 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:26.660457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:26.661659 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:26.662475 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 640/1290] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:11:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:26.668066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b359d-e876-4137-93e3-fd5b9581d8e0 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:26.768709 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b359d-e876-4137-93e3-fd5b9581d8e0 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:26.803800 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbabdf-2ef6-4ab0-8d85-fd648603c915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:26.804505 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbabdf-2ef6-4ab0-8d85-fd648603c915 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.804790 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbabdf-2ef6-4ab0-8d85-fd648603c915 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.805058 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbabdf-2ef6-4ab0-8d85-fd648603c915 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.805769 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbabdf-2ef6-4ab0-8d85-fd648603c915 None admin] RBAC: Adding query filter params (name=placement) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:26.806071 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbabdf-2ef6-4ab0-8d85-fd648603c915 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:26.807449 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbabdf-2ef6-4ab0-8d85-fd648603c915 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:26.812703 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 651/1291] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:11:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:26.818650 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8df0ae-023f-42cf-be54-a80aebe5c648 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:26.920656 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8df0ae-023f-42cf-be54-a80aebe5c648 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:26.954492 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b8a4a-a5a9-46ee-8339-83259739ea19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:26.955157 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b8a4a-a5a9-46ee-8339-83259739ea19 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:26.955447 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b8a4a-a5a9-46ee-8339-83259739ea19 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:26.955724 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b8a4a-a5a9-46ee-8339-83259739ea19 None admin] PATH_INFO: `/v3/services` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:26.956466 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b8a4a-a5a9-46ee-8339-83259739ea19 None admin] RBAC: Adding query filter params (type=placement) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:26.956780 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b8a4a-a5a9-46ee-8339-83259739ea19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:26.958159 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b8a4a-a5a9-46ee-8339-83259739ea19 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:26.962742 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 641/1292] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:11:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:29.195098 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe58ea4-2d8e-44fe-8cd7-7204ba01be9c None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:29.195622 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe58ea4-2d8e-44fe-8cd7-7204ba01be9c None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:29.195728 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe58ea4-2d8e-44fe-8cd7-7204ba01be9c None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:29.197066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 652/1293] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:29.202356 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666892d5-997e-4913-b8a3-edfd8336c86c None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:29.202755 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666892d5-997e-4913-b8a3-edfd8336c86c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:29.203073 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666892d5-997e-4913-b8a3-edfd8336c86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:29.223306 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-666892d5-997e-4913-b8a3-edfd8336c86c None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:29.224611 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-666892d5-997e-4913-b8a3-edfd8336c86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:29.254570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:29.291785 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 642/1294] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:11:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:29.316172 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe58ea4-2d8e-44fe-8cd7-7204ba01be9c None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:29.347538 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:29.427671 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe58ea4-2d8e-44fe-8cd7-7204ba01be9c None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:29.461418 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21884a-290d-4325-a79f-5541fa405189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:29.462080 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21884a-290d-4325-a79f-5541fa405189 None admin] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:29.462363 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21884a-290d-4325-a79f-5541fa405189 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:29.462637 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21884a-290d-4325-a79f-5541fa405189 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:29.463362 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21884a-290d-4325-a79f-5541fa405189 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:29.464773 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21884a-290d-4325-a79f-5541fa405189 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:29.536654 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 653/1295] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:11:29 2022] POST /identity/v3/services => generated 246 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:31.813638 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796b321-5060-4958-af07-d6e9fc2bc487 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.814243 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796b321-5060-4958-af07-d6e9fc2bc487 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.814243 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796b321-5060-4958-af07-d6e9fc2bc487 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.815613 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 643/1296] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:11:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:11:31.820956 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e5746-be40-4d92-8ea3-93c43408dd7b None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:31.821377 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e5746-be40-4d92-8ea3-93c43408dd7b None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:31.821522 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e5746-be40-4d92-8ea3-93c43408dd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:31.838165 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-fe1e5746-be40-4d92-8ea3-93c43408dd7b None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:31.843065 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-fe1e5746-be40-4d92-8ea3-93c43408dd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:31.912180 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 654/1297] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:11:31 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:31.935994 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796b321-5060-4958-af07-d6e9fc2bc487 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:32.037597 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796b321-5060-4958-af07-d6e9fc2bc487 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:32.074440 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7abb6-0f0e-452f-a855-4bf05a1ac019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:32.074440 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7abb6-0f0e-452f-a855-4bf05a1ac019 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:32.074440 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7abb6-0f0e-452f-a855-4bf05a1ac019 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:32.074440 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7abb6-0f0e-452f-a855-4bf05a1ac019 None admin] PATH_INFO: `/v3/services/placement` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:32.074877 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7abb6-0f0e-452f-a855-4bf05a1ac019 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:32.075625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7abb6-0f0e-452f-a855-4bf05a1ac019 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-09c7abb6-0f0e-452f-a855-4bf05a1ac019 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:32.088052 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:32.089257 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:32.090261 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 644/1298] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:11:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:32.096258 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e5746-be40-4d92-8ea3-93c43408dd7b None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:32.115277 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:32.198759 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e5746-be40-4d92-8ea3-93c43408dd7b None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:32.233208 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3928f-e61f-4713-a4e1-b0746e2285b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:32.233891 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3928f-e61f-4713-a4e1-b0746e2285b3 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:32.234208 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3928f-e61f-4713-a4e1-b0746e2285b3 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:32.234482 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a3928f-e61f-4713-a4e1-b0746e2285b3 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:32.235180 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3928f-e61f-4713-a4e1-b0746e2285b3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:32.235484 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3928f-e61f-4713-a4e1-b0746e2285b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:32.236896 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a3928f-e61f-4713-a4e1-b0746e2285b3 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:32.243220 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 655/1299] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:11:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:32.249432 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7abb6-0f0e-452f-a855-4bf05a1ac019 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:32.282558 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:32.355088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7abb6-0f0e-452f-a855-4bf05a1ac019 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:32.390133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1b6fb-93f0-48c9-b798-93ecb4ccb3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:32.390896 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b6fb-93f0-48c9-b798-93ecb4ccb3d8 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:32.391275 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b6fb-93f0-48c9-b798-93ecb4ccb3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:32.391744 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1b6fb-93f0-48c9-b798-93ecb4ccb3d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:32.392678 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1b6fb-93f0-48c9-b798-93ecb4ccb3d8 None admin] RBAC: Adding query filter params (interface=public, service_id=51febc04f3724daebf9436bf43e24860, region_id=RegionOne) {{(pid=144624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:32.393090 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1b6fb-93f0-48c9-b798-93ecb4ccb3d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:32.394582 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1b6fb-93f0-48c9-b798-93ecb4ccb3d8 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:32.399918 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 645/1300] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:11:32 2022] GET /identity/v3/endpoints?service_id=51febc04f3724daebf9436bf43e24860&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:11:32.405565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3928f-e61f-4713-a4e1-b0746e2285b3 None admin] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:32.490262 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:32.508303 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a3928f-e61f-4713-a4e1-b0746e2285b3 None admin] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:32.542814 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee19e91-f6d9-4111-89f3-6854b3bbfc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:32.543468 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee19e91-f6d9-4111-89f3-6854b3bbfc47 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:32.543749 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee19e91-f6d9-4111-89f3-6854b3bbfc47 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:32.544024 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee19e91-f6d9-4111-89f3-6854b3bbfc47 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:32.544731 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee19e91-f6d9-4111-89f3-6854b3bbfc47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:32.546279 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee19e91-f6d9-4111-89f3-6854b3bbfc47 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:32.554093 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 656/1301] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:11:32 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:11:34.789277 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02dae9-8e5f-4b30-b743-8f358083ea23 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.789818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02dae9-8e5f-4b30-b743-8f358083ea23 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.789818 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02dae9-8e5f-4b30-b743-8f358083ea23 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.791212 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 646/1302] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:11:34.797123 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e330ad-a49b-455c-ab8e-9c579c65192e None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:34.797572 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e330ad-a49b-455c-ab8e-9c579c65192e None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:34.797838 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e330ad-a49b-455c-ab8e-9c579c65192e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:34.815438 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-e8e330ad-a49b-455c-ab8e-9c579c65192e None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:11:34.816650 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-e8e330ad-a49b-455c-ab8e-9c579c65192e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:11:34.884088 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 657/1303] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:11:34 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:11:34.907045 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02dae9-8e5f-4b30-b743-8f358083ea23 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:35.010981 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:35.017792 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02dae9-8e5f-4b30-b743-8f358083ea23 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:35.052841 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83125e89-310f-4dea-98ba-0d73ca7d0a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:35.053561 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83125e89-310f-4dea-98ba-0d73ca7d0a14 None admin] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:35.053843 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83125e89-310f-4dea-98ba-0d73ca7d0a14 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:35.054120 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83125e89-310f-4dea-98ba-0d73ca7d0a14 None admin] PATH_INFO: `/v3/services/placement` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:35.054786 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83125e89-310f-4dea-98ba-0d73ca7d0a14 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:35.056163 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83125e89-310f-4dea-98ba-0d73ca7d0a14 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application [None req-83125e89-310f-4dea-98ba-0d73ca7d0a14 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:11:35.068195 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:11:35.069399 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application with Lock( May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self._enter() May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application return self.creator() May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application created_value = creator( May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: ERROR keystone.server.flask.application May 19 20:11:35.070571 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 647/1304] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:11:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:11:35.076550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e330ad-a49b-455c-ab8e-9c579c65192e None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:35.157893 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:35.182509 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e330ad-a49b-455c-ab8e-9c579c65192e None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:35.216911 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad31415-5b06-4eda-819d-370ab267a9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:35.217583 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad31415-5b06-4eda-819d-370ab267a9c9 None admin] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:35.217865 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad31415-5b06-4eda-819d-370ab267a9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:35.218136 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad31415-5b06-4eda-819d-370ab267a9c9 None admin] PATH_INFO: `/v3/services` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:35.219350 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad31415-5b06-4eda-819d-370ab267a9c9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=144623) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:11:35.219656 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad31415-5b06-4eda-819d-370ab267a9c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:35.221096 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad31415-5b06-4eda-819d-370ab267a9c9 None admin] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:35.226734 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 658/1305] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:11:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:11:35.232902 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83125e89-310f-4dea-98ba-0d73ca7d0a14 None admin] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:11:35.341440 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83125e89-310f-4dea-98ba-0d73ca7d0a14 None admin] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:11:35.379185 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6293eb9-45d8-42d2-8259-e5df6f2ec369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '48915a5b6f6e4fd8a53e478480a2c94c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:11:35.379965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6293eb9-45d8-42d2-8259-e5df6f2ec369 None admin] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:11:35.380247 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6293eb9-45d8-42d2-8259-e5df6f2ec369 None admin] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:11:35.380549 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6293eb9-45d8-42d2-8259-e5df6f2ec369 None admin] PATH_INFO: `/v3/endpoints` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:11:35.381294 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6293eb9-45d8-42d2-8259-e5df6f2ec369 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:11:35.382821 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6293eb9-45d8-42d2-8259-e5df6f2ec369 None admin] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:11:35.496760 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 648/1306] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:11:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:11:45.223326 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=30 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:11:45.491804 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:04.947727 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b9945-86e6-4acf-87d8-3c291ed06cf7 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.949559 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b9945-86e6-4acf-87d8-3c291ed06cf7 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.950028 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b9945-86e6-4acf-87d8-3c291ed06cf7 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:04.951694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 659/1307] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:04.957924 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c8ea-a2ec-422f-89c1-386d94bef893 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:04.958506 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c8ea-a2ec-422f-89c1-386d94bef893 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:04.958506 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046c8ea-a2ec-422f-89c1-386d94bef893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:05.145799 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-d046c8ea-a2ec-422f-89c1-386d94bef893 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:05.147033 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-d046c8ea-a2ec-422f-89c1-386d94bef893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:05.388265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 649/1308] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:05.459563 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b446d-8825-4ec8-861d-e15cf694a45b None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:05.460096 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b446d-8825-4ec8-861d-e15cf694a45b None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:05.460422 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b446d-8825-4ec8-861d-e15cf694a45b None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:05.462115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 660/1309] 10.222.0.88 () {54 vars in 964 bytes} [Thu May 19 20:19:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:05.469653 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da349a4-d3aa-4b8d-9293-17698fa9dd8f None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:05.470179 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da349a4-d3aa-4b8d-9293-17698fa9dd8f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:05.470618 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da349a4-d3aa-4b8d-9293-17698fa9dd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:05.646005 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-5da349a4-d3aa-4b8d-9293-17698fa9dd8f None None] MFA Rules not processed for user `d7917b3c79bd42cea17ec8eba5a265ed`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:05.647705 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-5da349a4-d3aa-4b8d-9293-17698fa9dd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:05.749017 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:05.938066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b446d-8825-4ec8-861d-e15cf694a45b None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:05.938880 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 650/1310] 10.222.0.88 () {58 vars in 1061 bytes} [Thu May 19 20:19:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:19:05.939947 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-364b446d-8825-4ec8-861d-e15cf694a45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:06.177180 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b446d-8825-4ec8-861d-e15cf694a45b None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:06.345330 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66cb26-1a51-478f-93c0-14042c009702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:06.346191 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66cb26-1a51-478f-93c0-14042c009702 service neutron] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:06.346485 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66cb26-1a51-478f-93c0-14042c009702 service neutron] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:06.346768 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66cb26-1a51-478f-93c0-14042c009702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:06.348540 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-5e66cb26-1a51-478f-93c0-14042c009702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:06.455520 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:06.513475 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66cb26-1a51-478f-93c0-14042c009702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:06.515516 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66cb26-1a51-478f-93c0-14042c009702 service neutron] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:06.535609 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 661/1311] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:09.262009 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a013d-3c54-4698-b188-238a501ccc5c None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:09.262661 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a013d-3c54-4698-b188-238a501ccc5c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:09.262661 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a013d-3c54-4698-b188-238a501ccc5c None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:09.264402 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 651/1312] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:09.271422 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabfe78-af89-486d-b4fc-129de5674768 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:09.272612 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabfe78-af89-486d-b4fc-129de5674768 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:09.273097 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabfe78-af89-486d-b4fc-129de5674768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:09.353487 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-6dabfe78-af89-486d-b4fc-129de5674768 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:09.354988 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-6dabfe78-af89-486d-b4fc-129de5674768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:09.462754 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 662/1313] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:09.502014 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a013d-3c54-4698-b188-238a501ccc5c None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:09.503753 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-947a013d-3c54-4698-b188-238a501ccc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:09.617532 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a013d-3c54-4698-b188-238a501ccc5c None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:09.704750 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:09.707909 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6875b7-6281-4a41-901a-ae2c0a8b0582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:09.708904 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6875b7-6281-4a41-901a-ae2c0a8b0582 service neutron] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:09.709191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6875b7-6281-4a41-901a-ae2c0a8b0582 service neutron] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:09.709510 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6875b7-6281-4a41-901a-ae2c0a8b0582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:09.711408 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-ab6875b7-6281-4a41-901a-ae2c0a8b0582 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:09.842777 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6875b7-6281-4a41-901a-ae2c0a8b0582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:09.844265 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6875b7-6281-4a41-901a-ae2c0a8b0582 service neutron] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:09.867860 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 652/1314] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:12.630244 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a77728-26a4-4151-88ee-9abf73235b4d None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:12.630781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a77728-26a4-4151-88ee-9abf73235b4d None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:12.630781 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a77728-26a4-4151-88ee-9abf73235b4d None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:12.632176 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 663/1315] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:12.637474 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10218497-0a35-4de0-b9db-aba289363f80 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:12.637920 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10218497-0a35-4de0-b9db-aba289363f80 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:12.638040 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10218497-0a35-4de0-b9db-aba289363f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:12.706276 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-10218497-0a35-4de0-b9db-aba289363f80 None None] MFA Rules not processed for user `e92111952a254ff58d562324db8f63fb`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:12.707522 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-10218497-0a35-4de0-b9db-aba289363f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:12.772620 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:12.820471 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 653/1316] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:12.873216 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2b7e0-e238-429d-bafb-abfc5207572f None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:12.873508 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2b7e0-e238-429d-bafb-abfc5207572f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:12.873792 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2b7e0-e238-429d-bafb-abfc5207572f None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:12.875181 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 664/1317] 10.222.0.88 () {54 vars in 964 bytes} [Thu May 19 20:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:12.881250 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b137829-761e-4dc5-a4a2-537eccd24d8f None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:12.881505 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b137829-761e-4dc5-a4a2-537eccd24d8f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:12.881787 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b137829-761e-4dc5-a4a2-537eccd24d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:12.960619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-3b137829-761e-4dc5-a4a2-537eccd24d8f None None] MFA Rules not processed for user `d7917b3c79bd42cea17ec8eba5a265ed`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:12.961845 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-3b137829-761e-4dc5-a4a2-537eccd24d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:13.038495 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 654/1318] 10.222.0.88 () {58 vars in 1061 bytes} [Thu May 19 20:19:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:13.047840 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2b7e0-e238-429d-bafb-abfc5207572f None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:13.049677 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-81f2b7e0-e238-429d-bafb-abfc5207572f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:13.069081 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:13.164166 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2b7e0-e238-429d-bafb-abfc5207572f None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:13.236648 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764d09a-98ba-4205-b8b7-4df044b69545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:13.237295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d09a-98ba-4205-b8b7-4df044b69545 service neutron] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:13.237576 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d09a-98ba-4205-b8b7-4df044b69545 service neutron] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:13.237855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764d09a-98ba-4205-b8b7-4df044b69545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:13.239507 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-4764d09a-98ba-4205-b8b7-4df044b69545 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:13.301548 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:13.348664 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764d09a-98ba-4205-b8b7-4df044b69545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:13.350149 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764d09a-98ba-4205-b8b7-4df044b69545 service neutron] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:13.370136 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 665/1319] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:16.782979 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905612fa-6808-4168-aab8-9d7d7f4ca7e0 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:16.783518 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905612fa-6808-4168-aab8-9d7d7f4ca7e0 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:16.783518 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905612fa-6808-4168-aab8-9d7d7f4ca7e0 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:16.784946 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 655/1320] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:16.790390 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a16f74-0e35-40b8-8cc7-6db5c7f06334 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:16.790808 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a16f74-0e35-40b8-8cc7-6db5c7f06334 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:16.790942 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a16f74-0e35-40b8-8cc7-6db5c7f06334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:16.856517 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-77a16f74-0e35-40b8-8cc7-6db5c7f06334 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:16.858222 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-77a16f74-0e35-40b8-8cc7-6db5c7f06334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:16.954865 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 666/1321] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:16.993100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905612fa-6808-4168-aab8-9d7d7f4ca7e0 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:16.994626 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-905612fa-6808-4168-aab8-9d7d7f4ca7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:17.045031 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:17.099400 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905612fa-6808-4168-aab8-9d7d7f4ca7e0 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:17.173379 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ca764-4c10-4212-8bfa-281f54111367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:17.174294 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca764-4c10-4212-8bfa-281f54111367 service neutron] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:17.174570 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca764-4c10-4212-8bfa-281f54111367 service neutron] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:17.174840 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ca764-4c10-4212-8bfa-281f54111367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:17.176516 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-7d7ca764-4c10-4212-8bfa-281f54111367 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:17.273417 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:17.288152 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ca764-4c10-4212-8bfa-281f54111367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:17.289619 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ca764-4c10-4212-8bfa-281f54111367 service neutron] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:17.308848 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 656/1322] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 20:19:20.645997 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68660d5-27dc-4143-816d-dc267fd4938a None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:20.646563 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68660d5-27dc-4143-816d-dc267fd4938a None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:20.646563 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68660d5-27dc-4143-816d-dc267fd4938a None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:20.647944 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 667/1323] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:20.653312 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9642be-7cf3-48ec-b27e-d48791e43424 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:20.653714 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9642be-7cf3-48ec-b27e-d48791e43424 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:20.654477 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9642be-7cf3-48ec-b27e-d48791e43424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:20.721197 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-4d9642be-7cf3-48ec-b27e-d48791e43424 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:20.722435 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-4d9642be-7cf3-48ec-b27e-d48791e43424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:20.840048 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 657/1324] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:20.880078 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68660d5-27dc-4143-816d-dc267fd4938a None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:20.881708 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-a68660d5-27dc-4143-816d-dc267fd4938a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:20.907042 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:20.993548 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68660d5-27dc-4143-816d-dc267fd4938a None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:21.070369 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b739d71-9059-427e-a9e9-f326292b4d03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:21.071020 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b739d71-9059-427e-a9e9-f326292b4d03 service neutron] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:21.071307 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b739d71-9059-427e-a9e9-f326292b4d03 service neutron] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:21.071587 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b739d71-9059-427e-a9e9-f326292b4d03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:21.073276 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-8b739d71-9059-427e-a9e9-f326292b4d03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:21.145817 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:21.200986 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b739d71-9059-427e-a9e9-f326292b4d03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:21.202356 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b739d71-9059-427e-a9e9-f326292b4d03 service neutron] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:21.221284 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 668/1325] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:23.572944 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d531393-62af-4718-9db7-0227eba07da0 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:23.573533 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d531393-62af-4718-9db7-0227eba07da0 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:23.573533 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d531393-62af-4718-9db7-0227eba07da0 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:23.574884 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 658/1326] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:23.580319 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f7b2b-5132-4851-ae6a-6099cc582e90 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:23.580782 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f7b2b-5132-4851-ae6a-6099cc582e90 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:23.580931 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5f7b2b-5132-4851-ae6a-6099cc582e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:23.646694 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-2d5f7b2b-5132-4851-ae6a-6099cc582e90 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:23.647837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-2d5f7b2b-5132-4851-ae6a-6099cc582e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:23.752489 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 669/1327] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:23.788910 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d531393-62af-4718-9db7-0227eba07da0 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:23.790569 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-9d531393-62af-4718-9db7-0227eba07da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:23.857584 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:23.904837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d531393-62af-4718-9db7-0227eba07da0 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:23.987536 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e37a2-7abd-4440-8b39-233adacd8412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:23.988193 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e37a2-7abd-4440-8b39-233adacd8412 service neutron] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:23.988506 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e37a2-7abd-4440-8b39-233adacd8412 service neutron] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:23.988786 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e37a2-7abd-4440-8b39-233adacd8412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:23.990674 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-424e37a2-7abd-4440-8b39-233adacd8412 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:24.117079 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:24.119999 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e37a2-7abd-4440-8b39-233adacd8412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:24.121533 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e37a2-7abd-4440-8b39-233adacd8412 service neutron] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:24.148033 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 659/1328] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:26.449546 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0f5f3-ae26-4e73-ae37-b0ad7877e22f None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:26.450111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0f5f3-ae26-4e73-ae37-b0ad7877e22f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:26.450111 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0f5f3-ae26-4e73-ae37-b0ad7877e22f None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:26.451445 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 670/1329] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:26.456914 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469160-e237-4604-b3e2-525b793a274f None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:26.457445 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469160-e237-4604-b3e2-525b793a274f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:26.457547 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae469160-e237-4604-b3e2-525b793a274f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:26.523448 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-ae469160-e237-4604-b3e2-525b793a274f None None] MFA Rules not processed for user `e92111952a254ff58d562324db8f63fb`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:26.524696 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-ae469160-e237-4604-b3e2-525b793a274f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:26.621006 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 660/1330] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:26.658013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0f5f3-ae26-4e73-ae37-b0ad7877e22f None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:26.659565 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-36f0f5f3-ae26-4e73-ae37-b0ad7877e22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:26.691285 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:26.763105 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0f5f3-ae26-4e73-ae37-b0ad7877e22f None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:26.835854 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aa168-4b95-473c-b82f-a5e59efb13db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:26.836527 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa168-4b95-473c-b82f-a5e59efb13db service neutron] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:26.836814 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa168-4b95-473c-b82f-a5e59efb13db service neutron] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:26.837092 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa168-4b95-473c-b82f-a5e59efb13db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:26.838739 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-892aa168-4b95-473c-b82f-a5e59efb13db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:26.922903 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:26.954764 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892aa168-4b95-473c-b82f-a5e59efb13db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:26.956228 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892aa168-4b95-473c-b82f-a5e59efb13db service neutron] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:26.974570 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 671/1331] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:26 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:29.652133 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55a60f-1916-4d71-a77f-778cef4f35ef None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:29.652732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55a60f-1916-4d71-a77f-778cef4f35ef None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:29.652732 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55a60f-1916-4d71-a77f-778cef4f35ef None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:29.654105 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 661/1332] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:29.659338 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551311-df37-42fb-b847-0284ddab99d5 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:29.659782 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551311-df37-42fb-b847-0284ddab99d5 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:29.659884 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551311-df37-42fb-b847-0284ddab99d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:29.724607 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-10551311-df37-42fb-b847-0284ddab99d5 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:29.725823 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-10551311-df37-42fb-b847-0284ddab99d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:29.824570 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 672/1333] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:29.860560 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55a60f-1916-4d71-a77f-778cef4f35ef None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:29.862120 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-fc55a60f-1916-4d71-a77f-778cef4f35ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:29.931215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:29.965201 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55a60f-1916-4d71-a77f-778cef4f35ef None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:30.039624 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc064d-d2dc-4cd4-9cac-6f38b5f74c9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:30.040293 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc064d-d2dc-4cd4-9cac-6f38b5f74c9b service neutron] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:30.040609 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc064d-d2dc-4cd4-9cac-6f38b5f74c9b service neutron] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:30.040884 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc064d-d2dc-4cd4-9cac-6f38b5f74c9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:30.042658 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-65bc064d-d2dc-4cd4-9cac-6f38b5f74c9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:30.158556 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc064d-d2dc-4cd4-9cac-6f38b5f74c9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:30.160003 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc064d-d2dc-4cd4-9cac-6f38b5f74c9b service neutron] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:30.170088 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:30.178466 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 662/1334] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:32.512402 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a9db5-10bb-4911-bcd8-100cf54fc563 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:32.513443 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a9db5-10bb-4911-bcd8-100cf54fc563 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:32.513443 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a9db5-10bb-4911-bcd8-100cf54fc563 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:32.514806 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 673/1335] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:32.520353 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a03091-a443-493e-bab0-e960cc1bfc87 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:32.520830 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a03091-a443-493e-bab0-e960cc1bfc87 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:32.521021 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a03091-a443-493e-bab0-e960cc1bfc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:32.587451 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-d3a03091-a443-493e-bab0-e960cc1bfc87 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:32.588722 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-d3a03091-a443-493e-bab0-e960cc1bfc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:32.691850 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 663/1336] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:32.731970 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a9db5-10bb-4911-bcd8-100cf54fc563 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:32.733626 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-2b2a9db5-10bb-4911-bcd8-100cf54fc563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:32.780632 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:32.848079 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a9db5-10bb-4911-bcd8-100cf54fc563 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:32.924507 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dbaa6-ee3d-4ede-b3c6-cf53898b375c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:32.925233 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dbaa6-ee3d-4ede-b3c6-cf53898b375c service neutron] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:32.925530 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dbaa6-ee3d-4ede-b3c6-cf53898b375c service neutron] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:32.925846 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dbaa6-ee3d-4ede-b3c6-cf53898b375c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:32.927566 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-fb6dbaa6-ee3d-4ede-b3c6-cf53898b375c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:33.014619 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:33.041189 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dbaa6-ee3d-4ede-b3c6-cf53898b375c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:33.042602 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dbaa6-ee3d-4ede-b3c6-cf53898b375c service neutron] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:33.062467 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 674/1337] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:35.945063 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ee6ca-c8e4-4e79-b921-70cdb3cc78e4 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:35.945660 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ee6ca-c8e4-4e79-b921-70cdb3cc78e4 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:35.945660 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ee6ca-c8e4-4e79-b921-70cdb3cc78e4 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:35.947066 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 664/1338] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:35.952558 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db946f2b-94a5-404f-bce9-fd7d811387e6 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:35.952976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db946f2b-94a5-404f-bce9-fd7d811387e6 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:35.957426 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db946f2b-94a5-404f-bce9-fd7d811387e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:36.028869 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-db946f2b-94a5-404f-bce9-fd7d811387e6 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:36.030100 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-db946f2b-94a5-404f-bce9-fd7d811387e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:36.136187 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 675/1339] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:36.172823 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ee6ca-c8e4-4e79-b921-70cdb3cc78e4 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:36.175414 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-061ee6ca-c8e4-4e79-b921-70cdb3cc78e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:36.255574 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=100 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:36.271590 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ee6ca-c8e4-4e79-b921-70cdb3cc78e4 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:36.275191 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3be4f-1416-4b97-865c-18853bb93fef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:36.275841 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3be4f-1416-4b97-865c-18853bb93fef service neutron] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:36.276121 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3be4f-1416-4b97-865c-18853bb93fef service neutron] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:36.276457 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3be4f-1416-4b97-865c-18853bb93fef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:36.286337 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3be4f-1416-4b97-865c-18853bb93fef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:36.287893 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3be4f-1416-4b97-865c-18853bb93fef service neutron] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:36.307201 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 665/1340] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:40.863813 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a300-6709-4b4c-a00e-4a2c3de6a6e4 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:40.864378 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a300-6709-4b4c-a00e-4a2c3de6a6e4 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:40.864463 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9a300-6709-4b4c-a00e-4a2c3de6a6e4 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:40.866550 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 676/1341] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:40.873366 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b5b47-2dbc-41fc-9f89-760b1fd24c0a None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:40.874303 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b5b47-2dbc-41fc-9f89-760b1fd24c0a None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:40.875016 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b5b47-2dbc-41fc-9f89-760b1fd24c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:40.908275 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-c10b5b47-2dbc-41fc-9f89-760b1fd24c0a None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:40.910678 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-c10b5b47-2dbc-41fc-9f89-760b1fd24c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:40.992897 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 666/1342] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:41.031979 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9a300-6709-4b4c-a00e-4a2c3de6a6e4 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:41.043231 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9a300-6709-4b4c-a00e-4a2c3de6a6e4 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:41.047268 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e84525-722c-4bf5-ad48-27ebdc28a7cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:41.047936 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e84525-722c-4bf5-ad48-27ebdc28a7cf service neutron] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:41.048218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e84525-722c-4bf5-ad48-27ebdc28a7cf service neutron] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:41.048779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e84525-722c-4bf5-ad48-27ebdc28a7cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:41.057923 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e84525-722c-4bf5-ad48-27ebdc28a7cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:41.059389 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e84525-722c-4bf5-ad48-27ebdc28a7cf service neutron] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:41.068868 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 677/1343] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:44.268837 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da964968-45ee-4d66-8819-5ef9120c4a85 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:44.269349 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da964968-45ee-4d66-8819-5ef9120c4a85 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:44.269349 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da964968-45ee-4d66-8819-5ef9120c4a85 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:44.270728 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 667/1344] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:44.276113 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebea23-8949-4f8c-a3f5-1d14f06c57db None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:44.276567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebea23-8949-4f8c-a3f5-1d14f06c57db None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:44.276713 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebea23-8949-4f8c-a3f5-1d14f06c57db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:44.294749 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-17ebea23-8949-4f8c-a3f5-1d14f06c57db None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:44.295883 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-17ebea23-8949-4f8c-a3f5-1d14f06c57db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:44.303922 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 678/1345] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:44.340743 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da964968-45ee-4d66-8819-5ef9120c4a85 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:44.351661 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da964968-45ee-4d66-8819-5ef9120c4a85 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:44.355192 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2501dbd-e34a-4e17-8dc1-23c9d5c1853f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:44.355840 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2501dbd-e34a-4e17-8dc1-23c9d5c1853f service neutron] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:44.356691 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2501dbd-e34a-4e17-8dc1-23c9d5c1853f service neutron] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:44.356978 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2501dbd-e34a-4e17-8dc1-23c9d5c1853f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:44.366137 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2501dbd-e34a-4e17-8dc1-23c9d5c1853f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:44.367577 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2501dbd-e34a-4e17-8dc1-23c9d5c1853f service neutron] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:44.377671 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 668/1346] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:48.896782 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a172d9a-5407-4c27-a645-5c67df9b5316 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:48.897335 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a172d9a-5407-4c27-a645-5c67df9b5316 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:48.897335 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a172d9a-5407-4c27-a645-5c67df9b5316 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:48.898673 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 679/1347] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:48.904043 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff8c9b-d894-4b7a-8850-f70124f55424 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:48.904585 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff8c9b-d894-4b7a-8850-f70124f55424 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:48.904678 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff8c9b-d894-4b7a-8850-f70124f55424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:48.922904 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-28ff8c9b-d894-4b7a-8850-f70124f55424 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:48.924380 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-28ff8c9b-d894-4b7a-8850-f70124f55424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:48.932686 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 669/1348] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:48.971179 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a172d9a-5407-4c27-a645-5c67df9b5316 None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:19:48.982818 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a172d9a-5407-4c27-a645-5c67df9b5316 None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:19:48.986447 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f03dd-ff02-46ea-942e-74690ea83679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd7917b3c79bd42cea17ec8eba5a265ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:19:48.987087 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f03dd-ff02-46ea-942e-74690ea83679 service neutron] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:48.987374 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f03dd-ff02-46ea-942e-74690ea83679 service neutron] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:48.987652 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f03dd-ff02-46ea-942e-74690ea83679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:48.997476 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f03dd-ff02-46ea-942e-74690ea83679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:19:48.998859 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f03dd-ff02-46ea-942e-74690ea83679 service neutron] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:19:49.008561 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 680/1349] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:19:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:19:54.503117 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c521f-d485-4b76-bf53-29eb32f59c85 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:54.503685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c521f-d485-4b76-bf53-29eb32f59c85 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:54.503685 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c521f-d485-4b76-bf53-29eb32f59c85 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:54.505427 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 670/1350] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 20:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:19:54.510438 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7721d-0675-4e4f-8694-e1a867ba4313 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:54.510906 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7721d-0675-4e4f-8694-e1a867ba4313 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:54.511218 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7721d-0675-4e4f-8694-e1a867ba4313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:54.527916 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:54.597327 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-a6b7721d-0675-4e4f-8694-e1a867ba4313 None None] MFA Rules not processed for user `34483f3f0a934c50805c31b4af99176d`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:54.598766 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-a6b7721d-0675-4e4f-8694-e1a867ba4313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:54.682875 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 681/1351] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 20:19:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:19:58.917010 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=61 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:19:59.265190 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191a635-892a-469d-8c1c-10f334963dac None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:59.265190 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191a635-892a-469d-8c1c-10f334963dac None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:59.265190 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191a635-892a-469d-8c1c-10f334963dac None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:59.265190 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 671/1352] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 20:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:19:59.266702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b46554-291d-4861-b9fe-7eafbacc3318 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:19:59.266702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b46554-291d-4861-b9fe-7eafbacc3318 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:19:59.266702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b46554-291d-4861-b9fe-7eafbacc3318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:19:59.266702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-f7b46554-291d-4861-b9fe-7eafbacc3318 None None] MFA Rules not processed for user `34483f3f0a934c50805c31b4af99176d`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:19:59.266702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-f7b46554-291d-4861-b9fe-7eafbacc3318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:19:59.266702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 682/1353] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 20:19:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:20:00.658780 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2924629-2610-49e3-a680-f38082e1ac0c None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:00.659557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2924629-2610-49e3-a680-f38082e1ac0c None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:00.659557 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2924629-2610-49e3-a680-f38082e1ac0c None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:00.660766 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 672/1354] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 20:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:20:00.666207 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ff697-a395-457c-8fb5-c0e892f0d22f None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:00.666702 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ff697-a395-457c-8fb5-c0e892f0d22f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:00.666803 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ff697-a395-457c-8fb5-c0e892f0d22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:00.687073 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-5a2ff697-a395-457c-8fb5-c0e892f0d22f None None] MFA Rules not processed for user `34483f3f0a934c50805c31b4af99176d`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:00.688308 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-5a2ff697-a395-457c-8fb5-c0e892f0d22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:00.698966 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 683/1355] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 20:20:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:20:04.869725 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601f7c1-89d3-4662-b70d-7235969a42d1 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:04.870271 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601f7c1-89d3-4662-b70d-7235969a42d1 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:04.870271 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601f7c1-89d3-4662-b70d-7235969a42d1 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:04.872272 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 673/1356] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:20:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:04.877976 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3412de0-3ce8-427f-9e72-584974adc30b None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:04.880979 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3412de0-3ce8-427f-9e72-584974adc30b None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:04.880979 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3412de0-3ce8-427f-9e72-584974adc30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:04.897368 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-f3412de0-3ce8-427f-9e72-584974adc30b None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:04.899115 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-f3412de0-3ce8-427f-9e72-584974adc30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:04.909848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 684/1357] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:20:04.971086 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebb1b3-b41b-4de2-8370-bb9231181ca8 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:04.972704 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebb1b3-b41b-4de2-8370-bb9231181ca8 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:04.972704 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebb1b3-b41b-4de2-8370-bb9231181ca8 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:04.974490 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 674/1358] 10.222.0.88 () {54 vars in 961 bytes} [Thu May 19 20:20:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:04.983590 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd735a8-c778-427c-8dc1-904b7a2b7034 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:04.984855 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd735a8-c778-427c-8dc1-904b7a2b7034 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:04.985829 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd735a8-c778-427c-8dc1-904b7a2b7034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:05.039652 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-1cd735a8-c778-427c-8dc1-904b7a2b7034 None None] MFA Rules not processed for user `f6d44c613f524fe1804d6c9b65fe97ca`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:05.040990 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-1cd735a8-c778-427c-8dc1-904b7a2b7034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:05.117718 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=100 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:05.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 685/1359] 10.222.0.88 () {58 vars in 1058 bytes} [Thu May 19 20:20:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:20:05.152716 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebb1b3-b41b-4de2-8370-bb9231181ca8 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:05.168567 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebb1b3-b41b-4de2-8370-bb9231181ca8 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:05.175308 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec11e45-72b4-4c62-912d-37578b24a8e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d44c613f524fe1804d6c9b65fe97ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:05.176100 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11e45-72b4-4c62-912d-37578b24a8e5 service nova] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:05.176526 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11e45-72b4-4c62-912d-37578b24a8e5 service nova] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:05.176911 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11e45-72b4-4c62-912d-37578b24a8e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:05.187908 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec11e45-72b4-4c62-912d-37578b24a8e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:05.189679 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec11e45-72b4-4c62-912d-37578b24a8e5 service nova] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:05.200657 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 675/1360] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:20:07.659848 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca4f3a-9ab9-46a5-8ed2-504bcc05af9f None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:07.660418 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca4f3a-9ab9-46a5-8ed2-504bcc05af9f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:07.660418 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca4f3a-9ab9-46a5-8ed2-504bcc05af9f None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:07.661837 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 686/1361] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:07.667333 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906bc08-6500-4752-a21e-cb6856c29f47 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:07.667782 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906bc08-6500-4752-a21e-cb6856c29f47 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:07.667894 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906bc08-6500-4752-a21e-cb6856c29f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:07.686501 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-b906bc08-6500-4752-a21e-cb6856c29f47 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:07.687712 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-b906bc08-6500-4752-a21e-cb6856c29f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:07.695805 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 676/1362] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:20:07.740067 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca4f3a-9ab9-46a5-8ed2-504bcc05af9f None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:07.755245 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca4f3a-9ab9-46a5-8ed2-504bcc05af9f None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:07.760696 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc18ce9-44e9-45c7-a941-f982b2aefa4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d44c613f524fe1804d6c9b65fe97ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:07.761877 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc18ce9-44e9-45c7-a941-f982b2aefa4b service nova] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:07.762213 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc18ce9-44e9-45c7-a941-f982b2aefa4b service nova] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:07.765759 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc18ce9-44e9-45c7-a941-f982b2aefa4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:07.775107 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc18ce9-44e9-45c7-a941-f982b2aefa4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:07.777379 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc18ce9-44e9-45c7-a941-f982b2aefa4b service nova] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:07.788457 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 687/1363] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:20:10.486122 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb0fb5-2ee1-45a3-be77-6669a54a1835 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:10.486625 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb0fb5-2ee1-45a3-be77-6669a54a1835 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:10.486725 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb0fb5-2ee1-45a3-be77-6669a54a1835 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:10.488132 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 677/1364] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:10.496744 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050f31b-97f9-47ba-8233-f04fae8ad546 None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:10.496744 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050f31b-97f9-47ba-8233-f04fae8ad546 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:10.496744 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050f31b-97f9-47ba-8233-f04fae8ad546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:10.515701 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-8050f31b-97f9-47ba-8233-f04fae8ad546 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:10.517317 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-8050f31b-97f9-47ba-8233-f04fae8ad546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:10.525998 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 688/1365] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:20:10.574413 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7202e-2903-497f-a36e-e7292d54c299 None None] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:10.574735 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7202e-2903-497f-a36e-e7292d54c299 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:10.574995 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7202e-2903-497f-a36e-e7292d54c299 None None] PATH_INFO: `/` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:10.577374 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 678/1366] 10.222.0.88 () {54 vars in 961 bytes} [Thu May 19 20:20:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:10.582492 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a0d6a-6ed3-4267-afd1-10c301bfee7f None None] REQUEST_METHOD: `POST` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:10.582779 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a0d6a-6ed3-4267-afd1-10c301bfee7f None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:10.583053 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a0d6a-6ed3-4267-afd1-10c301bfee7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:10.602429 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.auth.core [None req-821a0d6a-6ed3-4267-afd1-10c301bfee7f None None] MFA Rules not processed for user `f6d44c613f524fe1804d6c9b65fe97ca`. Rule list: `[]` (Enabled: `True`). {{(pid=144623) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:10.604132 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.fernet_utils [None req-821a0d6a-6ed3-4267-afd1-10c301bfee7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144623) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:10.614066 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 689/1367] 10.222.0.88 () {58 vars in 1058 bytes} [Thu May 19 20:20:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:20:10.622213 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7202e-2903-497f-a36e-e7292d54c299 None None] Authenticating user token {{(pid=144624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:10.637599 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7202e-2903-497f-a36e-e7292d54c299 None None] Validating token access rules against request {{(pid=144624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:10.643263 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e98b4a-179a-4484-ad81-53da09b98f37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d44c613f524fe1804d6c9b65fe97ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:10.644004 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e98b4a-179a-4484-ad81-53da09b98f37 service nova] REQUEST_METHOD: `GET` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:10.644386 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e98b4a-179a-4484-ad81-53da09b98f37 service nova] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:10.644743 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e98b4a-179a-4484-ad81-53da09b98f37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:10.654263 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e98b4a-179a-4484-ad81-53da09b98f37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:10.655965 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e98b4a-179a-4484-ad81-53da09b98f37 service nova] RBAC: Authorization granted {{(pid=144624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:10.666935 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 679/1368] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:20:11.441937 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f9776-52a1-43ec-8acb-9411ea04a9e2 None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:11.442260 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f9776-52a1-43ec-8acb-9411ea04a9e2 None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:11.442495 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f9776-52a1-43ec-8acb-9411ea04a9e2 None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:11.444126 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 690/1369] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:11.449937 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87e2e-0d2e-47ec-baf0-fb9561bd8997 None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:11.450255 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87e2e-0d2e-47ec-baf0-fb9561bd8997 None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:11.450503 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d87e2e-0d2e-47ec-baf0-fb9561bd8997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:11.471641 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-74d87e2e-0d2e-47ec-baf0-fb9561bd8997 None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:11.472962 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-74d87e2e-0d2e-47ec-baf0-fb9561bd8997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:11.482610 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 680/1370] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:20:13.164888 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf3fad-65eb-4676-a2c6-53f1ebb46d7b None None] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:13.165438 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf3fad-65eb-4676-a2c6-53f1ebb46d7b None None] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:13.165438 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf3fad-65eb-4676-a2c6-53f1ebb46d7b None None] PATH_INFO: `/` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:13.166865 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 691/1371] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:20:13.172814 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10ae72-2909-4e5c-8390-f719d51c4b2f None None] REQUEST_METHOD: `POST` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:13.173468 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10ae72-2909-4e5c-8390-f719d51c4b2f None None] SCRIPT_NAME: `/identity` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:13.173630 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10ae72-2909-4e5c-8390-f719d51c4b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=144624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:13.192215 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.auth.core [None req-ca10ae72-2909-4e5c-8390-f719d51c4b2f None None] MFA Rules not processed for user `48915a5b6f6e4fd8a53e478480a2c94c`. Rule list: `[]` (Enabled: `True`). {{(pid=144624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:20:13.193451 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG keystone.common.fernet_utils [None req-ca10ae72-2909-4e5c-8390-f719d51c4b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=144624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:20:13.201405 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: [pid: 144624|app: 0|req: 681/1372] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:20:13.245763 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf3fad-65eb-4676-a2c6-53f1ebb46d7b None None] Authenticating user token {{(pid=144623) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:20:13.259932 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf3fad-65eb-4676-a2c6-53f1ebb46d7b None None] Validating token access rules against request {{(pid=144623) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:20:13.265347 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2105b7ac-8f4d-4782-afc7-6e43a99436d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6d44c613f524fe1804d6c9b65fe97ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'adcff914910e46c181b074c0f7497994', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=144623) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:20:13.266013 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105b7ac-8f4d-4782-afc7-6e43a99436d8 service nova] REQUEST_METHOD: `GET` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:20:13.266295 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105b7ac-8f4d-4782-afc7-6e43a99436d8 service nova] SCRIPT_NAME: `/identity` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:20:13.266567 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2105b7ac-8f4d-4782-afc7-6e43a99436d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=144623) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:20:13.274910 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105b7ac-8f4d-4782-afc7-6e43a99436d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:20:13.277022 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2105b7ac-8f4d-4782-afc7-6e43a99436d8 service nova] RBAC: Authorization granted {{(pid=144623) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:20:13.286877 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: [pid: 144623|app: 0|req: 692/1373] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:20:23.186960 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: DEBUG dbcounter [-] [144624] Writing DB stats keystone:SELECT=21 {{(pid=144624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:23.282286 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: DEBUG dbcounter [-] [144623] Writing DB stats keystone:SELECT=23 {{(pid=144623) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:20:34.921114 dv-cinder-841302-4-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 20:20:34.921435 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: Thu May 19 20:20:34 2022 - graceful shutdown triggered... May 19 20:20:34.922113 dv-cinder-841302-4-iscsi devstack@keystone.service[144623]: Gracefully killing worker 1 (pid: 144623)... May 19 20:20:34.922493 dv-cinder-841302-4-iscsi devstack@keystone.service[144624]: Gracefully killing worker 2 (pid: 144624)... May 19 20:20:35.922460 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: worker 1 buried after 1 seconds May 19 20:20:35.922460 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: worker 2 buried after 1 seconds May 19 20:20:35.922460 dv-cinder-841302-4-iscsi devstack@keystone.service[144609]: goodbye to uWSGI. May 19 20:20:35.924257 dv-cinder-841302-4-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 20:20:35.924848 dv-cinder-841302-4-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 20:27:17.575045 dv-cinder-841302-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 20:27:17.584004 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 20:27:17.586018 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 20:27:17 2022] *** May 19 20:27:17.586018 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 20:27:17.586018 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 20:27:17.586018 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: nodename: dv-cinder-841302-4-iscsi May 19 20:27:17.586018 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: machine: x86_64 May 19 20:27:17.586018 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: clock source: unix May 19 20:27:17.586018 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: pcre jit disabled May 19 20:27:17.586018 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: detected number of CPU cores: 4 May 19 20:27:17.586018 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: current working directory: / May 19 20:27:17.586018 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: detected binary path: /usr/bin/uwsgi-core May 19 20:27:17.586018 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: your processes number limit is 47680 May 19 20:27:17.587343 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: your memory page size is 4096 bytes May 19 20:27:17.587343 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: detected max file descriptor number: 2048 May 19 20:27:17.587343 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: lock engine: pthread robust mutexes May 19 20:27:17.587343 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: thunder lock: enabled May 19 20:27:17.587343 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 20:27:17.587343 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 20:27:17.631735 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: Python main interpreter initialized at 0x5621419d89c0 May 19 20:27:17.631735 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: python threads support enabled May 19 20:27:17.631735 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: your server socket listen backlog is limited to 100 connections May 19 20:27:17.631735 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: your mercy for graceful operations on workers is 90 seconds May 19 20:27:17.632122 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: mapped 403077 bytes (393 KB) for 2 cores May 19 20:27:17.632201 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: *** Operational MODE: preforking *** May 19 20:27:17.632451 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: *** uWSGI is running in multiple interpreter mode *** May 19 20:27:17.632451 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: spawned uWSGI master process (pid: 208570) May 19 20:27:17.632654 dv-cinder-841302-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 20:27:17.634298 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: spawned uWSGI worker 1 (pid: 208581, cores: 1) May 19 20:27:17.634298 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: spawned uWSGI worker 2 (pid: 208582, cores: 1) May 19 20:27:17.634298 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 20:27:23.512698 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:27:23.512698 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: from cryptography.utils import int_from_bytes May 19 20:27:23.513584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:27:23.513584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: from cryptography.utils import int_from_bytes May 19 20:27:23.514069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:27:23.514069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: from cryptography.utils import int_from_bytes May 19 20:27:23.514231 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:27:23.514231 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: from cryptography.utils import int_from_bytes May 19 20:27:23.564116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:27:23.564366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:27:23.564483 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:27:23.564589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:27:23.564681 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] command line args: [] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:27:23.564771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] command line args: [] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:27:23.564860 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:27:23.564949 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:27:23.565037 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ================================================================================ {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:27:23.565209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ================================================================================ {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:27:23.565311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] admin_token = **** {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.565402 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] admin_token = **** {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.565495 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] config_dir = [] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.565585 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] config_dir = [] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.565668 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.565774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.565865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] config_source = [] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.565948 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] config_source = [] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.566035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] debug = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.566123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] debug = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.566218 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.566420 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.566525 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.566606 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.566682 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] insecure_debug = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.566762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] insecure_debug = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.566929 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.567053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.567170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.567289 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.567406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] list_limit = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.567528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] list_limit = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.567655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] log_config_append = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.567774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] log_config_append = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.567903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.568020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.568136 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] log_dir = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.568258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] log_dir = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.568411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] log_file = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.568516 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] log_file = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.568606 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.568694 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.568787 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.568881 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.568971 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.569059 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.569148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.569246 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.569343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.569434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.569518 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.569606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.569695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.569785 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.569879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.569963 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.570105 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.570209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.570298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.570429 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.570525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.570618 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.570716 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.570805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.570948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.571110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.571204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] notification_format = cadf {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.571295 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] notification_format = cadf {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.571473 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.571618 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.571971 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.572108 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.572208 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] publish_errors = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.572297 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] publish_errors = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.572411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.572556 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.572650 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.572775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.572894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.572984 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.573072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.573196 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.573302 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.573434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.573541 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] standard_threads = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.573673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] standard_threads = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.573770 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] strict_password_check = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.573907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] strict_password_check = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574011 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574140 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] use_eventlog = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574326 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] use_eventlog = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574417 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] use_journal = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574513 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] use_journal = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] use_json = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574700 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] use_stderr = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574788 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] use_json = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574877 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] use_syslog = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574966 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] use_stderr = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.574966 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] use_syslog = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.575132 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] watch_log_file = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.575220 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] watch_log_file = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:27:23.575313 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.575458 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.575556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.575702 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.575796 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.575965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.576062 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.576207 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.576313 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.576494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.576594 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.576738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.576834 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.576975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.577071 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.577220 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.577317 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.577470 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.577559 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.577718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.577813 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.577952 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.578056 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.578190 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.578286 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.578431 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.578582 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.578687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.578781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.578918 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.579009 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.579159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.579257 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.579402 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.579499 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.579659 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.579751 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.579904 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.580011 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.580145 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.580242 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.580404 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.580507 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.580645 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.580743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.580912 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.581010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.581167 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.581400 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.581496 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.581635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.581732 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.581864 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.581954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.582099 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.582201 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.582336 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.582434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.582586 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.582681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.582822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.582924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.583056 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.583153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.583287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.583377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.583535 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.583638 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.583773 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.583905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.584024 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.584152 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.584249 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.584427 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.584522 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.584664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.584762 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.584902 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.585000 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.585141 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.585239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.585374 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.585471 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.585608 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.585705 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.585874 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.586465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.587168 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.587856 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.587963 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.588053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.588153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.588243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.588327 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.588446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.588537 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.588626 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.588721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.588810 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.588898 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.588898 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.589061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.589150 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.589150 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.589326 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.589415 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.589503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.589591 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.589591 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.589758 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.589855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.589945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.590039 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.590039 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.590195 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.590299 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.590479 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.590577 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.590720 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.590826 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.590954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.591203 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.591295 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.591450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.591575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.591673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.591924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.592064 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.592162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.592385 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.592503 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.592677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.592767 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.592929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.593112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.593210 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.593408 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.593497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.593647 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.593821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.593919 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.594131 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.enabled = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.594220 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.594370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.594580 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.594684 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.594841 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.594937 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.595081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.595321 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.595420 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.595559 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.595747 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.595837 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.596022 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.596146 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.596288 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.596557 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.596655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.596800 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.596939 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.597030 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.597271 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.597367 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.enabled = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.597511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.597655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.597753 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.597979 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.598080 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.598211 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.598377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.598474 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.598662 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.598768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.598898 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.599107 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.599205 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.599359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.599471 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.599596 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.599832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.599930 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.600078 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.600197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.600491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.600627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.600762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.601003 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.601100 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.601239 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] auth.external = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.601386 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.601484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] auth.mapped = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.602331 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.602481 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.602711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.602809 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.602953 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] auth.password = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.603078 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.603195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] auth.token = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.603425 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.603515 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.603667 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] catalog.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.603793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.603918 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.604146 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.604244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.604419 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.604510 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.604674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.604870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.604974 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.605139 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] credential.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.605242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.605391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] credential.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.605571 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.605669 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.605882 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.605984 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.606124 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.606285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.606381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.606596 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.606692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.606867 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.607017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.607108 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.607349 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.607571 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.607669 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.607885 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.607976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.608133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.608356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.608460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.608666 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.608765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] auth.external = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.608921 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.609066 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] auth.mapped = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.609164 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] federation.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.609388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.609527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.609631 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] federation.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.609870 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.609964 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.610103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.610232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] auth.password = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.610350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.610590 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] auth.token = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.610688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.610851 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.610952 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.611089 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.611311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] catalog.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.611484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.611570 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.611693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.611807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.612047 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.612143 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.612296 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.612466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.612553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.612796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.612886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.613033 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.613166 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.613265 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.613500 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.613605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] credential.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.613739 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.613893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] credential.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.613991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.614229 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.614325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.614466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.614632 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.614736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.614921 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.615011 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.615167 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.615353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.615444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.615658 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.615748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.615915 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.616092 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.616191 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.616447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.616557 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.616690 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.616929 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.617029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.617177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.617420 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.617521 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.617665 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.617903 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.618009 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.618146 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.618723 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.618888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.618888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.619050 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.619146 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.619146 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.619493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.619589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.619589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.619767 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.619853 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.620048 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.620148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] federation.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.620282 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.620549 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.620651 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.620782 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.password = **** {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.621018 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] federation.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.621112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.621400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.621500 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.621650 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.621891 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.621995 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.622131 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.622354 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.622453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.622624 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.623040 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.623145 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.623145 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.623290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.623362 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.623570 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.623662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.623809 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.624046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.624144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.624283 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.624543 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.624642 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.624786 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.624990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.625089 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.625275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.625449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.625537 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.625759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.625930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.626018 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.626283 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.626519 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.626689 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.626801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.626904 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.627083 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.627192 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.627287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.627494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.627593 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.627747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.627869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.627995 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.628119 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.628242 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.628392 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.628517 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.628650 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.628753 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.628891 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.628991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.629131 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.629223 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.629367 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.629472 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.629630 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.629730 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.629883 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.629983 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.630118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.630222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.630367 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.630467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.630616 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.630710 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.630853 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.630947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.631106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] policy.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.631211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.631439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.631572 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.631665 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.631810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.632043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.632142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] receipt.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.632300 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.632420 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.632540 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.632641 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.632781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.633411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.633562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.633661 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.633795 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.633894 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.634037 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] resource.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.634134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.634284 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.634376 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.634523 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.634622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] resource.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.634758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.634850 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.635007 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.635105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.635243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.635351 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.635484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.635583 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] revoke.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.635716 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.password = **** {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.635817 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.635957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.636058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.636196 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.636295 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] role.cache_time = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.636458 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] role.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.636553 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.636689 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] role.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.636795 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.636923 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] role.list_limit = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.637023 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.637176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.637270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.637425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.637517 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.637661 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.637753 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.637896 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.637996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.638141 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.638280 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.638392 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.638639 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.638807 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.638895 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.639096 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.639355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.639355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.639675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.639775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.639867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.639959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.640045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.640135 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.640226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.640481 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.640589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.640722 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.640818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.640971 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.641161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.641256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.641329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.641456 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.641549 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.641707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.641817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.641945 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.642038 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.642180 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.642359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.642459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.642618 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.642713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.642858 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.642951 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.643094 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.643209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.643331 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.643431 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.643563 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.643656 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.643810 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.643908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.644058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.644159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.644291 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.644400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.644556 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.644664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.644792 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.644886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.645025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] token.cache_time = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.645117 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.645261 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] token.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.645362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.645501 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.645647 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.645745 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.645830 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] token.provider = fernet {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.645972 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.646063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.646218 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.646318 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.646473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.646565 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] policy.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.646706 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.646798 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.646957 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.647049 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.647192 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.647284 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.647432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] receipt.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.647525 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] trust.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.647672 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.647772 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.647911 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.648010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.648148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.648241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.648400 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.648558 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.649242 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.649342 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.649479 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.649572 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] resource.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.649726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.649826 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.650008 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] resource.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.650116 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.650249 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.650354 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.connection = **** {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.650501 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.650607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.650773 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.650920 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.651060 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] revoke.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.651151 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.651236 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.651326 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.651454 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.651555 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.651689 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.651782 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] role.cache_time = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.651927 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.652059 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] role.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.652190 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.652292 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] role.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.652415 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.652510 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] role.list_limit = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.652658 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.652759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.653097 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.653200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.653200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.653364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.653448 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.653532 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.653616 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.653708 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.653832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.653932 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.654066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.654160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.654309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.654415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.654541 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.654640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.654733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=208581) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:27:23.654819 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.655025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.655274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.655522 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.655955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.656202 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.656482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.656734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.656990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.657241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.657477 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.657733 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.657989 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.658231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.658466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.658700 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.658934 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.659188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.659444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.659695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.659935 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.660172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.660432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] token.cache_time = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.660673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] token.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.660912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.661162 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] token.provider = fernet {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.661400 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.661673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.661923 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.662163 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.662407 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.662652 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.662901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] trust.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.663139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.663379 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.663616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.663863 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.664147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.664403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.664650 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.664948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.665212 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.connection = **** {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.665460 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.665709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.665949 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.666193 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.666429 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.666667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.666905 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.667147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.667393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.667630 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.667875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.668384 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.668644 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.668883 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.669130 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.669363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.669605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.669842 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:27:23.670014 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=208582) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:27:23.686472 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.686705 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.686933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.696455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.697558 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.697793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.698015 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.706953 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.725440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.725676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.725995 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.726229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.779554 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.780200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.831657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.831870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.832760 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.833002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.897179 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.897394 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.897618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.897713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.897839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.897948 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.898045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.898132 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.898247 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.898335 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.899116 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.899338 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.922945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.923180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.923284 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.923400 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.923507 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.923609 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.923696 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.923819 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.923914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.924013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.924112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.924230 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.924325 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.924483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.935496 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.935955 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.950421 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.950710 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.950830 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.950920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.951084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.951261 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.951368 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.951467 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.951557 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.951643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.951737 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=208582) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.951824 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.951993 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.952207 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=208581) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:27:23.969555 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.969703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:27:23.970040 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.970139 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.970233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:27:23.970324 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:27:23.970496 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.970627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.970757 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.970849 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:27:23.971014 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.971107 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.971269 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.971363 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.971577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.971832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.982444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.982723 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.982979 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.983071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.983256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.983347 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.983523 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.983806 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.991417 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:23.992070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:23.995267 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:23.995546 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.995828 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.995933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:23.996177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.996269 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.996486 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.996582 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.996778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.996872 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.997036 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.997143 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.997309 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.997417 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.997582 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.997675 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.997854 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.997946 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.998122 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.998222 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.998431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:23.998700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.025364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.025890 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.026999 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.027274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.027644 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.027922 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.032601 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.032882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.033160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.033274 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.033497 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.033777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.040541 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.040825 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.041136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.041418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.045724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.046009 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.046292 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.046464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.046558 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.046754 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.046847 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.047038 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.047138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.047317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.047409 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.047591 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.047695 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.047866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.048138 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.048430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.066861 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.067138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.067695 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.067979 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.072394 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.073110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.076056 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.076366 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.076888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.077185 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.081439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.082152 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.085069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.085755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.088912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.089213 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.089498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.089684 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.089795 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.090098 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.090203 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.090375 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.090490 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.090659 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.091031 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.091230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.109517 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.109899 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.110177 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.110350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.110465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.110737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.111008 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.111283 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.118959 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.119816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.120727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.121007 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.121701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.121977 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.124141 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.124446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.124730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.125019 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.125112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.125308 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.125402 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.125646 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.125931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.126211 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.139388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.139673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.139956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.140236 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.140548 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.140643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.140822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.140915 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.141110 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.141227 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.141427 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.141707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.141990 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.142277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.159700 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.160900 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.163026 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.164243 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.166417 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.167595 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.168020 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.169662 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.169960 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.170243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.170521 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.171635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.171925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.172210 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.178005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.180180 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.180300 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.181947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.183465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.183757 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.184041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.184319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.185228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.185519 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.185806 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.186085 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.193659 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.194025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.194333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.196240 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.196692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.196976 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.202846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=208582) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.203123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.203402 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.203684 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.203956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.204233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.204544 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.204821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.204965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=208581) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:27:24.205105 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.205197 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.205385 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.205480 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.205662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.205754 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.205922 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.206023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.206199 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.206292 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.206460 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=208582) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.206559 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.206723 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.207008 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.207278 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.207557 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.207818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.208086 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.208361 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=208581) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:27:24.246046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5621419d89c0 pid: 208582 (default app) May 19 20:27:24.248115 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5621419d89c0 pid: 208581 (default app) May 19 20:27:24.249505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0740af-fcb6-4ca1-a8b0-ad98c27141d8 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:24.249815 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0740af-fcb6-4ca1-a8b0-ad98c27141d8 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:24.250099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0740af-fcb6-4ca1-a8b0-ad98c27141d8 None None] PATH_INFO: `/v3` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:24.254339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 1/1] 10.222.0.88 () {50 vars in 791 bytes} [Thu May 19 20:27:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:33.295493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bdbfb-c262-433b-b255-eeaf47d61db1 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:33.296101 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bdbfb-c262-433b-b255-eeaf47d61db1 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:33.296101 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bdbfb-c262-433b-b255-eeaf47d61db1 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:33.300613 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 1/2] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:27:33.307051 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fc890-53de-4ca6-9b26-2ff8b6ab46ba None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:33.307593 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fc890-53de-4ca6-9b26-2ff8b6ab46ba None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:33.307593 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fc890-53de-4ca6-9b26-2ff8b6ab46ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:33.384489 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: INFO dbcounter [None req-bd5fc890-53de-4ca6-9b26-2ff8b6ab46ba None None] Registered counter for database keystone May 19 20:27:33.411339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG oslo_db.sqlalchemy.engines [None req-bd5fc890-53de-4ca6-9b26-2ff8b6ab46ba None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=208582) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:27:33.418895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writer thread running {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:27:33.606561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG passlib.handlers.bcrypt [None req-bd5fc890-53de-4ca6-9b26-2ff8b6ab46ba None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=208582) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:27:33.606889 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG passlib.handlers.bcrypt [None req-bd5fc890-53de-4ca6-9b26-2ff8b6ab46ba None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=208582) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:27:33.665427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-bd5fc890-53de-4ca6-9b26-2ff8b6ab46ba None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:33.667110 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-bd5fc890-53de-4ca6-9b26-2ff8b6ab46ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:33.807911 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 2/3] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 20:27:33.831321 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bdbfb-c262-433b-b255-eeaf47d61db1 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:33.902501 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: INFO dbcounter [None req-916bdbfb-c262-433b-b255-eeaf47d61db1 None None] Registered counter for database keystone May 19 20:27:33.924272 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG oslo_db.sqlalchemy.engines [None req-916bdbfb-c262-433b-b255-eeaf47d61db1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=208581) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:27:33.930822 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writer thread running {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:27:34.055258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bdbfb-c262-433b-b255-eeaf47d61db1 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:34.059796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80daaf-8ef4-4593-8f26-e27438a43303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e556dd7d9fbd4a6e9dbb4191eee92638', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:34.060471 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80daaf-8ef4-4593-8f26-e27438a43303 admin admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:34.060759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80daaf-8ef4-4593-8f26-e27438a43303 admin admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:34.061041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80daaf-8ef4-4593-8f26-e27438a43303 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:34.070445 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80daaf-8ef4-4593-8f26-e27438a43303 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:34.122375 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eb80daaf-8ef4-4593-8f26-e27438a43303 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:27:34.122717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80daaf-8ef4-4593-8f26-e27438a43303 admin admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:34.133873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 2/4] 10.222.0.88 () {58 vars in 1324 bytes} [Thu May 19 20:27:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:27:34.140054 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fc890-53de-4ca6-9b26-2ff8b6ab46ba None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:34.155690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fc890-53de-4ca6-9b26-2ff8b6ab46ba None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:34.160240 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa5fcf-8ab4-452b-ad55-59d0943ab88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e556dd7d9fbd4a6e9dbb4191eee92638', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:34.161412 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa5fcf-8ab4-452b-ad55-59d0943ab88d admin admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:34.161699 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa5fcf-8ab4-452b-ad55-59d0943ab88d admin admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:34.161977 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa5fcf-8ab4-452b-ad55-59d0943ab88d admin admin] PATH_INFO: `/v3/projects/e556dd7d9fbd4a6e9dbb4191eee92638` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:34.163535 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa5fcf-8ab4-452b-ad55-59d0943ab88d admin admin] RBAC: Authorizing `identity:get_project(project_id=e556dd7d9fbd4a6e9dbb4191eee92638)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:34.324499 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d4aa5fcf-8ab4-452b-ad55-59d0943ab88d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:27:34.325151 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa5fcf-8ab4-452b-ad55-59d0943ab88d admin admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:34.329674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 3/5] 10.222.0.88 () {56 vars in 1207 bytes} [Thu May 19 20:27:34 2022] GET /identity/v3/projects/e556dd7d9fbd4a6e9dbb4191eee92638 => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:36.560168 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cac60b-5440-4638-9a50-234e92e50023 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:36.560751 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cac60b-5440-4638-9a50-234e92e50023 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:36.560847 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cac60b-5440-4638-9a50-234e92e50023 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:36.564985 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 3/6] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:36.571110 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494633f-79b0-4f43-97cf-7d0553980685 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:36.571569 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494633f-79b0-4f43-97cf-7d0553980685 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:36.571684 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494633f-79b0-4f43-97cf-7d0553980685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:36.591211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-4494633f-79b0-4f43-97cf-7d0553980685 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:36.592776 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-4494633f-79b0-4f43-97cf-7d0553980685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:36.600783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 4/7] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 20:27:36.623008 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cac60b-5440-4638-9a50-234e92e50023 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:36.635883 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cac60b-5440-4638-9a50-234e92e50023 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:36.640230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38670c12-6eec-4f2b-ad4b-5d81902bcb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e556dd7d9fbd4a6e9dbb4191eee92638', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:36.640916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38670c12-6eec-4f2b-ad4b-5d81902bcb13 admin admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:36.641205 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38670c12-6eec-4f2b-ad4b-5d81902bcb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:36.641482 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38670c12-6eec-4f2b-ad4b-5d81902bcb13 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:36.650135 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38670c12-6eec-4f2b-ad4b-5d81902bcb13 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:36.651948 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38670c12-6eec-4f2b-ad4b-5d81902bcb13 admin admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:36.661031 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 4/8] 10.222.0.88 () {58 vars in 1324 bytes} [Thu May 19 20:27:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:27:36.666848 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494633f-79b0-4f43-97cf-7d0553980685 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:36.676219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4494633f-79b0-4f43-97cf-7d0553980685 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:36.680707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b5815-edac-4ad8-bc2f-31aa84cabf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e556dd7d9fbd4a6e9dbb4191eee92638', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:36.681574 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b5815-edac-4ad8-bc2f-31aa84cabf42 admin admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:36.681861 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b5815-edac-4ad8-bc2f-31aa84cabf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:36.682134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b5815-edac-4ad8-bc2f-31aa84cabf42 admin admin] PATH_INFO: `/v3/users/cc336177e5d04523af605ab7e66532eb` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:36.683579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b5815-edac-4ad8-bc2f-31aa84cabf42 admin admin] RBAC: Authorizing `identity:get_user(user_id=cc336177e5d04523af605ab7e66532eb)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:36.685429 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b5815-edac-4ad8-bc2f-31aa84cabf42 admin admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:36.687047 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 5/9] 10.222.0.88 () {56 vars in 1198 bytes} [Thu May 19 20:27:36 2022] GET /identity/v3/users/cc336177e5d04523af605ab7e66532eb => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:38.982998 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277af23b-7ecf-47cb-82cb-5c8710f540cb None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:38.983548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277af23b-7ecf-47cb-82cb-5c8710f540cb None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:38.983548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277af23b-7ecf-47cb-82cb-5c8710f540cb None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:38.985035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 5/10] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:27:38.990497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645500c-ad93-4f28-a6b1-148bb4864769 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:38.990983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645500c-ad93-4f28-a6b1-148bb4864769 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:38.991081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645500c-ad93-4f28-a6b1-148bb4864769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:39.008157 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-3645500c-ad93-4f28-a6b1-148bb4864769 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:39.009376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-3645500c-ad93-4f28-a6b1-148bb4864769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:39.046908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e34a8a-e45a-42a3-a934-b6c96a62189e None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:39.047201 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e34a8a-e45a-42a3-a934-b6c96a62189e None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:39.047479 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e34a8a-e45a-42a3-a934-b6c96a62189e None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:39.048880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 6/11] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:27:39.054251 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c278f-934b-499b-a58d-e22b7c5b300c None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:39.054543 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c278f-934b-499b-a58d-e22b7c5b300c None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:39.054830 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c278f-934b-499b-a58d-e22b7c5b300c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:39.093324 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 6/12] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:39.095004 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG passlib.handlers.bcrypt [None req-689c278f-934b-499b-a58d-e22b7c5b300c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=208581) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:27:39.095430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG passlib.handlers.bcrypt [None req-689c278f-934b-499b-a58d-e22b7c5b300c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=208581) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:27:39.116946 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645500c-ad93-4f28-a6b1-148bb4864769 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:39.140739 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:39.142123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-689c278f-934b-499b-a58d-e22b7c5b300c None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:39.143745 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-689c278f-934b-499b-a58d-e22b7c5b300c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:39.229509 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645500c-ad93-4f28-a6b1-148bb4864769 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:39.248562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 7/13] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:27:39.263045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccec18-321f-4af1-85e4-9387a311c124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:39.263856 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccec18-321f-4af1-85e4-9387a311c124 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:39.264139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccec18-321f-4af1-85e4-9387a311c124 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:39.264452 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ccec18-321f-4af1-85e4-9387a311c124 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:39.270539 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c278f-934b-499b-a58d-e22b7c5b300c None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:39.277047 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ccec18-321f-4af1-85e4-9387a311c124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-55ccec18-321f-4af1-85e4-9387a311c124 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:39.289325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:39.290440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:27:39.291398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 7/14] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:27:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:27:39.296567 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccec18-321f-4af1-85e4-9387a311c124 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:39.385269 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:39.405357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ccec18-321f-4af1-85e4-9387a311c124 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:39.441595 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec96d50-24c6-4d94-baf1-0e2326576941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:39.442270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec96d50-24c6-4d94-baf1-0e2326576941 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:39.442558 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec96d50-24c6-4d94-baf1-0e2326576941 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:39.442837 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec96d50-24c6-4d94-baf1-0e2326576941 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:39.443603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec96d50-24c6-4d94-baf1-0e2326576941 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:39.443916 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec96d50-24c6-4d94-baf1-0e2326576941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:39.445926 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec96d50-24c6-4d94-baf1-0e2326576941 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:39.465253 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 8/15] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:27:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:27:39.466467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec96d50-24c6-4d94-baf1-0e2326576941 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:39.496359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c278f-934b-499b-a58d-e22b7c5b300c None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:39.499985 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:39.539208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27510325-b32c-47f3-8c1d-b192e7ec5c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:39.540389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27510325-b32c-47f3-8c1d-b192e7ec5c49 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:39.541117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27510325-b32c-47f3-8c1d-b192e7ec5c49 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:39.541460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27510325-b32c-47f3-8c1d-b192e7ec5c49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:39.576993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec96d50-24c6-4d94-baf1-0e2326576941 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:39.611780 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27510325-b32c-47f3-8c1d-b192e7ec5c49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:39.611986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addb2d6-bbc4-4cff-8e95-62beda020363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:39.612905 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addb2d6-bbc4-4cff-8e95-62beda020363 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:39.613198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addb2d6-bbc4-4cff-8e95-62beda020363 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:39.613328 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27510325-b32c-47f3-8c1d-b192e7ec5c49 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:39.613482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addb2d6-bbc4-4cff-8e95-62beda020363 None admin] PATH_INFO: `/v3/users/cc336177e5d04523af605ab7e66532eb` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:39.614790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addb2d6-bbc4-4cff-8e95-62beda020363 None admin] RBAC: Authorizing `identity:get_user(user_id=cc336177e5d04523af605ab7e66532eb)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:39.616261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addb2d6-bbc4-4cff-8e95-62beda020363 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:39.618191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 9/16] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:27:39 2022] GET /identity/v3/users/cc336177e5d04523af605ab7e66532eb => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:27:39.623391 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addb2d6-bbc4-4cff-8e95-62beda020363 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:39.638256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:39.718250 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 8/17] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:27:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:27:39.724401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27510325-b32c-47f3-8c1d-b192e7ec5c49 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:39.727566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:39.732368 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addb2d6-bbc4-4cff-8e95-62beda020363 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:39.770788 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796eb29-b81c-4921-a6b2-c4e4c2206d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:39.771531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796eb29-b81c-4921-a6b2-c4e4c2206d90 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:39.771822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796eb29-b81c-4921-a6b2-c4e4c2206d90 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:39.772098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796eb29-b81c-4921-a6b2-c4e4c2206d90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:39.773517 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796eb29-b81c-4921-a6b2-c4e4c2206d90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:39.775036 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8796eb29-b81c-4921-a6b2-c4e4c2206d90 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:39.776836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 10/18] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:27:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:27:39.782900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796eb29-b81c-4921-a6b2-c4e4c2206d90 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:39.836578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27510325-b32c-47f3-8c1d-b192e7ec5c49 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:39.872357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a7114-c0b5-48bb-bec9-32d83281a157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:39.872600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:39.873352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a7114-c0b5-48bb-bec9-32d83281a157 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:39.873506 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a7114-c0b5-48bb-bec9-32d83281a157 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:39.873806 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a7114-c0b5-48bb-bec9-32d83281a157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:39.892729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796eb29-b81c-4921-a6b2-c4e4c2206d90 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:39.894548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a7114-c0b5-48bb-bec9-32d83281a157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:39.896061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a7114-c0b5-48bb-bec9-32d83281a157 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-698a7114-c0b5-48bb-bec9-32d83281a157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:27:39.917713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:39.919171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:39.920120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:27:39.920120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:27:39.920120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:27:39.920120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:27:39.920120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 9/19] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:27:39.925375 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a7114-c0b5-48bb-bec9-32d83281a157 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:39.931727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636e1ac-38d0-4dcd-b42b-3465d879fa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:39.932441 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e1ac-38d0-4dcd-b42b-3465d879fa09 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:39.932872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e1ac-38d0-4dcd-b42b-3465d879fa09 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:39.933012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e1ac-38d0-4dcd-b42b-3465d879fa09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:39.933870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636e1ac-38d0-4dcd-b42b-3465d879fa09 None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.domain.id=default, user.id=cc336177e5d04523af605ab7e66532eb) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:39.934191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636e1ac-38d0-4dcd-b42b-3465d879fa09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:39.935704 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636e1ac-38d0-4dcd-b42b-3465d879fa09 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:39.943179 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 11/20] 10.222.0.88 () {56 vars in 1322 bytes} [Thu May 19 20:27:39 2022] GET /identity/v3/role_assignments?user.id=cc336177e5d04523af605ab7e66532eb&scope.domain.id=default&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:27:39.992806 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:40.043784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a7114-c0b5-48bb-bec9-32d83281a157 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:40.088447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105e3ba-f674-48c8-a135-8736866f4c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:40.089168 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105e3ba-f674-48c8-a135-8736866f4c4f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:40.089499 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105e3ba-f674-48c8-a135-8736866f4c4f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:40.089777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105e3ba-f674-48c8-a135-8736866f4c4f None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:40.090586 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105e3ba-f674-48c8-a135-8736866f4c4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:40.090902 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105e3ba-f674-48c8-a135-8736866f4c4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:40.092470 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105e3ba-f674-48c8-a135-8736866f4c4f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:40.114205 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 10/21] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:27:42.605417 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91546a79-805a-46ba-a505-39d084d9fb69 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:42.605983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91546a79-805a-46ba-a505-39d084d9fb69 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:42.605983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91546a79-805a-46ba-a505-39d084d9fb69 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:42.607341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 12/22] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:42.613019 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec08a88-0ed0-4c18-80b5-cfe372bcc893 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:42.613898 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec08a88-0ed0-4c18-80b5-cfe372bcc893 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:42.614393 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec08a88-0ed0-4c18-80b5-cfe372bcc893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:42.641389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-cec08a88-0ed0-4c18-80b5-cfe372bcc893 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:42.643014 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-cec08a88-0ed0-4c18-80b5-cfe372bcc893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:42.734706 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 11/23] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:42.777831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91546a79-805a-46ba-a505-39d084d9fb69 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:42.922680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91546a79-805a-46ba-a505-39d084d9fb69 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:42.972945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f896b-ca57-4cc2-87ee-94c5746a5299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:42.973759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f896b-ca57-4cc2-87ee-94c5746a5299 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:42.974043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f896b-ca57-4cc2-87ee-94c5746a5299 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:42.974320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f896b-ca57-4cc2-87ee-94c5746a5299 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:42.995573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f896b-ca57-4cc2-87ee-94c5746a5299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-187f896b-ca57-4cc2-87ee-94c5746a5299 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:43.003231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:43.004241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:27:43.005257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 13/24] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:27:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:43.011127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec08a88-0ed0-4c18-80b5-cfe372bcc893 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:43.044042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d557f-78dc-4569-a947-223266a2108d None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:43.044443 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d557f-78dc-4569-a947-223266a2108d None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:43.044651 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d557f-78dc-4569-a947-223266a2108d None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:43.046102 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 14/25] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:27:43.058440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2d335-af22-4534-9f30-78c55f76c91b None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:43.059069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2d335-af22-4534-9f30-78c55f76c91b None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:43.059553 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2d335-af22-4534-9f30-78c55f76c91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:43.084049 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:43.105242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-e4b2d335-af22-4534-9f30-78c55f76c91b None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:43.106900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-e4b2d335-af22-4534-9f30-78c55f76c91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:43.114314 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:43.195410 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec08a88-0ed0-4c18-80b5-cfe372bcc893 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:43.247748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a01298-2f18-471e-a5ad-89946e5faf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:43.249638 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a01298-2f18-471e-a5ad-89946e5faf10 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:43.251137 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a01298-2f18-471e-a5ad-89946e5faf10 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:43.251581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a01298-2f18-471e-a5ad-89946e5faf10 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:43.252569 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a01298-2f18-471e-a5ad-89946e5faf10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:43.253400 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 15/26] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:43.254592 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a01298-2f18-471e-a5ad-89946e5faf10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:43.255471 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf73d4-8135-4832-82db-887837650c08 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:43.257113 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a01298-2f18-471e-a5ad-89946e5faf10 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:43.257582 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf73d4-8135-4832-82db-887837650c08 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:43.258045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf73d4-8135-4832-82db-887837650c08 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:43.261562 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 16/27] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:43.267529 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673a65b-a138-4970-b0ac-1a2646fc264f None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:43.269896 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673a65b-a138-4970-b0ac-1a2646fc264f None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:43.270346 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673a65b-a138-4970-b0ac-1a2646fc264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:43.293166 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 12/28] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:27:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:43.296955 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a01298-2f18-471e-a5ad-89946e5faf10 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:43.305574 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-f673a65b-a138-4970-b0ac-1a2646fc264f None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:43.307274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-f673a65b-a138-4970-b0ac-1a2646fc264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:43.434043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a01298-2f18-471e-a5ad-89946e5faf10 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:43.451280 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:43.476648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66d838-3a84-41bd-9c2e-2cacad41ca2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:43.477888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66d838-3a84-41bd-9c2e-2cacad41ca2e None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:43.478347 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66d838-3a84-41bd-9c2e-2cacad41ca2e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:43.478858 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66d838-3a84-41bd-9c2e-2cacad41ca2e None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:43.479845 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66d838-3a84-41bd-9c2e-2cacad41ca2e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:43.481915 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66d838-3a84-41bd-9c2e-2cacad41ca2e None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:43.507818 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 17/29] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:27:43.509039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673a65b-a138-4970-b0ac-1a2646fc264f None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:43.583473 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:43.624305 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673a65b-a138-4970-b0ac-1a2646fc264f None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:43.664590 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f9d34-7a2c-47ce-8cce-30de7e9c79bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:43.665489 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f9d34-7a2c-47ce-8cce-30de7e9c79bd None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:43.665781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f9d34-7a2c-47ce-8cce-30de7e9c79bd None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:43.666061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f9d34-7a2c-47ce-8cce-30de7e9c79bd None admin] PATH_INFO: `/v3/users/cc336177e5d04523af605ab7e66532eb` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:43.667326 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f9d34-7a2c-47ce-8cce-30de7e9c79bd None admin] RBAC: Authorizing `identity:get_user(user_id=cc336177e5d04523af605ab7e66532eb)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:43.677499 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f9d34-7a2c-47ce-8cce-30de7e9c79bd None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:43.684612 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 18/30] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:27:43 2022] GET /identity/v3/users/cc336177e5d04523af605ab7e66532eb => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:43.685549 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f9d34-7a2c-47ce-8cce-30de7e9c79bd None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:43.761123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 13/31] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:27:43 2022] POST /identity/v3/roles => generated 219 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:27:43.762709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66d838-3a84-41bd-9c2e-2cacad41ca2e None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:43.853867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f9d34-7a2c-47ce-8cce-30de7e9c79bd None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:43.927431 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:43.938527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388f98b-41d0-4c89-bfc1-ac8d52e92259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:43.940677 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388f98b-41d0-4c89-bfc1-ac8d52e92259 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:43.941002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388f98b-41d0-4c89-bfc1-ac8d52e92259 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:43.941289 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388f98b-41d0-4c89-bfc1-ac8d52e92259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:43.958503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388f98b-41d0-4c89-bfc1-ac8d52e92259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:43.960785 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388f98b-41d0-4c89-bfc1-ac8d52e92259 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-b388f98b-41d0-4c89-bfc1-ac8d52e92259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:27:43.984359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:43.985406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:43.986371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:27:43.986371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:27:43.986371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:27:43.986371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:27:43.986371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 19/32] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:43.993435 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66d838-3a84-41bd-9c2e-2cacad41ca2e None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:43.993634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2cfe85-2301-4cc6-86ef-86e681bda6db None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:43.993845 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2cfe85-2301-4cc6-86ef-86e681bda6db None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:43.994144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2cfe85-2301-4cc6-86ef-86e681bda6db None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:43.996044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 20/33] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:43.996943 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2cfe85-2301-4cc6-86ef-86e681bda6db None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:44.042422 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcf69e-bba0-4532-96c5-1b7b6de6cd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:44.043602 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf69e-bba0-4532-96c5-1b7b6de6cd1f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:44.043953 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf69e-bba0-4532-96c5-1b7b6de6cd1f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:44.044302 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcf69e-bba0-4532-96c5-1b7b6de6cd1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:44.046323 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcf69e-bba0-4532-96c5-1b7b6de6cd1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:44.047972 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcf69e-bba0-4532-96c5-1b7b6de6cd1f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:44.049751 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 14/34] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:27:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:44.055742 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9ffa4-fca3-42c1-b4d8-ccb416142269 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:44.056047 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9ffa4-fca3-42c1-b4d8-ccb416142269 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:44.056306 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9ffa4-fca3-42c1-b4d8-ccb416142269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:44.077353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-27d9ffa4-fca3-42c1-b4d8-ccb416142269 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:44.080780 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-27d9ffa4-fca3-42c1-b4d8-ccb416142269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:44.120358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:44.127057 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2cfe85-2301-4cc6-86ef-86e681bda6db None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:44.171943 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 15/35] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:44.174160 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9ffa4-fca3-42c1-b4d8-ccb416142269 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:44.176366 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc1e99-6b32-4390-97cc-310ddb4a86c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:44.177086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc1e99-6b32-4390-97cc-310ddb4a86c4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:44.177372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc1e99-6b32-4390-97cc-310ddb4a86c4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:44.177650 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc1e99-6b32-4390-97cc-310ddb4a86c4 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:44.179318 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc1e99-6b32-4390-97cc-310ddb4a86c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:44.179636 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc1e99-6b32-4390-97cc-310ddb4a86c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:44.182016 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc1e99-6b32-4390-97cc-310ddb4a86c4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:44.212246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 21/36] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:44.214682 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701c5d9-6365-421a-b3e8-54d952e29d23 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:44.215587 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701c5d9-6365-421a-b3e8-54d952e29d23 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:44.216323 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701c5d9-6365-421a-b3e8-54d952e29d23 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:44.218556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 22/37] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:44.219943 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701c5d9-6365-421a-b3e8-54d952e29d23 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:44.280567 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:44.298112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9ffa4-fca3-42c1-b4d8-ccb416142269 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:44.336938 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c59ad-ea0a-4830-ad5d-db1aafcfbe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:44.337853 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c59ad-ea0a-4830-ad5d-db1aafcfbe9d None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:44.338143 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c59ad-ea0a-4830-ad5d-db1aafcfbe9d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:44.338439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c59ad-ea0a-4830-ad5d-db1aafcfbe9d None admin] PATH_INFO: `/v3/domains/default/users/cc336177e5d04523af605ab7e66532eb/roles/8eaff9cd613a49b29548e90d7e3e9215` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:44.341476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c59ad-ea0a-4830-ad5d-db1aafcfbe9d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cc336177e5d04523af605ab7e66532eb, role_id=8eaff9cd613a49b29548e90d7e3e9215)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:44.342983 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c59ad-ea0a-4830-ad5d-db1aafcfbe9d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:44.348312 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701c5d9-6365-421a-b3e8-54d952e29d23 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:44.386567 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730468c3-b987-4056-93a5-fc670cb988b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:44.387264 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730468c3-b987-4056-93a5-fc670cb988b1 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:44.387551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730468c3-b987-4056-93a5-fc670cb988b1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:44.387831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730468c3-b987-4056-93a5-fc670cb988b1 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:44.388642 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730468c3-b987-4056-93a5-fc670cb988b1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:44.390064 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730468c3-b987-4056-93a5-fc670cb988b1 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:44.437049 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 16/38] 10.222.0.88 () {58 vars in 1361 bytes} [Thu May 19 20:27:44 2022] PUT /identity/v3/domains/default/users/cc336177e5d04523af605ab7e66532eb/roles/8eaff9cd613a49b29548e90d7e3e9215 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:27:44.437937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c59ad-ea0a-4830-ad5d-db1aafcfbe9d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:44.486872 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:44.576718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 23/39] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:27:44 2022] POST /identity/v3/roles => generated 225 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:27:44.578659 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f81fb-df3a-4c22-9ba5-8766b50dbd82 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:44.579202 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f81fb-df3a-4c22-9ba5-8766b50dbd82 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:44.579709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f81fb-df3a-4c22-9ba5-8766b50dbd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:44.600671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-1f2f81fb-df3a-4c22-9ba5-8766b50dbd82 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:44.601414 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c59ad-ea0a-4830-ad5d-db1aafcfbe9d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:44.603413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-1f2f81fb-df3a-4c22-9ba5-8766b50dbd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:44.674912 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dafcd4-9b14-4354-bc4d-53810e74e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:44.676459 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dafcd4-9b14-4354-bc4d-53810e74e7a3 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:44.676608 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dafcd4-9b14-4354-bc4d-53810e74e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:44.676822 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dafcd4-9b14-4354-bc4d-53810e74e7a3 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:44.677588 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dafcd4-9b14-4354-bc4d-53810e74e7a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:44.679592 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dafcd4-9b14-4354-bc4d-53810e74e7a3 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:44.703595 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 24/40] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:44.705176 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3e0e8-39f1-4e2c-8036-5a23b3c1ad1f None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:44.705474 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3e0e8-39f1-4e2c-8036-5a23b3c1ad1f None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:44.705767 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3e0e8-39f1-4e2c-8036-5a23b3c1ad1f None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:44.712383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 25/41] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:44.713715 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6ea0-873e-4f93-9ef0-dad3264ac297 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:44.713962 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6ea0-873e-4f93-9ef0-dad3264ac297 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:44.714249 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6ea0-873e-4f93-9ef0-dad3264ac297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:44.747925 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-df2c6ea0-873e-4f93-9ef0-dad3264ac297 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:44.750242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-df2c6ea0-873e-4f93-9ef0-dad3264ac297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:44.787096 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:44.836573 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 17/42] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:27:44 2022] POST /identity/v3/projects => generated 301 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:27:44.837617 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dafcd4-9b14-4354-bc4d-53810e74e7a3 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:44.846536 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 26/43] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:44.870875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c6ea0-873e-4f93-9ef0-dad3264ac297 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:44.955187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:44.982670 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dafcd4-9b14-4354-bc4d-53810e74e7a3 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:44.988086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c6ea0-873e-4f93-9ef0-dad3264ac297 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:45.026634 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e9b2f-507a-4944-91f9-cb7941fed5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:45.027596 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e9b2f-507a-4944-91f9-cb7941fed5d9 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:45.028047 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e9b2f-507a-4944-91f9-cb7941fed5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:45.028553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e9b2f-507a-4944-91f9-cb7941fed5d9 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:45.029601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e9b2f-507a-4944-91f9-cb7941fed5d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:45.031268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e9b2f-507a-4944-91f9-cb7941fed5d9 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:45.037207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276ea86-16fb-4401-9bbc-8aaa0a0eb11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:45.038097 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276ea86-16fb-4401-9bbc-8aaa0a0eb11d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:45.038388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276ea86-16fb-4401-9bbc-8aaa0a0eb11d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:45.038671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e276ea86-16fb-4401-9bbc-8aaa0a0eb11d None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:45.039986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276ea86-16fb-4401-9bbc-8aaa0a0eb11d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:45.041727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e276ea86-16fb-4401-9bbc-8aaa0a0eb11d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:45.044205 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 27/44] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:27:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:27:45.049678 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276ea86-16fb-4401-9bbc-8aaa0a0eb11d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:45.088667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:45.105033 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 18/45] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:27:44 2022] POST /identity/v3/roles => generated 223 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:27:45.189388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e276ea86-16fb-4401-9bbc-8aaa0a0eb11d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:45.229810 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b2648-1ba5-45e7-921f-c4e6d490fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:45.230548 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2648-1ba5-45e7-921f-c4e6d490fda6 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:45.230844 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2648-1ba5-45e7-921f-c4e6d490fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:45.231127 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b2648-1ba5-45e7-921f-c4e6d490fda6 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:45.231863 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b2648-1ba5-45e7-921f-c4e6d490fda6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:45.233417 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b2648-1ba5-45e7-921f-c4e6d490fda6 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:45.326653 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 28/46] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:27:45 2022] POST /identity/v3/projects => generated 312 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:27:46.788852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba1fd1-5eb9-45e1-87e9-c47022aa7d36 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:46.789384 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba1fd1-5eb9-45e1-87e9-c47022aa7d36 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:46.789384 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba1fd1-5eb9-45e1-87e9-c47022aa7d36 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:46.790793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 19/47] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:46.796577 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a04111-4df9-4c22-9e88-e0489f60fb3d None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:46.797009 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a04111-4df9-4c22-9e88-e0489f60fb3d None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:46.797150 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a04111-4df9-4c22-9e88-e0489f60fb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:46.814277 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-a0a04111-4df9-4c22-9e88-e0489f60fb3d None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:46.815466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-a0a04111-4df9-4c22-9e88-e0489f60fb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:46.882402 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 29/48] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:46.905852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba1fd1-5eb9-45e1-87e9-c47022aa7d36 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:47.014476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba1fd1-5eb9-45e1-87e9-c47022aa7d36 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:47.050679 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28bd1d-9aa9-400c-89cc-2b8992f620de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:47.051459 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28bd1d-9aa9-400c-89cc-2b8992f620de None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:47.051759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28bd1d-9aa9-400c-89cc-2b8992f620de None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:47.052061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28bd1d-9aa9-400c-89cc-2b8992f620de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:47.061632 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:47.066063 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28bd1d-9aa9-400c-89cc-2b8992f620de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-ff28bd1d-9aa9-400c-89cc-2b8992f620de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:47.075005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:47.081884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:27:47.082757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 20/49] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:27:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:47.083531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a04111-4df9-4c22-9e88-e0489f60fb3d None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:47.097871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:47.196612 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a04111-4df9-4c22-9e88-e0489f60fb3d None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:47.233564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7152e-8208-43f7-86ee-e568bbc2add1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:47.234247 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7152e-8208-43f7-86ee-e568bbc2add1 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:47.234542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7152e-8208-43f7-86ee-e568bbc2add1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:47.234827 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7152e-8208-43f7-86ee-e568bbc2add1 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:47.235600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7152e-8208-43f7-86ee-e568bbc2add1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:47.235918 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7152e-8208-43f7-86ee-e568bbc2add1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:47.237386 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7152e-8208-43f7-86ee-e568bbc2add1 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:47.246438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 30/50] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:27:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:47.252053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28bd1d-9aa9-400c-89cc-2b8992f620de None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:47.369033 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28bd1d-9aa9-400c-89cc-2b8992f620de None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:47.404551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b2494-74a8-4e24-a5fb-4265bd58db6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:47.405441 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b2494-74a8-4e24-a5fb-4265bd58db6d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:47.405726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b2494-74a8-4e24-a5fb-4265bd58db6d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:47.406006 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b2494-74a8-4e24-a5fb-4265bd58db6d None admin] PATH_INFO: `/v3/users/cc336177e5d04523af605ab7e66532eb` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:47.407274 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b2494-74a8-4e24-a5fb-4265bd58db6d None admin] RBAC: Authorizing `identity:get_user(user_id=cc336177e5d04523af605ab7e66532eb)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:47.408787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1b2494-74a8-4e24-a5fb-4265bd58db6d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:47.410475 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 21/51] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:27:47 2022] GET /identity/v3/users/cc336177e5d04523af605ab7e66532eb => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:47.417045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7152e-8208-43f7-86ee-e568bbc2add1 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:47.519732 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:47.546323 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7152e-8208-43f7-86ee-e568bbc2add1 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:47.587646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973c6f9-8225-4c2b-ba0e-f831fa64d3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:47.588622 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973c6f9-8225-4c2b-ba0e-f831fa64d3b0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:47.588919 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973c6f9-8225-4c2b-ba0e-f831fa64d3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:47.589206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973c6f9-8225-4c2b-ba0e-f831fa64d3b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:47.590695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973c6f9-8225-4c2b-ba0e-f831fa64d3b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:47.592272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9973c6f9-8225-4c2b-ba0e-f831fa64d3b0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:47.594153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 31/52] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:27:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:47.603397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b2494-74a8-4e24-a5fb-4265bd58db6d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:47.691192 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a9199-223e-4eb9-bad4-833efeba74ca None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:47.691438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a9199-223e-4eb9-bad4-833efeba74ca None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:47.691728 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a9199-223e-4eb9-bad4-833efeba74ca None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:47.693175 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 32/53] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:47.698716 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66b23-7cdb-4404-9546-84dfd7c627b0 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:47.699101 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66b23-7cdb-4404-9546-84dfd7c627b0 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:47.699458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66b23-7cdb-4404-9546-84dfd7c627b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:47.716925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:47.722529 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-52f66b23-7cdb-4404-9546-84dfd7c627b0 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:47.723537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-52f66b23-7cdb-4404-9546-84dfd7c627b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:47.766547 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1b2494-74a8-4e24-a5fb-4265bd58db6d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:47.806920 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311b40a-94e4-481a-9a9f-0a84931a2ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:47.807612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311b40a-94e4-481a-9a9f-0a84931a2ebf None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:47.807878 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311b40a-94e4-481a-9a9f-0a84931a2ebf None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:47.808157 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311b40a-94e4-481a-9a9f-0a84931a2ebf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:47.809045 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311b40a-94e4-481a-9a9f-0a84931a2ebf None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.domain.id=default, user.id=cc336177e5d04523af605ab7e66532eb) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:47.809371 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311b40a-94e4-481a-9a9f-0a84931a2ebf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:47.810855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311b40a-94e4-481a-9a9f-0a84931a2ebf None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:47.813021 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 33/54] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:27:47.820188 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 22/55] 10.222.0.88 () {56 vars in 1322 bytes} [Thu May 19 20:27:47 2022] GET /identity/v3/role_assignments?user.id=cc336177e5d04523af605ab7e66532eb&scope.domain.id=default&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 530 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:47.837651 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f66b23-7cdb-4404-9546-84dfd7c627b0 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:47.935451 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:47.970899 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f66b23-7cdb-4404-9546-84dfd7c627b0 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:48.016680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a6380-2e77-4085-9918-27329b611693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:48.017508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a6380-2e77-4085-9918-27329b611693 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:48.017799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a6380-2e77-4085-9918-27329b611693 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:48.018082 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a6380-2e77-4085-9918-27329b611693 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:48.019889 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a6380-2e77-4085-9918-27329b611693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:48.021405 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a6380-2e77-4085-9918-27329b611693 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:48.023832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 34/56] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:27:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:48.031522 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311b40a-94e4-481a-9a9f-0a84931a2ebf None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:48.161058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311b40a-94e4-481a-9a9f-0a84931a2ebf None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:48.192020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:48.198354 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aefd09-265a-4744-a2c2-083ebce567d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:48.199041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aefd09-265a-4744-a2c2-083ebce567d7 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:48.199334 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aefd09-265a-4744-a2c2-083ebce567d7 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:48.199624 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aefd09-265a-4744-a2c2-083ebce567d7 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:48.200396 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aefd09-265a-4744-a2c2-083ebce567d7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:48.201903 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aefd09-265a-4744-a2c2-083ebce567d7 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:48.312490 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 23/57] 10.222.0.88 () {60 vars in 1140 bytes} [Thu May 19 20:27:48 2022] POST /identity/v3/projects => generated 298 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:27:50.626948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a433b6-d54f-4d1e-9b89-8a30a176b4cf None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:50.627484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a433b6-d54f-4d1e-9b89-8a30a176b4cf None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:50.627624 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a433b6-d54f-4d1e-9b89-8a30a176b4cf None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:50.628958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 35/58] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:50.634357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de42f2-7ea4-42c2-bcc4-ad7e269e0e8a None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:50.634835 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de42f2-7ea4-42c2-bcc4-ad7e269e0e8a None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:50.634949 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de42f2-7ea4-42c2-bcc4-ad7e269e0e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:50.651881 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-28de42f2-7ea4-42c2-bcc4-ad7e269e0e8a None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:50.653158 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-28de42f2-7ea4-42c2-bcc4-ad7e269e0e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:50.721883 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 24/59] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:50.747000 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a433b6-d54f-4d1e-9b89-8a30a176b4cf None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:50.852793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a433b6-d54f-4d1e-9b89-8a30a176b4cf None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:50.887494 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be2e0c-2a2a-4310-9a20-647e488fe8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:50.888390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be2e0c-2a2a-4310-9a20-647e488fe8f9 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:50.888679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be2e0c-2a2a-4310-9a20-647e488fe8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:50.888956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be2e0c-2a2a-4310-9a20-647e488fe8f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:50.890185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be2e0c-2a2a-4310-9a20-647e488fe8f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:50.891649 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be2e0c-2a2a-4310-9a20-647e488fe8f9 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:50.893361 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 36/60] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:27:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:50.899875 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de42f2-7ea4-42c2-bcc4-ad7e269e0e8a None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:51.008683 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de42f2-7ea4-42c2-bcc4-ad7e269e0e8a None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:51.023043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:51.046126 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e16c3-57ed-4429-aa63-c13e41781cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:51.046848 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e16c3-57ed-4429-aa63-c13e41781cfb None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:51.047134 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e16c3-57ed-4429-aa63-c13e41781cfb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:51.047410 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e16c3-57ed-4429-aa63-c13e41781cfb None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:51.048130 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e16c3-57ed-4429-aa63-c13e41781cfb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:51.049608 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e16c3-57ed-4429-aa63-c13e41781cfb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:51.169580 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 25/61] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:27:50 2022] POST /identity/v3/users => generated 275 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:27:54.680513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e5e0-2a7f-4524-a3a2-3d7a27b52d02 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:54.681472 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e5e0-2a7f-4524-a3a2-3d7a27b52d02 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:54.681918 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e5e0-2a7f-4524-a3a2-3d7a27b52d02 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:54.683503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 37/62] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:54.689649 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38de957-4118-456b-aba5-1ea9c02a0d7a None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:54.690547 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38de957-4118-456b-aba5-1ea9c02a0d7a None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:54.691002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38de957-4118-456b-aba5-1ea9c02a0d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:54.722723 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-a38de957-4118-456b-aba5-1ea9c02a0d7a None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:54.724198 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-a38de957-4118-456b-aba5-1ea9c02a0d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:54.822926 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 26/63] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:27:54.867255 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8351f4-f1dd-4d0b-b5c5-4d0e10b73cd0 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:54.867839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8351f4-f1dd-4d0b-b5c5-4d0e10b73cd0 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:54.868300 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8351f4-f1dd-4d0b-b5c5-4d0e10b73cd0 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:54.869935 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 38/64] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:27:54.875971 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5f0a9-1aaa-4004-9729-fbdb58b2f34a None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:54.876548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5f0a9-1aaa-4004-9729-fbdb58b2f34a None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:54.877011 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5f0a9-1aaa-4004-9729-fbdb58b2f34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:54.885736 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8351f4-f1dd-4d0b-b5c5-4d0e10b73cd0 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:54.902853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:54.922273 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-bfc5f0a9-1aaa-4004-9729-fbdb58b2f34a None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:54.923543 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-bfc5f0a9-1aaa-4004-9729-fbdb58b2f34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:55.057818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 27/65] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:27:55.082168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8351f4-f1dd-4d0b-b5c5-4d0e10b73cd0 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:55.113986 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5f0a9-1aaa-4004-9729-fbdb58b2f34a None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:55.137312 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce7b89-e167-468b-ad3d-ca20e0cc6847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:55.138151 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce7b89-e167-468b-ad3d-ca20e0cc6847 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.138440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce7b89-e167-468b-ad3d-ca20e0cc6847 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.138772 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce7b89-e167-468b-ad3d-ca20e0cc6847 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.160474 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce7b89-e167-468b-ad3d-ca20e0cc6847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:55.168910 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-e9ce7b89-e167-468b-ad3d-ca20e0cc6847 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:55.170895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:55.172061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:27:55.173065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:27:55.175346 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 39/66] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:27:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:55.181390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce7b89-e167-468b-ad3d-ca20e0cc6847 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:55.278878 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5f0a9-1aaa-4004-9729-fbdb58b2f34a None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:55.320109 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688b61f-197a-4f14-97c3-64e5ed8b43c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:55.321232 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688b61f-197a-4f14-97c3-64e5ed8b43c0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.321689 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688b61f-197a-4f14-97c3-64e5ed8b43c0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.322129 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688b61f-197a-4f14-97c3-64e5ed8b43c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.323484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce7b89-e167-468b-ad3d-ca20e0cc6847 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:55.333930 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688b61f-197a-4f14-97c3-64e5ed8b43c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-4688b61f-197a-4f14-97c3-64e5ed8b43c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:55.342735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:55.343843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:27:55.344783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:27:55.347347 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 28/67] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:27:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:55.353185 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e99db2-1cd0-460f-8ae8-3da5b79abec0 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.353757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e99db2-1cd0-460f-8ae8-3da5b79abec0 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.354249 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e99db2-1cd0-460f-8ae8-3da5b79abec0 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.355892 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 29/68] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:55.357043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:55.358141 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e99db2-1cd0-460f-8ae8-3da5b79abec0 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:55.369355 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bea5e4-8389-4bc8-8f83-6a1cfc4570e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:55.370113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bea5e4-8389-4bc8-8f83-6a1cfc4570e2 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.370402 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bea5e4-8389-4bc8-8f83-6a1cfc4570e2 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.370682 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bea5e4-8389-4bc8-8f83-6a1cfc4570e2 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.371912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bea5e4-8389-4bc8-8f83-6a1cfc4570e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:55.372273 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bea5e4-8389-4bc8-8f83-6a1cfc4570e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:55.374097 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bea5e4-8389-4bc8-8f83-6a1cfc4570e2 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:55.391248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 40/69] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:27:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:27:55.391248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fdea3-c08d-4fcc-9861-8cb7ce33af43 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.391248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fdea3-c08d-4fcc-9861-8cb7ce33af43 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.391248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fdea3-c08d-4fcc-9861-8cb7ce33af43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.408836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-755fdea3-c08d-4fcc-9861-8cb7ce33af43 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:55.410058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-755fdea3-c08d-4fcc-9861-8cb7ce33af43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:55.474296 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:55.488052 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e99db2-1cd0-460f-8ae8-3da5b79abec0 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:55.490196 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 41/70] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:55.498153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fdea3-c08d-4fcc-9861-8cb7ce33af43 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:55.533631 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e377d-02f0-4f05-9374-77940401c987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:55.534478 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e377d-02f0-4f05-9374-77940401c987 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.534879 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e377d-02f0-4f05-9374-77940401c987 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.535169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e377d-02f0-4f05-9374-77940401c987 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.536097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e377d-02f0-4f05-9374-77940401c987 None admin] RBAC: Adding query filter params (name=member) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:55.536444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e377d-02f0-4f05-9374-77940401c987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:55.538213 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e377d-02f0-4f05-9374-77940401c987 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:55.548673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 30/71] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:27:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:55.550038 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c305e-ba67-4cf8-9a67-03c0557fa017 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.550418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c305e-ba67-4cf8-9a67-03c0557fa017 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.550613 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c305e-ba67-4cf8-9a67-03c0557fa017 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.556643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 31/72] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:55.557913 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c305e-ba67-4cf8-9a67-03c0557fa017 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:55.577306 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:55.621968 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fdea3-c08d-4fcc-9861-8cb7ce33af43 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:55.657888 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d9ebb-2ce7-4176-9961-4afa74a209a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:55.658780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d9ebb-2ce7-4176-9961-4afa74a209a2 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.659071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d9ebb-2ce7-4176-9961-4afa74a209a2 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.659359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d9ebb-2ce7-4176-9961-4afa74a209a2 None admin] PATH_INFO: `/v3/users/cc336177e5d04523af605ab7e66532eb` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.660640 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d9ebb-2ce7-4176-9961-4afa74a209a2 None admin] RBAC: Authorizing `identity:get_user(user_id=cc336177e5d04523af605ab7e66532eb)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:55.662131 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d9ebb-2ce7-4176-9961-4afa74a209a2 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:55.669045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 42/73] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:27:55 2022] GET /identity/v3/users/cc336177e5d04523af605ab7e66532eb => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:55.669045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d9ebb-2ce7-4176-9961-4afa74a209a2 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:55.679928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c305e-ba67-4cf8-9a67-03c0557fa017 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:55.717284 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6e8db-ccff-40c2-8bd2-702a46d2c2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:55.718110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6e8db-ccff-40c2-8bd2-702a46d2c2dd None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.718400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6e8db-ccff-40c2-8bd2-702a46d2c2dd None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.718682 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6e8db-ccff-40c2-8bd2-702a46d2c2dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.729473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6e8db-ccff-40c2-8bd2-702a46d2c2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-c1a6e8db-ccff-40c2-8bd2-702a46d2c2dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:55.737621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:55.738796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:27:55.739933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 32/74] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:27:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:55.741308 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6af802-ea6a-4702-a1d6-174e6dedd67d None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.741308 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6af802-ea6a-4702-a1d6-174e6dedd67d None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.741463 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6af802-ea6a-4702-a1d6-174e6dedd67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.758744 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-ad6af802-ea6a-4702-a1d6-174e6dedd67d None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:55.759967 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-ad6af802-ea6a-4702-a1d6-174e6dedd67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:55.762883 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:55.774652 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d9ebb-2ce7-4176-9961-4afa74a209a2 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:55.812313 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:55.818367 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67111f-8121-4e56-a288-0342e88f2754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:55.819786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67111f-8121-4e56-a288-0342e88f2754 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.820069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67111f-8121-4e56-a288-0342e88f2754 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.820420 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67111f-8121-4e56-a288-0342e88f2754 None admin] PATH_INFO: `/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.832382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 33/75] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:55.833187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6af802-ea6a-4702-a1d6-174e6dedd67d None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:55.834419 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67111f-8121-4e56-a288-0342e88f2754 None admin] RBAC: Authorizing `identity:get_user(user_id=ff96da569f6448c9b0a3b1aebf6f10e0)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:55.835897 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67111f-8121-4e56-a288-0342e88f2754 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:55.837921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 43/76] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:27:55 2022] GET /identity/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:55.839489 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4372b-91d2-47c6-a374-4b8915eb777c None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.839489 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4372b-91d2-47c6-a374-4b8915eb777c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.839740 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4372b-91d2-47c6-a374-4b8915eb777c None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.841868 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 44/77] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:55.842016 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4372b-91d2-47c6-a374-4b8915eb777c None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:55.945324 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6af802-ea6a-4702-a1d6-174e6dedd67d None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:55.956504 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4372b-91d2-47c6-a374-4b8915eb777c None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:55.979970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50b730-8a4c-4b9a-a1a5-316849ef9222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:55.980678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b730-8a4c-4b9a-a1a5-316849ef9222 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.980966 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b730-8a4c-4b9a-a1a5-316849ef9222 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.981249 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b730-8a4c-4b9a-a1a5-316849ef9222 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.993290 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:55.995586 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f40713-7961-4ba8-9531-b12051ca8105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:55.996266 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f40713-7961-4ba8-9531-b12051ca8105 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:55.996583 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f40713-7961-4ba8-9531-b12051ca8105 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:55.996862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f40713-7961-4ba8-9531-b12051ca8105 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:55.997700 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f40713-7961-4ba8-9531-b12051ca8105 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:55.998010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f40713-7961-4ba8-9531-b12051ca8105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:55.999464 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f40713-7961-4ba8-9531-b12051ca8105 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:56.008351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 45/78] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:27:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:56.009312 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f40713-7961-4ba8-9531-b12051ca8105 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:56.057836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50b730-8a4c-4b9a-a1a5-316849ef9222 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:56.059343 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50b730-8a4c-4b9a-a1a5-316849ef9222 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:56.063146 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:56.116908 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f40713-7961-4ba8-9531-b12051ca8105 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:56.150967 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e2002-578e-40ea-8ec5-7a7159551c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:56.151639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e2002-578e-40ea-8ec5-7a7159551c68 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:56.151990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e2002-578e-40ea-8ec5-7a7159551c68 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:56.152470 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e2002-578e-40ea-8ec5-7a7159551c68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:56.166661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 34/79] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:27:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:27:56.169231 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac9219-dd45-481a-9a6a-7f3b42ac5727 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:56.169753 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac9219-dd45-481a-9a6a-7f3b42ac5727 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:56.170243 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac9219-dd45-481a-9a6a-7f3b42ac5727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:56.188532 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-fbac9219-dd45-481a-9a6a-7f3b42ac5727 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:56.189893 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-fbac9219-dd45-481a-9a6a-7f3b42ac5727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:56.228397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:56.236875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e2002-578e-40ea-8ec5-7a7159551c68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:56.238378 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e2002-578e-40ea-8ec5-7a7159551c68 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:56.263082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 35/80] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:56.263987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac9219-dd45-481a-9a6a-7f3b42ac5727 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:56.298325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:56.373518 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 46/81] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:27:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:27:56.374304 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e2002-578e-40ea-8ec5-7a7159551c68 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:56.380887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac9219-dd45-481a-9a6a-7f3b42ac5727 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:56.420618 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72129ef5-3e21-4eb6-a4a4-8ced580553b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:56.421476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72129ef5-3e21-4eb6-a4a4-8ced580553b2 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:56.421760 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72129ef5-3e21-4eb6-a4a4-8ced580553b2 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:56.422035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72129ef5-3e21-4eb6-a4a4-8ced580553b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:56.432789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72129ef5-3e21-4eb6-a4a4-8ced580553b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-72129ef5-3e21-4eb6-a4a4-8ced580553b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:27:56.440717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:27:56.441759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:27:56.442664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 36/82] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:27:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:27:56.443502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72129ef5-3e21-4eb6-a4a4-8ced580553b2 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:56.477458 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:56.494535 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e2002-578e-40ea-8ec5-7a7159551c68 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:56.531270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cee02-ce2a-4472-a21a-995f4127a83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:56.532145 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cee02-ce2a-4472-a21a-995f4127a83d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:56.532457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cee02-ce2a-4472-a21a-995f4127a83d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:56.532740 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9cee02-ce2a-4472-a21a-995f4127a83d None admin] PATH_INFO: `/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:56.533956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cee02-ce2a-4472-a21a-995f4127a83d None admin] RBAC: Authorizing `identity:get_user(user_id=ff96da569f6448c9b0a3b1aebf6f10e0)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:56.535973 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9cee02-ce2a-4472-a21a-995f4127a83d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:56.538403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 47/83] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:27:56 2022] GET /identity/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:56.539001 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cee02-ce2a-4472-a21a-995f4127a83d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:56.553746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72129ef5-3e21-4eb6-a4a4-8ced580553b2 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:56.558467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:56.588622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2d8b9-a4ed-4e26-abb5-d9eee06efe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:56.589491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2d8b9-a4ed-4e26-abb5-d9eee06efe51 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:56.589778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2d8b9-a4ed-4e26-abb5-d9eee06efe51 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:56.590054 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2d8b9-a4ed-4e26-abb5-d9eee06efe51 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:56.591402 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2d8b9-a4ed-4e26-abb5-d9eee06efe51 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:56.592978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2d8b9-a4ed-4e26-abb5-d9eee06efe51 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:56.594687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 37/84] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:27:56 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:56.595545 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2d8b9-a4ed-4e26-abb5-d9eee06efe51 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:56.641328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9cee02-ce2a-4472-a21a-995f4127a83d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:56.681175 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2f19a-1d98-4cb5-afd8-49de622898f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:56.681975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2f19a-1d98-4cb5-afd8-49de622898f6 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:56.682257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2f19a-1d98-4cb5-afd8-49de622898f6 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:56.682531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2f19a-1d98-4cb5-afd8-49de622898f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:56.683908 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2f19a-1d98-4cb5-afd8-49de622898f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:56.685514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2f19a-1d98-4cb5-afd8-49de622898f6 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:56.687428 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 48/85] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:27:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:56.688027 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2f19a-1d98-4cb5-afd8-49de622898f6 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:56.700986 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:56.704843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2d8b9-a4ed-4e26-abb5-d9eee06efe51 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:56.739399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e864c52-2202-4bec-8a3b-1fcbbba87545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:56.740310 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e864c52-2202-4bec-8a3b-1fcbbba87545 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:56.740728 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e864c52-2202-4bec-8a3b-1fcbbba87545 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:56.741333 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e864c52-2202-4bec-8a3b-1fcbbba87545 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:56.742904 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e864c52-2202-4bec-8a3b-1fcbbba87545 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:56.744908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e864c52-2202-4bec-8a3b-1fcbbba87545 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:56.746641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 38/86] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:27:56 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:56.747416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e864c52-2202-4bec-8a3b-1fcbbba87545 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:56.784255 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:56.803069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2f19a-1d98-4cb5-afd8-49de622898f6 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:56.839871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4a593-ec34-4c80-b5ed-4dd4ae5d7107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:56.840578 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4a593-ec34-4c80-b5ed-4dd4ae5d7107 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:56.841020 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4a593-ec34-4c80-b5ed-4dd4ae5d7107 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:56.841250 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4a593-ec34-4c80-b5ed-4dd4ae5d7107 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:56.841881 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4a593-ec34-4c80-b5ed-4dd4ae5d7107 None admin] RBAC: Adding query filter params (name=member) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:56.842189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4a593-ec34-4c80-b5ed-4dd4ae5d7107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:56.843577 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4a593-ec34-4c80-b5ed-4dd4ae5d7107 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:56.851489 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e864c52-2202-4bec-8a3b-1fcbbba87545 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:56.852022 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 49/87] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:27:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:56.857218 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4a593-ec34-4c80-b5ed-4dd4ae5d7107 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:56.888636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fd188-a66d-408d-b39f-824a65145049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:56.889300 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fd188-a66d-408d-b39f-824a65145049 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:56.889585 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fd188-a66d-408d-b39f-824a65145049 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:56.889935 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fd188-a66d-408d-b39f-824a65145049 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:56.924123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:56.958327 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fd188-a66d-408d-b39f-824a65145049 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:56.960450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49fd188-a66d-408d-b39f-824a65145049 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:56.964252 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4a593-ec34-4c80-b5ed-4dd4ae5d7107 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:56.998279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08085f7-0798-49ad-8aa9-5b5259e7a621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:56.998947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08085f7-0798-49ad-8aa9-5b5259e7a621 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:56.999232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08085f7-0798-49ad-8aa9-5b5259e7a621 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:56.999510 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08085f7-0798-49ad-8aa9-5b5259e7a621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:57.000430 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08085f7-0798-49ad-8aa9-5b5259e7a621 None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=464c75ba6b6340e4b017049e6993fe4d, user.id=cc336177e5d04523af605ab7e66532eb) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:57.000752 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08085f7-0798-49ad-8aa9-5b5259e7a621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:57.002190 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08085f7-0798-49ad-8aa9-5b5259e7a621 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:57.008237 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 50/88] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:27:56 2022] GET /identity/v3/role_assignments?user.id=cc336177e5d04523af605ab7e66532eb&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:57.009041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08085f7-0798-49ad-8aa9-5b5259e7a621 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:57.026565 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:57.083531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 39/89] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:27:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:27:57.084460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fd188-a66d-408d-b39f-824a65145049 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:57.138379 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08085f7-0798-49ad-8aa9-5b5259e7a621 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:57.163147 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:57.182144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08887dd9-cfea-450f-92e8-0683de685f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:57.182909 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887dd9-cfea-450f-92e8-0683de685f82 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:57.183034 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887dd9-cfea-450f-92e8-0683de685f82 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:57.183315 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08887dd9-cfea-450f-92e8-0683de685f82 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:57.184058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08887dd9-cfea-450f-92e8-0683de685f82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:57.185632 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08887dd9-cfea-450f-92e8-0683de685f82 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:57.202597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fd188-a66d-408d-b39f-824a65145049 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:57.240024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292e1ac-4e07-4184-8057-5d58957c68e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:57.241308 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292e1ac-4e07-4184-8057-5d58957c68e2 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:57.241598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292e1ac-4e07-4184-8057-5d58957c68e2 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:57.241881 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292e1ac-4e07-4184-8057-5d58957c68e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:57.242738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292e1ac-4e07-4184-8057-5d58957c68e2 None admin] RBAC: Adding query filter params (role.id=c8be6a897df546ca9b0e7b9541fb94d5, scope.project.id=464c75ba6b6340e4b017049e6993fe4d, user.id=ff96da569f6448c9b0a3b1aebf6f10e0) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:57.243059 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292e1ac-4e07-4184-8057-5d58957c68e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:57.244548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292e1ac-4e07-4184-8057-5d58957c68e2 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:57.250842 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 40/90] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:27:57 2022] GET /identity/v3/role_assignments?user.id=ff96da569f6448c9b0a3b1aebf6f10e0&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:57.251731 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292e1ac-4e07-4184-8057-5d58957c68e2 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:57.336942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 51/91] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:27:57 2022] POST /identity/v3/users => generated 289 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:27:57.338409 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08887dd9-cfea-450f-92e8-0683de685f82 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:57.400772 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292e1ac-4e07-4184-8057-5d58957c68e2 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:57.440670 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:57.448919 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78a11e-c68b-441a-9f43-40fb303040e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:57.450275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78a11e-c68b-441a-9f43-40fb303040e6 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:57.450949 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78a11e-c68b-441a-9f43-40fb303040e6 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:57.451463 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78a11e-c68b-441a-9f43-40fb303040e6 None admin] PATH_INFO: `/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:57.453259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78a11e-c68b-441a-9f43-40fb303040e6 None admin] RBAC: Authorizing `identity:get_user(user_id=ff96da569f6448c9b0a3b1aebf6f10e0)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:57.455231 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78a11e-c68b-441a-9f43-40fb303040e6 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:57.462078 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 41/92] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:27:57 2022] GET /identity/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:57.469427 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78a11e-c68b-441a-9f43-40fb303040e6 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:57.496770 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:57.593037 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08887dd9-cfea-450f-92e8-0683de685f82 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:57.637227 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78a11e-c68b-441a-9f43-40fb303040e6 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:57.643780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39f4ea-8ef0-489c-8bdc-9114e943eeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:57.644806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39f4ea-8ef0-489c-8bdc-9114e943eeee None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:57.645174 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39f4ea-8ef0-489c-8bdc-9114e943eeee None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:57.645534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39f4ea-8ef0-489c-8bdc-9114e943eeee None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:57.646982 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39f4ea-8ef0-489c-8bdc-9114e943eeee None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:57.648681 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39f4ea-8ef0-489c-8bdc-9114e943eeee None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:57.650760 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 52/93] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:27:57 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:57.656748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39f4ea-8ef0-489c-8bdc-9114e943eeee None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:57.706687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d03ff0-49fe-4f13-871a-19736a5b5992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:57.707407 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d03ff0-49fe-4f13-871a-19736a5b5992 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:57.707772 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d03ff0-49fe-4f13-871a-19736a5b5992 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:57.708133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d03ff0-49fe-4f13-871a-19736a5b5992 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:57.787254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:57.791402 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39f4ea-8ef0-489c-8bdc-9114e943eeee None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:57.791856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d03ff0-49fe-4f13-871a-19736a5b5992 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:57.793927 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d03ff0-49fe-4f13-871a-19736a5b5992 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:57.830278 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee832e5-d080-4d32-87cf-13c1ac687055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:57.830964 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee832e5-d080-4d32-87cf-13c1ac687055 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:57.831257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee832e5-d080-4d32-87cf-13c1ac687055 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:57.831567 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee832e5-d080-4d32-87cf-13c1ac687055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:57.832650 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee832e5-d080-4d32-87cf-13c1ac687055 None admin] RBAC: Adding query filter params (role.id=ff499a8782db4cce8e162a849be212ee, scope.project.id=464c75ba6b6340e4b017049e6993fe4d, user.id=ff96da569f6448c9b0a3b1aebf6f10e0) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:57.832981 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee832e5-d080-4d32-87cf-13c1ac687055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:57.835026 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee832e5-d080-4d32-87cf-13c1ac687055 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:57.838475 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:57.842300 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 53/94] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:27:57 2022] GET /identity/v3/role_assignments?user.id=ff96da569f6448c9b0a3b1aebf6f10e0&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=ff499a8782db4cce8e162a849be212ee => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:27:57.925151 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 42/95] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:27:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:27:57.931146 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee832e5-d080-4d32-87cf-13c1ac687055 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:58.077194 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee832e5-d080-4d32-87cf-13c1ac687055 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:58.119016 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc4034-648a-49be-9a95-ad4b3c2906ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:58.119913 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc4034-648a-49be-9a95-ad4b3c2906ee None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:58.120196 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc4034-648a-49be-9a95-ad4b3c2906ee None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:58.120503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc4034-648a-49be-9a95-ad4b3c2906ee None admin] PATH_INFO: `/v3/projects/6f73be14e6644cbf87eb6b64d960dcaa` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:58.121775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc4034-648a-49be-9a95-ad4b3c2906ee None admin] RBAC: Authorizing `identity:get_project(project_id=6f73be14e6644cbf87eb6b64d960dcaa)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:58.123361 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc4034-648a-49be-9a95-ad4b3c2906ee None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:58.125388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 54/96] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:27:57 2022] GET /identity/v3/projects/6f73be14e6644cbf87eb6b64d960dcaa => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:58.131957 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d03ff0-49fe-4f13-871a-19736a5b5992 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:27:58.252995 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:27:58.256744 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d03ff0-49fe-4f13-871a-19736a5b5992 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:27:58.298315 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4b5b6b-c5ab-463b-b84d-080edbe78041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:27:58.299187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b5b6b-c5ab-463b-b84d-080edbe78041 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:58.299851 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b5b6b-c5ab-463b-b84d-080edbe78041 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:58.300139 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4b5b6b-c5ab-463b-b84d-080edbe78041 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:58.301029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b5b6b-c5ab-463b-b84d-080edbe78041 None admin] RBAC: Adding query filter params (role.id=c8be6a897df546ca9b0e7b9541fb94d5, scope.project.id=6f73be14e6644cbf87eb6b64d960dcaa, user.id=ff96da569f6448c9b0a3b1aebf6f10e0) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:27:58.301353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b5b6b-c5ab-463b-b84d-080edbe78041 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:27:58.302818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4b5b6b-c5ab-463b-b84d-080edbe78041 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:27:58.313128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 43/97] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:27:58 2022] GET /identity/v3/role_assignments?user.id=ff96da569f6448c9b0a3b1aebf6f10e0&scope.project.id=6f73be14e6644cbf87eb6b64d960dcaa&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:27:59.796464 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9f0e4-9b91-475b-8595-cd8500ed5553 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:59.797621 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9f0e4-9b91-475b-8595-cd8500ed5553 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:59.798222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9f0e4-9b91-475b-8595-cd8500ed5553 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:59.799967 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 55/98] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:27:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:27:59.805792 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f5e94-e18a-412f-a44f-3b0e5ff87ed5 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:27:59.806714 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f5e94-e18a-412f-a44f-3b0e5ff87ed5 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:27:59.807256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f5e94-e18a-412f-a44f-3b0e5ff87ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:27:59.829383 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-c99f5e94-e18a-412f-a44f-3b0e5ff87ed5 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:27:59.830962 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-c99f5e94-e18a-412f-a44f-3b0e5ff87ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:27:59.922727 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 44/99] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:27:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:27:59.952125 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9f0e4-9b91-475b-8595-cd8500ed5553 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:00.051052 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:00.095893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9f0e4-9b91-475b-8595-cd8500ed5553 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:00.141392 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d7a16-6a2d-4706-993d-19a3759121b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:00.142209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7a16-6a2d-4706-993d-19a3759121b7 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:00.142498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7a16-6a2d-4706-993d-19a3759121b7 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:00.142780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d7a16-6a2d-4706-993d-19a3759121b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:00.154689 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d7a16-6a2d-4706-993d-19a3759121b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-0e0d7a16-6a2d-4706-993d-19a3759121b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:00.162639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:00.163669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:00.164797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 56/100] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:27:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:00.170145 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f5e94-e18a-412f-a44f-3b0e5ff87ed5 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:00.319251 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:00.327969 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f5e94-e18a-412f-a44f-3b0e5ff87ed5 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:00.378092 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850ee361-bff2-4f31-9257-de71d8c4a560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:00.378781 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ee361-bff2-4f31-9257-de71d8c4a560 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:00.379070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ee361-bff2-4f31-9257-de71d8c4a560 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:00.379349 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ee361-bff2-4f31-9257-de71d8c4a560 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:00.380112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ee361-bff2-4f31-9257-de71d8c4a560 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:00.380485 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ee361-bff2-4f31-9257-de71d8c4a560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:00.382112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850ee361-bff2-4f31-9257-de71d8c4a560 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:00.396563 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 45/101] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:00.402499 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d7a16-6a2d-4706-993d-19a3759121b7 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:00.629746 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d7a16-6a2d-4706-993d-19a3759121b7 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:00.631167 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f781b-0059-4e1b-93a2-d40698b34563 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:00.631464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f781b-0059-4e1b-93a2-d40698b34563 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:00.631745 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f781b-0059-4e1b-93a2-d40698b34563 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:00.633189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 46/102] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:28:00.639776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bb214-87eb-4106-9feb-553b9bc3bef3 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:00.640092 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bb214-87eb-4106-9feb-553b9bc3bef3 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:00.640380 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9bb214-87eb-4106-9feb-553b9bc3bef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:00.666387 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-5f9bb214-87eb-4106-9feb-553b9bc3bef3 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:00.667850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-5f9bb214-87eb-4106-9feb-553b9bc3bef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:00.675908 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:00.698070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f09524-51f9-4528-b605-7a9055c05b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:00.699404 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f09524-51f9-4528-b605-7a9055c05b72 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:00.699909 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f09524-51f9-4528-b605-7a9055c05b72 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:00.700434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f09524-51f9-4528-b605-7a9055c05b72 None admin] PATH_INFO: `/v3/users/cc336177e5d04523af605ab7e66532eb` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:00.701981 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f09524-51f9-4528-b605-7a9055c05b72 None admin] RBAC: Authorizing `identity:get_user(user_id=cc336177e5d04523af605ab7e66532eb)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:00.703735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f09524-51f9-4528-b605-7a9055c05b72 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:00.706021 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 57/103] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:00 2022] GET /identity/v3/users/cc336177e5d04523af605ab7e66532eb => generated 247 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:00.712560 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f09524-51f9-4528-b605-7a9055c05b72 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:00.849160 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 47/104] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:00.886504 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bb214-87eb-4106-9feb-553b9bc3bef3 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:00.922158 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f09524-51f9-4528-b605-7a9055c05b72 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:01.003054 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda339c-4408-4316-82be-594aa2624110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:01.003948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda339c-4408-4316-82be-594aa2624110 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.004234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda339c-4408-4316-82be-594aa2624110 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.004548 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda339c-4408-4316-82be-594aa2624110 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.005793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda339c-4408-4316-82be-594aa2624110 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:01.007308 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda339c-4408-4316-82be-594aa2624110 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:01.009085 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 58/105] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:00 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:01.015316 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda339c-4408-4316-82be-594aa2624110 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:01.091472 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:01.116730 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9bb214-87eb-4106-9feb-553b9bc3bef3 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:01.168703 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5820f-4b98-4d92-91ec-9d331d18929e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:01.169439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5820f-4b98-4d92-91ec-9d331d18929e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.169727 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5820f-4b98-4d92-91ec-9d331d18929e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.170007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5820f-4b98-4d92-91ec-9d331d18929e None admin] PATH_INFO: `/v3/roles/member` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.170883 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:01.196805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5820f-4b98-4d92-91ec-9d331d18929e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-d4f5820f-4b98-4d92-91ec-9d331d18929e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:01.204940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:01.213442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:01.214359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 48/106] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:01.221574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98dd6-988c-4c63-bd7f-f9d0ae554bf6 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.221874 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98dd6-988c-4c63-bd7f-f9d0ae554bf6 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.222133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd98dd6-988c-4c63-bd7f-f9d0ae554bf6 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.223860 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 49/107] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:01.230817 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd98dd6-988c-4c63-bd7f-f9d0ae554bf6 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:01.259902 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda339c-4408-4316-82be-594aa2624110 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:01.345920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ed277-957f-4716-bbda-ce0196e3a119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:01.347561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed277-957f-4716-bbda-ce0196e3a119 None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.348244 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed277-957f-4716-bbda-ce0196e3a119 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.348810 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed277-957f-4716-bbda-ce0196e3a119 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d/users/cc336177e5d04523af605ab7e66532eb/roles/8eaff9cd613a49b29548e90d7e3e9215` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.360944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ed277-957f-4716-bbda-ce0196e3a119 None admin] RBAC: Authorizing `identity:create_grant(project_id=464c75ba6b6340e4b017049e6993fe4d, user_id=cc336177e5d04523af605ab7e66532eb, role_id=8eaff9cd613a49b29548e90d7e3e9215)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:01.363145 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ed277-957f-4716-bbda-ce0196e3a119 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:01.406727 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd98dd6-988c-4c63-bd7f-f9d0ae554bf6 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:01.452831 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbccd0-3c10-417c-b604-83762db9fcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:01.453546 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbccd0-3c10-417c-b604-83762db9fcc9 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.453862 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbccd0-3c10-417c-b604-83762db9fcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.454151 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbccd0-3c10-417c-b604-83762db9fcc9 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.454920 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbccd0-3c10-417c-b604-83762db9fcc9 None admin] RBAC: Adding query filter params (name=member) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:01.455287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbccd0-3c10-417c-b604-83762db9fcc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:01.456788 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbccd0-3c10-417c-b604-83762db9fcc9 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:01.461879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 59/108] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:28:01 2022] PUT /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d/users/cc336177e5d04523af605ab7e66532eb/roles/8eaff9cd613a49b29548e90d7e3e9215 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:28:01.463040 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f60a-a21f-4427-b9fc-0f1e7b132db3 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.463488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f60a-a21f-4427-b9fc-0f1e7b132db3 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.463783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0f60a-a21f-4427-b9fc-0f1e7b132db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.468557 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 50/109] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:01.478627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbccd0-3c10-417c-b604-83762db9fcc9 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:01.492801 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:01.505022 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-6bf0f60a-a21f-4427-b9fc-0f1e7b132db3 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:01.510900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-6bf0f60a-a21f-4427-b9fc-0f1e7b132db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:01.591359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 60/110] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:01.593139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caece64b-82d2-45ea-a856-9922bd80bf37 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.593563 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caece64b-82d2-45ea-a856-9922bd80bf37 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.593946 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caece64b-82d2-45ea-a856-9922bd80bf37 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.595748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 61/111] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:01.601015 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a82c00-43ff-4ecb-b36e-dfa94960ace6 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.601432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a82c00-43ff-4ecb-b36e-dfa94960ace6 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.601827 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a82c00-43ff-4ecb-b36e-dfa94960ace6 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.603613 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 62/112] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:01.604965 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aaad3-312c-4114-acc5-5b4f47116853 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.605261 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbccd0-3c10-417c-b604-83762db9fcc9 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:01.605447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aaad3-312c-4114-acc5-5b4f47116853 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.605833 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2aaad3-312c-4114-acc5-5b4f47116853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.625127 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-ff2aaad3-312c-4114-acc5-5b4f47116853 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:01.626504 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-ff2aaad3-312c-4114-acc5-5b4f47116853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:01.647972 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50496826-08dc-4b61-8775-8674031154a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:01.648991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50496826-08dc-4b61-8775-8674031154a0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.649282 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50496826-08dc-4b61-8775-8674031154a0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.649563 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50496826-08dc-4b61-8775-8674031154a0 None admin] PATH_INFO: `/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.651068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50496826-08dc-4b61-8775-8674031154a0 None admin] RBAC: Authorizing `identity:get_user(user_id=ff96da569f6448c9b0a3b1aebf6f10e0)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:01.653131 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50496826-08dc-4b61-8775-8674031154a0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:01.655957 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 51/113] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:01 2022] GET /identity/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:01.657693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ad495-e461-4d55-bbe4-aec83270536c None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.658041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ad495-e461-4d55-bbe4-aec83270536c None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.658380 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ad495-e461-4d55-bbe4-aec83270536c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.680120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-c05ad495-e461-4d55-bbe4-aec83270536c None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:01.681693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-c05ad495-e461-4d55-bbe4-aec83270536c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:01.698134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:01.718952 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 63/114] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:01.724947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2aaad3-312c-4114-acc5-5b4f47116853 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:01.782781 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 52/115] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:01.783676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ad495-e461-4d55-bbe4-aec83270536c None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:01.790123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:01.874511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2aaad3-312c-4114-acc5-5b4f47116853 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:01.904363 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ad495-e461-4d55-bbe4-aec83270536c None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:01.919276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8b0e4-e263-4e7e-a720-f00760888575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:01.920119 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8b0e4-e263-4e7e-a720-f00760888575 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.920449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8b0e4-e263-4e7e-a720-f00760888575 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.920739 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8b0e4-e263-4e7e-a720-f00760888575 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.922169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8b0e4-e263-4e7e-a720-f00760888575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:01.923703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8b0e4-e263-4e7e-a720-f00760888575 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:01.930421 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 64/116] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:01.931263 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8b0e4-e263-4e7e-a720-f00760888575 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:01.946095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070ca28-689c-4f9b-8226-ef7d09582e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:01.946996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070ca28-689c-4f9b-8226-ef7d09582e8e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:01.947272 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070ca28-689c-4f9b-8226-ef7d09582e8e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:01.947552 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070ca28-689c-4f9b-8226-ef7d09582e8e None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:01.949067 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070ca28-689c-4f9b-8226-ef7d09582e8e None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:01.950546 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070ca28-689c-4f9b-8226-ef7d09582e8e None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:01.953030 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 53/117] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:01 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:01.953675 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070ca28-689c-4f9b-8226-ef7d09582e8e None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:01.993794 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:02.035932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:02.054830 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8b0e4-e263-4e7e-a720-f00760888575 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:02.073956 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070ca28-689c-4f9b-8226-ef7d09582e8e None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:02.096046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c13f72-3147-4345-afb6-5f07e148454f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:02.097155 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c13f72-3147-4345-afb6-5f07e148454f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:02.097593 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c13f72-3147-4345-afb6-5f07e148454f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:02.098105 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c13f72-3147-4345-afb6-5f07e148454f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:02.109742 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c13f72-3147-4345-afb6-5f07e148454f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:02.111604 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a682a01-dae6-4b86-b7c0-02c98b473741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:02.112673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682a01-dae6-4b86-b7c0-02c98b473741 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:02.113110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682a01-dae6-4b86-b7c0-02c98b473741 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:02.113554 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a682a01-dae6-4b86-b7c0-02c98b473741 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-61c13f72-3147-4345-afb6-5f07e148454f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:02.117690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:02.118921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:02.120017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 65/118] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:02.125025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffd03d-e427-475e-a597-ddcea23fd26b None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:02.125329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffd03d-e427-475e-a597-ddcea23fd26b None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:02.125616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ffd03d-e427-475e-a597-ddcea23fd26b None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:02.126962 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 66/119] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:02.131572 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffd03d-e427-475e-a597-ddcea23fd26b None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:02.137423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a682a01-dae6-4b86-b7c0-02c98b473741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-8a682a01-dae6-4b86-b7c0-02c98b473741 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:02.145805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:02.147412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:02.148368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 54/120] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:02.153102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a682a01-dae6-4b86-b7c0-02c98b473741 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:02.243450 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ffd03d-e427-475e-a597-ddcea23fd26b None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:02.259824 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:02.266473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a682a01-dae6-4b86-b7c0-02c98b473741 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:02.281368 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5728e1-1f4f-4fde-8d26-36fac10f76b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:02.282088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5728e1-1f4f-4fde-8d26-36fac10f76b6 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:02.282376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5728e1-1f4f-4fde-8d26-36fac10f76b6 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:02.282658 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5728e1-1f4f-4fde-8d26-36fac10f76b6 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:02.283386 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5728e1-1f4f-4fde-8d26-36fac10f76b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:02.284897 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5728e1-1f4f-4fde-8d26-36fac10f76b6 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:02.299763 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:02.305992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0b99c-804c-478f-aeec-786b9bfa8696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:02.306894 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0b99c-804c-478f-aeec-786b9bfa8696 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:02.307179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0b99c-804c-478f-aeec-786b9bfa8696 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:02.307454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0b99c-804c-478f-aeec-786b9bfa8696 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d/users/ff96da569f6448c9b0a3b1aebf6f10e0/roles/c8be6a897df546ca9b0e7b9541fb94d5` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:02.309489 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0b99c-804c-478f-aeec-786b9bfa8696 None admin] RBAC: Authorizing `identity:create_grant(project_id=464c75ba6b6340e4b017049e6993fe4d, user_id=ff96da569f6448c9b0a3b1aebf6f10e0, role_id=c8be6a897df546ca9b0e7b9541fb94d5)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:02.310942 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0b99c-804c-478f-aeec-786b9bfa8696 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:02.371890 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 67/121] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:28:02 2022] POST /identity/v3/projects => generated 302 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:02.371890 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5728e1-1f4f-4fde-8d26-36fac10f76b6 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:02.436368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 55/122] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:28:02 2022] PUT /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d/users/ff96da569f6448c9b0a3b1aebf6f10e0/roles/c8be6a897df546ca9b0e7b9541fb94d5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:02.438834 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ceb6d-b083-4e04-a924-6d7b20ece1bd None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:02.439242 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ceb6d-b083-4e04-a924-6d7b20ece1bd None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:02.439594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ceb6d-b083-4e04-a924-6d7b20ece1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:02.482956 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-029ceb6d-b083-4e04-a924-6d7b20ece1bd None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:02.486936 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-029ceb6d-b083-4e04-a924-6d7b20ece1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:02.522991 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5728e1-1f4f-4fde-8d26-36fac10f76b6 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:02.571518 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420435b2-7c4c-4701-9afb-704f0071b71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:02.574228 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420435b2-7c4c-4701-9afb-704f0071b71e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:02.574527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420435b2-7c4c-4701-9afb-704f0071b71e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:02.574870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420435b2-7c4c-4701-9afb-704f0071b71e None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:02.575673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420435b2-7c4c-4701-9afb-704f0071b71e None admin] RBAC: Adding query filter params (name=reader) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:02.577449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420435b2-7c4c-4701-9afb-704f0071b71e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:02.581283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420435b2-7c4c-4701-9afb-704f0071b71e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:02.602505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 68/123] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:02.605597 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420435b2-7c4c-4701-9afb-704f0071b71e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:02.636483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 56/124] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:02.637759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ceb6d-b083-4e04-a924-6d7b20ece1bd None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:02.643508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:02.731860 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420435b2-7c4c-4701-9afb-704f0071b71e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:02.758498 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ceb6d-b083-4e04-a924-6d7b20ece1bd None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:02.774771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106311e-9ca3-445a-9176-bdf3020b97e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:02.775776 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106311e-9ca3-445a-9176-bdf3020b97e4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:02.776258 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106311e-9ca3-445a-9176-bdf3020b97e4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:02.776769 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106311e-9ca3-445a-9176-bdf3020b97e4 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:02.777737 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106311e-9ca3-445a-9176-bdf3020b97e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:02.778051 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106311e-9ca3-445a-9176-bdf3020b97e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:02.779439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1106311e-9ca3-445a-9176-bdf3020b97e4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:02.788697 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:02.790661 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 69/125] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:02.792943 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106311e-9ca3-445a-9176-bdf3020b97e4 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:02.810984 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01cb31-e651-4a31-8217-b4d7be7ecbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:02.812585 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01cb31-e651-4a31-8217-b4d7be7ecbaa None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:02.813110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01cb31-e651-4a31-8217-b4d7be7ecbaa None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:02.813596 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01cb31-e651-4a31-8217-b4d7be7ecbaa None admin] PATH_INFO: `/v3/users/e3cc98fcaad44f8aac8763b6cd5825a7` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:02.838050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01cb31-e651-4a31-8217-b4d7be7ecbaa None admin] RBAC: Authorizing `identity:get_user(user_id=e3cc98fcaad44f8aac8763b6cd5825a7)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:02.841252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a01cb31-e651-4a31-8217-b4d7be7ecbaa None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:02.842654 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 57/126] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:02 2022] GET /identity/v3/users/e3cc98fcaad44f8aac8763b6cd5825a7 => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:02.844625 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01cb31-e651-4a31-8217-b4d7be7ecbaa None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:02.912727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:02.926794 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106311e-9ca3-445a-9176-bdf3020b97e4 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:02.978022 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51326079-c64d-46c2-84e9-a7eedcfb1d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:02.978979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51326079-c64d-46c2-84e9-a7eedcfb1d94 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:02.979425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51326079-c64d-46c2-84e9-a7eedcfb1d94 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:02.979776 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51326079-c64d-46c2-84e9-a7eedcfb1d94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:02.986824 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01cb31-e651-4a31-8217-b4d7be7ecbaa None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:03.006875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51326079-c64d-46c2-84e9-a7eedcfb1d94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-51326079-c64d-46c2-84e9-a7eedcfb1d94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:03.015365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:03.016600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:03.017718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 70/127] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:03.018465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51326079-c64d-46c2-84e9-a7eedcfb1d94 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:03.031733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92e75b-85f7-471a-9f9b-a038de6828c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:03.033567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92e75b-85f7-471a-9f9b-a038de6828c8 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:03.034068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92e75b-85f7-471a-9f9b-a038de6828c8 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:03.034581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92e75b-85f7-471a-9f9b-a038de6828c8 None admin] PATH_INFO: `/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:03.036693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92e75b-85f7-471a-9f9b-a038de6828c8 None admin] RBAC: Authorizing `identity:get_user(user_id=ff96da569f6448c9b0a3b1aebf6f10e0)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:03.039457 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92e75b-85f7-471a-9f9b-a038de6828c8 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:03.042019 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 58/128] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:02 2022] GET /identity/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:03.043322 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92e75b-85f7-471a-9f9b-a038de6828c8 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:03.111665 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:03.136366 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51326079-c64d-46c2-84e9-a7eedcfb1d94 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:03.185921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a7941-0bea-444c-88f3-6f883b779f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:03.186593 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a7941-0bea-444c-88f3-6f883b779f57 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:03.187762 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a7941-0bea-444c-88f3-6f883b779f57 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:03.188047 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a7941-0bea-444c-88f3-6f883b779f57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:03.193315 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92e75b-85f7-471a-9f9b-a038de6828c8 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:03.221906 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:03.260722 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e933551-7292-4c3d-9e07-5a646e92a199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:03.261462 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e933551-7292-4c3d-9e07-5a646e92a199 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:03.261748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e933551-7292-4c3d-9e07-5a646e92a199 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:03.262030 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e933551-7292-4c3d-9e07-5a646e92a199 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:03.262899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e933551-7292-4c3d-9e07-5a646e92a199 None admin] RBAC: Adding query filter params (name=member) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:03.263238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e933551-7292-4c3d-9e07-5a646e92a199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:03.265827 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e933551-7292-4c3d-9e07-5a646e92a199 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:03.284477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 59/129] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:03.285312 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e933551-7292-4c3d-9e07-5a646e92a199 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:03.314495 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a7941-0bea-444c-88f3-6f883b779f57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:03.316379 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1a7941-0bea-444c-88f3-6f883b779f57 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:03.443013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 71/130] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:03.449418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a7941-0bea-444c-88f3-6f883b779f57 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:03.482260 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e933551-7292-4c3d-9e07-5a646e92a199 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:03.509420 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:03.512399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:03.562072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a7941-0bea-444c-88f3-6f883b779f57 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:03.575342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e49baa-8fdf-419a-96be-40567ab70ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:03.576600 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e49baa-8fdf-419a-96be-40567ab70ad1 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:03.577117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e49baa-8fdf-419a-96be-40567ab70ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:03.577621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e49baa-8fdf-419a-96be-40567ab70ad1 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:03.581284 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e49baa-8fdf-419a-96be-40567ab70ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:03.583002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e49baa-8fdf-419a-96be-40567ab70ad1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:03.585612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 60/131] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:03 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:03.593338 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e49baa-8fdf-419a-96be-40567ab70ad1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:03.637017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3e53a-ed6d-4d1f-ba27-90d07f48e29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:03.638322 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3e53a-ed6d-4d1f-ba27-90d07f48e29e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:03.638832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3e53a-ed6d-4d1f-ba27-90d07f48e29e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:03.639333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3e53a-ed6d-4d1f-ba27-90d07f48e29e None admin] PATH_INFO: `/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:03.641708 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3e53a-ed6d-4d1f-ba27-90d07f48e29e None admin] RBAC: Authorizing `identity:get_user(user_id=ff96da569f6448c9b0a3b1aebf6f10e0)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:03.643415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a3e53a-ed6d-4d1f-ba27-90d07f48e29e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:03.646125 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 72/132] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:03 2022] GET /identity/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:03.647879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3e53a-ed6d-4d1f-ba27-90d07f48e29e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:03.804931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e49baa-8fdf-419a-96be-40567ab70ad1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:03.853173 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf139d1d-e9d3-4b7a-9008-a3abde6a6e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:03.854531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf139d1d-e9d3-4b7a-9008-a3abde6a6e1c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:03.856984 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf139d1d-e9d3-4b7a-9008-a3abde6a6e1c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:03.857570 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf139d1d-e9d3-4b7a-9008-a3abde6a6e1c None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:03.860736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf139d1d-e9d3-4b7a-9008-a3abde6a6e1c None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:03.862503 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf139d1d-e9d3-4b7a-9008-a3abde6a6e1c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:03.866466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3e53a-ed6d-4d1f-ba27-90d07f48e29e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:03.870703 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 61/133] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:03 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:03.876571 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf139d1d-e9d3-4b7a-9008-a3abde6a6e1c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:03.911748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:03.960615 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23a330-f38b-4af2-874f-7869fe051cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:03.961567 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23a330-f38b-4af2-874f-7869fe051cbb None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:03.961848 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23a330-f38b-4af2-874f-7869fe051cbb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:03.962132 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23a330-f38b-4af2-874f-7869fe051cbb None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d/users/ff96da569f6448c9b0a3b1aebf6f10e0/roles/ff499a8782db4cce8e162a849be212ee` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:03.964684 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23a330-f38b-4af2-874f-7869fe051cbb None admin] RBAC: Authorizing `identity:create_grant(project_id=464c75ba6b6340e4b017049e6993fe4d, user_id=ff96da569f6448c9b0a3b1aebf6f10e0, role_id=ff499a8782db4cce8e162a849be212ee)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:03.966365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23a330-f38b-4af2-874f-7869fe051cbb None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:03.976731 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:04.019000 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf139d1d-e9d3-4b7a-9008-a3abde6a6e1c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:04.057474 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789d992-f7c4-478b-849e-6981c3d33f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:04.064913 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 73/134] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:28:03 2022] PUT /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d/users/ff96da569f6448c9b0a3b1aebf6f10e0/roles/ff499a8782db4cce8e162a849be212ee => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:04.066064 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23a330-f38b-4af2-874f-7869fe051cbb None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:04.067023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789d992-f7c4-478b-849e-6981c3d33f2b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:04.070634 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789d992-f7c4-478b-849e-6981c3d33f2b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:04.071832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789d992-f7c4-478b-849e-6981c3d33f2b None admin] PATH_INFO: `/v3/projects/6f73be14e6644cbf87eb6b64d960dcaa` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:04.073867 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789d992-f7c4-478b-849e-6981c3d33f2b None admin] RBAC: Authorizing `identity:get_project(project_id=6f73be14e6644cbf87eb6b64d960dcaa)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:04.080871 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f789d992-f7c4-478b-849e-6981c3d33f2b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:04.083312 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 62/135] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:03 2022] GET /identity/v3/projects/6f73be14e6644cbf87eb6b64d960dcaa => generated 312 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:04.090209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789d992-f7c4-478b-849e-6981c3d33f2b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:04.218236 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789d992-f7c4-478b-849e-6981c3d33f2b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:04.241361 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:04.258195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059c680-2163-4d19-b0b2-3229f3ad41c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:04.264040 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059c680-2163-4d19-b0b2-3229f3ad41c5 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:04.264040 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059c680-2163-4d19-b0b2-3229f3ad41c5 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:04.264040 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059c680-2163-4d19-b0b2-3229f3ad41c5 None admin] PATH_INFO: `/v3/projects/6f73be14e6644cbf87eb6b64d960dcaa/users/ff96da569f6448c9b0a3b1aebf6f10e0/roles/c8be6a897df546ca9b0e7b9541fb94d5` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:04.264040 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059c680-2163-4d19-b0b2-3229f3ad41c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f73be14e6644cbf87eb6b64d960dcaa, user_id=ff96da569f6448c9b0a3b1aebf6f10e0, role_id=c8be6a897df546ca9b0e7b9541fb94d5)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:04.264040 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059c680-2163-4d19-b0b2-3229f3ad41c5 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:04.285760 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23a330-f38b-4af2-874f-7869fe051cbb None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:04.325432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d0f19-7f5a-4515-ada0-bc3731d123b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:04.326126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d0f19-7f5a-4515-ada0-bc3731d123b4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:04.326414 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d0f19-7f5a-4515-ada0-bc3731d123b4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:04.326696 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d0f19-7f5a-4515-ada0-bc3731d123b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:04.327549 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d0f19-7f5a-4515-ada0-bc3731d123b4 None admin] RBAC: Adding query filter params (role.id=7b870ddcb3b04c6088d9d80f29ab8193, scope.project.id=464c75ba6b6340e4b017049e6993fe4d, user.id=e3cc98fcaad44f8aac8763b6cd5825a7) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:04.327869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d0f19-7f5a-4515-ada0-bc3731d123b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:04.329391 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d0f19-7f5a-4515-ada0-bc3731d123b4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:04.336785 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 74/136] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:04 2022] GET /identity/v3/role_assignments?user.id=e3cc98fcaad44f8aac8763b6cd5825a7&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=7b870ddcb3b04c6088d9d80f29ab8193 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:04.338324 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f15fb-8ff6-42e9-ad8c-535e793590e8 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:04.338612 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f15fb-8ff6-42e9-ad8c-535e793590e8 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:04.338896 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f15fb-8ff6-42e9-ad8c-535e793590e8 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:04.340708 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 75/137] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:04.346996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc20448-1bb9-4aa8-956d-dd9b0e8fae3a None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:04.347323 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc20448-1bb9-4aa8-956d-dd9b0e8fae3a None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:04.347575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc20448-1bb9-4aa8-956d-dd9b0e8fae3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:04.368114 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 63/138] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:28:04 2022] PUT /identity/v3/projects/6f73be14e6644cbf87eb6b64d960dcaa/users/ff96da569f6448c9b0a3b1aebf6f10e0/roles/c8be6a897df546ca9b0e7b9541fb94d5 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:28:04.374063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-4bc20448-1bb9-4aa8-956d-dd9b0e8fae3a None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:04.375326 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-4bc20448-1bb9-4aa8-956d-dd9b0e8fae3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:04.416930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:04.460823 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 76/139] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:04.487229 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059c680-2163-4d19-b0b2-3229f3ad41c5 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:04.640713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059c680-2163-4d19-b0b2-3229f3ad41c5 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:04.689986 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996546e-6f21-4b71-aef2-0b85210aa078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:04.691218 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996546e-6f21-4b71-aef2-0b85210aa078 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:04.691742 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996546e-6f21-4b71-aef2-0b85210aa078 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:04.692234 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996546e-6f21-4b71-aef2-0b85210aa078 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:04.710845 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996546e-6f21-4b71-aef2-0b85210aa078 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-4996546e-6f21-4b71-aef2-0b85210aa078 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:04.719450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:04.720550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:04.721453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:04.724059 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 64/140] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:28:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:04.731084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc20448-1bb9-4aa8-956d-dd9b0e8fae3a None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:04.850108 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc20448-1bb9-4aa8-956d-dd9b0e8fae3a None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:04.886720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8911e-62c0-41d1-a30c-718984a8595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:04.887411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8911e-62c0-41d1-a30c-718984a8595a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:04.887697 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8911e-62c0-41d1-a30c-718984a8595a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:04.887981 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8911e-62c0-41d1-a30c-718984a8595a None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:04.888901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8911e-62c0-41d1-a30c-718984a8595a None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:04.889215 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8911e-62c0-41d1-a30c-718984a8595a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:04.890631 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff8911e-62c0-41d1-a30c-718984a8595a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:04.904452 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 77/141] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:04.906404 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996546e-6f21-4b71-aef2-0b85210aa078 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:04.948978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:05.019143 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996546e-6f21-4b71-aef2-0b85210aa078 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:05.065139 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e701b-47fa-4d37-9d8a-1d9fc64ff8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:05.066453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e701b-47fa-4d37-9d8a-1d9fc64ff8eb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:05.066961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e701b-47fa-4d37-9d8a-1d9fc64ff8eb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:05.067470 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e701b-47fa-4d37-9d8a-1d9fc64ff8eb None admin] PATH_INFO: `/v3/users/cc336177e5d04523af605ab7e66532eb` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:05.069700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e701b-47fa-4d37-9d8a-1d9fc64ff8eb None admin] RBAC: Authorizing `identity:get_user(user_id=cc336177e5d04523af605ab7e66532eb)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:05.071410 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e701b-47fa-4d37-9d8a-1d9fc64ff8eb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:05.073364 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 65/142] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:04 2022] GET /identity/v3/users/cc336177e5d04523af605ab7e66532eb => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:05.080758 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8911e-62c0-41d1-a30c-718984a8595a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:05.146152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:05.245317 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8911e-62c0-41d1-a30c-718984a8595a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:05.293999 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440053c-09c8-4c96-a973-dad8c63543ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:05.294686 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440053c-09c8-4c96-a973-dad8c63543ba None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:05.294971 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440053c-09c8-4c96-a973-dad8c63543ba None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:05.295250 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440053c-09c8-4c96-a973-dad8c63543ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:05.387897 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440053c-09c8-4c96-a973-dad8c63543ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:05.390035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440053c-09c8-4c96-a973-dad8c63543ba None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:05.436128 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:05.440802 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a06644-89b6-4735-8d4a-486a517e5fb4 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:05.441382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a06644-89b6-4735-8d4a-486a517e5fb4 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:05.441986 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a06644-89b6-4735-8d4a-486a517e5fb4 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:05.451375 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 66/143] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:05.458040 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ce1fd-fee2-486c-b632-8b97495f4650 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:05.458647 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ce1fd-fee2-486c-b632-8b97495f4650 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:05.459165 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ce1fd-fee2-486c-b632-8b97495f4650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:05.483268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-d92ce1fd-fee2-486c-b632-8b97495f4650 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:05.485709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-d92ce1fd-fee2-486c-b632-8b97495f4650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:05.604177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 67/144] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:05.605836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 78/145] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:05.611370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ce1fd-fee2-486c-b632-8b97495f4650 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:05.641632 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440053c-09c8-4c96-a973-dad8c63543ba None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:05.663642 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:05.748912 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ce1fd-fee2-486c-b632-8b97495f4650 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:05.791611 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdc3b0-7b8f-4da7-80ec-90e7ca982907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:05.793778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdc3b0-7b8f-4da7-80ec-90e7ca982907 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:05.794447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdc3b0-7b8f-4da7-80ec-90e7ca982907 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:05.795131 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdc3b0-7b8f-4da7-80ec-90e7ca982907 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:05.797770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdc3b0-7b8f-4da7-80ec-90e7ca982907 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:05.800240 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdc3b0-7b8f-4da7-80ec-90e7ca982907 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:05.807607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 68/146] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:05 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:05.814540 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdc3b0-7b8f-4da7-80ec-90e7ca982907 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:05.878500 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440053c-09c8-4c96-a973-dad8c63543ba None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:05.939163 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:05.949010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686bc38-e9e5-411d-a451-1d59db02c8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:05.949833 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686bc38-e9e5-411d-a451-1d59db02c8c2 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:05.950141 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686bc38-e9e5-411d-a451-1d59db02c8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:05.950488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686bc38-e9e5-411d-a451-1d59db02c8c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:05.957939 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:05.962027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdc3b0-7b8f-4da7-80ec-90e7ca982907 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:05.964365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686bc38-e9e5-411d-a451-1d59db02c8c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c686bc38-e9e5-411d-a451-1d59db02c8c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:05.980667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:05.981765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:05.982662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 79/147] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:05.983902 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5a5a-73af-4805-993f-b93124faeebb None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:05.984026 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5a5a-73af-4805-993f-b93124faeebb None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:05.984251 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d5a5a-73af-4805-993f-b93124faeebb None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:05.985715 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 80/148] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:05.991554 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6a75c-089c-4624-a8a4-fcbd060bf3c4 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:05.991841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6a75c-089c-4624-a8a4-fcbd060bf3c4 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:05.992172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6a75c-089c-4624-a8a4-fcbd060bf3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:06.019302 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1e7cf-2304-4e1e-982e-651843898078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:06.019963 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1e7cf-2304-4e1e-982e-651843898078 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:06.020183 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1e7cf-2304-4e1e-982e-651843898078 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:06.020497 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1e7cf-2304-4e1e-982e-651843898078 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:06.021878 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1e7cf-2304-4e1e-982e-651843898078 None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=464c75ba6b6340e4b017049e6993fe4d, user.id=cc336177e5d04523af605ab7e66532eb) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:06.022203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1e7cf-2304-4e1e-982e-651843898078 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:06.024415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-7ae6a75c-089c-4624-a8a4-fcbd060bf3c4 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:06.025896 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-7ae6a75c-089c-4624-a8a4-fcbd060bf3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:06.026911 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1e7cf-2304-4e1e-982e-651843898078 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:06.040678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 69/149] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:05 2022] GET /identity/v3/role_assignments?user.id=cc336177e5d04523af605ab7e66532eb&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:06.045206 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1e7cf-2304-4e1e-982e-651843898078 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:06.173030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 81/150] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:06.210715 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6a75c-089c-4624-a8a4-fcbd060bf3c4 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:06.294926 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1e7cf-2304-4e1e-982e-651843898078 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:06.390273 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3f411-fe8f-45c8-b235-4a63d18acdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:06.391299 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3f411-fe8f-45c8-b235-4a63d18acdde None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:06.391592 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3f411-fe8f-45c8-b235-4a63d18acdde None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:06.391876 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3f411-fe8f-45c8-b235-4a63d18acdde None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:06.392873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3f411-fe8f-45c8-b235-4a63d18acdde None admin] RBAC: Adding query filter params (name=member) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:06.393186 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3f411-fe8f-45c8-b235-4a63d18acdde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:06.394560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3f411-fe8f-45c8-b235-4a63d18acdde None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:06.413914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 70/151] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:06.420192 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3f411-fe8f-45c8-b235-4a63d18acdde None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:06.444077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6a75c-089c-4624-a8a4-fcbd060bf3c4 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:06.496743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:06.522319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:06.540112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f04abd-202d-4c12-8689-c2f2a04e8f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:06.540975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f04abd-202d-4c12-8689-c2f2a04e8f46 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:06.541261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f04abd-202d-4c12-8689-c2f2a04e8f46 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:06.541541 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f04abd-202d-4c12-8689-c2f2a04e8f46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:06.544091 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f04abd-202d-4c12-8689-c2f2a04e8f46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:06.550772 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f04abd-202d-4c12-8689-c2f2a04e8f46 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:06.550772 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 82/152] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:06.554360 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f04abd-202d-4c12-8689-c2f2a04e8f46 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:06.639699 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3f411-fe8f-45c8-b235-4a63d18acdde None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:06.706428 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4050e80-9cb6-48d3-ae1b-92558b739cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:06.707328 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4050e80-9cb6-48d3-ae1b-92558b739cba None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:06.707613 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4050e80-9cb6-48d3-ae1b-92558b739cba None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:06.707887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4050e80-9cb6-48d3-ae1b-92558b739cba None admin] PATH_INFO: `/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:06.710527 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4050e80-9cb6-48d3-ae1b-92558b739cba None admin] RBAC: Authorizing `identity:get_user(user_id=ff96da569f6448c9b0a3b1aebf6f10e0)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:06.711974 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4050e80-9cb6-48d3-ae1b-92558b739cba None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:06.713663 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 71/153] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:06 2022] GET /identity/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0 => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:06.721912 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4050e80-9cb6-48d3-ae1b-92558b739cba None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:06.749722 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f04abd-202d-4c12-8689-c2f2a04e8f46 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:06.817159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976587e-104e-4e1b-a97c-41d12b735f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:06.817765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976587e-104e-4e1b-a97c-41d12b735f8d None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:06.818073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976587e-104e-4e1b-a97c-41d12b735f8d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:06.818884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976587e-104e-4e1b-a97c-41d12b735f8d None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:06.819611 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976587e-104e-4e1b-a97c-41d12b735f8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:06.821147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976587e-104e-4e1b-a97c-41d12b735f8d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:06.876706 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:06.883625 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4050e80-9cb6-48d3-ae1b-92558b739cba None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:06.920634 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b34e14-f792-480a-bd71-0768881aad44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:06.921311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b34e14-f792-480a-bd71-0768881aad44 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:06.927641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b34e14-f792-480a-bd71-0768881aad44 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:06.927641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b34e14-f792-480a-bd71-0768881aad44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:06.959637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 83/154] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:28:06 2022] POST /identity/v3/users => generated 283 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:07.011482 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b34e14-f792-480a-bd71-0768881aad44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:07.013089 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b34e14-f792-480a-bd71-0768881aad44 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:07.135059 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:07.143119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 72/155] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:07.149603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976587e-104e-4e1b-a97c-41d12b735f8d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:07.223885 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:07.288239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976587e-104e-4e1b-a97c-41d12b735f8d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:07.325351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce2920d-3732-43ff-be2d-e1485f1f043d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:07.326235 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2920d-3732-43ff-be2d-e1485f1f043d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:07.326519 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2920d-3732-43ff-be2d-e1485f1f043d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:07.326796 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2920d-3732-43ff-be2d-e1485f1f043d None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:07.328056 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2920d-3732-43ff-be2d-e1485f1f043d None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:07.329598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce2920d-3732-43ff-be2d-e1485f1f043d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:07.331900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 84/156] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:07 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:07.338400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b34e14-f792-480a-bd71-0768881aad44 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:07.380132 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c6c9e-c350-4a56-876b-6b6a8ba01f58 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:07.380489 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c6c9e-c350-4a56-876b-6b6a8ba01f58 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:07.381274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c6c9e-c350-4a56-876b-6b6a8ba01f58 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:07.382651 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 85/157] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:28:07.388207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238f997-d76d-467d-8cdb-005d6ce159ac None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:07.388468 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238f997-d76d-467d-8cdb-005d6ce159ac None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:07.388751 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238f997-d76d-467d-8cdb-005d6ce159ac None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:07.390093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 86/158] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:28:07.391180 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e91d-b23b-4981-8958-4f8af75558a4 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:07.391466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e91d-b23b-4981-8958-4f8af75558a4 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:07.391835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e91d-b23b-4981-8958-4f8af75558a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:07.414313 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-bc51e91d-b23b-4981-8958-4f8af75558a4 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:07.415347 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-bc51e91d-b23b-4981-8958-4f8af75558a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:07.466647 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b34e14-f792-480a-bd71-0768881aad44 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:07.514573 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02783d3-fe0a-4fff-8a2e-c9b621a6abe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:07.515245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02783d3-fe0a-4fff-8a2e-c9b621a6abe9 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:07.515675 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02783d3-fe0a-4fff-8a2e-c9b621a6abe9 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:07.515883 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02783d3-fe0a-4fff-8a2e-c9b621a6abe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:07.516706 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02783d3-fe0a-4fff-8a2e-c9b621a6abe9 None admin] RBAC: Adding query filter params (role.id=c8be6a897df546ca9b0e7b9541fb94d5, scope.project.id=464c75ba6b6340e4b017049e6993fe4d, user.id=ff96da569f6448c9b0a3b1aebf6f10e0) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:07.517021 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02783d3-fe0a-4fff-8a2e-c9b621a6abe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:07.518512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02783d3-fe0a-4fff-8a2e-c9b621a6abe9 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:07.526075 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 87/159] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:07.527511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072a970-1e25-47fc-a778-d2abbded0357 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:07.528808 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072a970-1e25-47fc-a778-d2abbded0357 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:07.528808 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072a970-1e25-47fc-a778-d2abbded0357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:07.532896 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 73/160] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:07 2022] GET /identity/v3/role_assignments?user.id=ff96da569f6448c9b0a3b1aebf6f10e0&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:07.557441 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02783d3-fe0a-4fff-8a2e-c9b621a6abe9 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:07.558758 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-a072a970-1e25-47fc-a778-d2abbded0357 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:07.560227 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-a072a970-1e25-47fc-a778-d2abbded0357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:07.612988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:07.636755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:07.646572 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 88/161] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:07.679699 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072a970-1e25-47fc-a778-d2abbded0357 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:07.705814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02783d3-fe0a-4fff-8a2e-c9b621a6abe9 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:07.749609 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb957b-9a81-4571-b994-50570d734577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:07.750876 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb957b-9a81-4571-b994-50570d734577 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:07.751351 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb957b-9a81-4571-b994-50570d734577 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:07.751812 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb957b-9a81-4571-b994-50570d734577 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:07.765680 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbb957b-9a81-4571-b994-50570d734577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-1bbb957b-9a81-4571-b994-50570d734577 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:07.774636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:07.775707 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:07.776641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:07.779709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 74/162] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:07.794070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb957b-9a81-4571-b994-50570d734577 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:07.809890 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072a970-1e25-47fc-a778-d2abbded0357 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:07.866168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736da51-06de-481e-9a2c-e1c07f91a339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:07.868829 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736da51-06de-481e-9a2c-e1c07f91a339 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:07.868829 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736da51-06de-481e-9a2c-e1c07f91a339 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:07.868829 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736da51-06de-481e-9a2c-e1c07f91a339 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:07.893381 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5736da51-06de-481e-9a2c-e1c07f91a339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-5736da51-06de-481e-9a2c-e1c07f91a339 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:07.903527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:07.904790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:07.905894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:07.908089 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 89/163] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:07.909880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00cbfe-1860-4902-bb36-a47124bb3996 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:07.910348 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00cbfe-1860-4902-bb36-a47124bb3996 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:07.910777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00cbfe-1860-4902-bb36-a47124bb3996 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:07.912322 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 90/164] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:07.918218 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0478d-e781-4a23-ab33-7fae67e0c6ba None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:07.918682 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0478d-e781-4a23-ab33-7fae67e0c6ba None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:07.919117 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0478d-e781-4a23-ab33-7fae67e0c6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:07.926043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb957b-9a81-4571-b994-50570d734577 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:07.939659 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-76d0478d-e781-4a23-ab33-7fae67e0c6ba None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:07.940897 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-76d0478d-e781-4a23-ab33-7fae67e0c6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:07.954035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:07.972508 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c628ab6-e61d-4018-b06e-d483f7c2da13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:07.973592 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c628ab6-e61d-4018-b06e-d483f7c2da13 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:07.974153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c628ab6-e61d-4018-b06e-d483f7c2da13 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:07.974526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c628ab6-e61d-4018-b06e-d483f7c2da13 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:07.975422 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c628ab6-e61d-4018-b06e-d483f7c2da13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:07.975788 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c628ab6-e61d-4018-b06e-d483f7c2da13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:07.977420 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c628ab6-e61d-4018-b06e-d483f7c2da13 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:07.978078 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:07.991347 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 75/165] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:07.992313 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c628ab6-e61d-4018-b06e-d483f7c2da13 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:08.030374 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 91/166] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:08.031574 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0478d-e781-4a23-ab33-7fae67e0c6ba None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:08.134461 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c628ab6-e61d-4018-b06e-d483f7c2da13 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:08.156111 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0478d-e781-4a23-ab33-7fae67e0c6ba None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:08.179565 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e922bf-5a63-461d-aee4-3dbabd4cf96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:08.180423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e922bf-5a63-461d-aee4-3dbabd4cf96c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:08.180571 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e922bf-5a63-461d-aee4-3dbabd4cf96c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:08.180885 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e922bf-5a63-461d-aee4-3dbabd4cf96c None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:08.181595 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e922bf-5a63-461d-aee4-3dbabd4cf96c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:08.181911 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e922bf-5a63-461d-aee4-3dbabd4cf96c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:08.183636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e922bf-5a63-461d-aee4-3dbabd4cf96c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:08.196067 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 76/167] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:08.201224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e922bf-5a63-461d-aee4-3dbabd4cf96c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:08.210893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47dba7-ba94-4a64-af92-1ef384c8270d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:08.211792 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47dba7-ba94-4a64-af92-1ef384c8270d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:08.212079 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47dba7-ba94-4a64-af92-1ef384c8270d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:08.212467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47dba7-ba94-4a64-af92-1ef384c8270d None admin] PATH_INFO: `/v3/users/e3cc98fcaad44f8aac8763b6cd5825a7` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:08.213990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47dba7-ba94-4a64-af92-1ef384c8270d None admin] RBAC: Authorizing `identity:get_user(user_id=e3cc98fcaad44f8aac8763b6cd5825a7)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:08.215647 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47dba7-ba94-4a64-af92-1ef384c8270d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:08.217738 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 92/168] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:08 2022] GET /identity/v3/users/e3cc98fcaad44f8aac8763b6cd5825a7 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:08.218524 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47dba7-ba94-4a64-af92-1ef384c8270d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:08.254241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:08.263354 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:08.341691 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47dba7-ba94-4a64-af92-1ef384c8270d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:08.379483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e922bf-5a63-461d-aee4-3dbabd4cf96c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:08.398118 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084969b-0d34-4d0d-9767-c1cfe661988e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:08.399413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084969b-0d34-4d0d-9767-c1cfe661988e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:08.399872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084969b-0d34-4d0d-9767-c1cfe661988e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:08.400365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084969b-0d34-4d0d-9767-c1cfe661988e None admin] PATH_INFO: `/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:08.403382 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084969b-0d34-4d0d-9767-c1cfe661988e None admin] RBAC: Authorizing `identity:get_user(user_id=ff96da569f6448c9b0a3b1aebf6f10e0)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:08.405156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084969b-0d34-4d0d-9767-c1cfe661988e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:08.407174 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 93/169] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:08 2022] GET /identity/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:08.408153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084969b-0d34-4d0d-9767-c1cfe661988e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:08.422165 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f21b83-6d36-4be1-ba35-bd74da403f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:08.423188 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f21b83-6d36-4be1-ba35-bd74da403f43 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:08.423543 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f21b83-6d36-4be1-ba35-bd74da403f43 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:08.423895 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f21b83-6d36-4be1-ba35-bd74da403f43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:08.437187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f21b83-6d36-4be1-ba35-bd74da403f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-e3f21b83-6d36-4be1-ba35-bd74da403f43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:08.447136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:08.448258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:08.449238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:08.453911 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 77/170] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:08.454943 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f21b83-6d36-4be1-ba35-bd74da403f43 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:08.608929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f21b83-6d36-4be1-ba35-bd74da403f43 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:08.611371 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:08.649748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992c149-0a7c-401c-a197-0a067d9f77d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:08.651096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992c149-0a7c-401c-a197-0a067d9f77d3 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:08.651938 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992c149-0a7c-401c-a197-0a067d9f77d3 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:08.653425 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992c149-0a7c-401c-a197-0a067d9f77d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:08.669836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:08.678605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084969b-0d34-4d0d-9767-c1cfe661988e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:08.737290 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992c149-0a7c-401c-a197-0a067d9f77d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:08.739436 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992c149-0a7c-401c-a197-0a067d9f77d3 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:08.785181 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf90fd-751a-4e9d-b314-e3e883ba2a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:08.786299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf90fd-751a-4e9d-b314-e3e883ba2a25 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:08.786418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf90fd-751a-4e9d-b314-e3e883ba2a25 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:08.786630 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf90fd-751a-4e9d-b314-e3e883ba2a25 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:08.793502 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf90fd-751a-4e9d-b314-e3e883ba2a25 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:08.795007 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf90fd-751a-4e9d-b314-e3e883ba2a25 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:08.797231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 94/171] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:08 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:08.798040 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf90fd-751a-4e9d-b314-e3e883ba2a25 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:08.864161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 78/172] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:08.867078 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992c149-0a7c-401c-a197-0a067d9f77d3 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:08.870852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:09.008431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992c149-0a7c-401c-a197-0a067d9f77d3 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:09.046715 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f8da9-aca0-43a8-ac78-b1fd1ba279e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:09.047997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8da9-aca0-43a8-ac78-b1fd1ba279e1 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:09.048547 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8da9-aca0-43a8-ac78-b1fd1ba279e1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:09.049219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f8da9-aca0-43a8-ac78-b1fd1ba279e1 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d/users/e3cc98fcaad44f8aac8763b6cd5825a7/roles/7b870ddcb3b04c6088d9d80f29ab8193` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:09.051616 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f8da9-aca0-43a8-ac78-b1fd1ba279e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=464c75ba6b6340e4b017049e6993fe4d, user_id=e3cc98fcaad44f8aac8763b6cd5825a7, role_id=7b870ddcb3b04c6088d9d80f29ab8193)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:09.053323 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f8da9-aca0-43a8-ac78-b1fd1ba279e1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:09.064381 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf90fd-751a-4e9d-b314-e3e883ba2a25 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:09.107362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c2870-02be-4c7f-9939-0c28dcad22b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:09.108083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c2870-02be-4c7f-9939-0c28dcad22b3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:09.108421 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c2870-02be-4c7f-9939-0c28dcad22b3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:09.108787 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c2870-02be-4c7f-9939-0c28dcad22b3 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:09.109548 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c2870-02be-4c7f-9939-0c28dcad22b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:09.109863 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c2870-02be-4c7f-9939-0c28dcad22b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:09.111377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c2870-02be-4c7f-9939-0c28dcad22b3 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:09.121029 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 95/173] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:09.121733 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c2870-02be-4c7f-9939-0c28dcad22b3 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:09.135209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 79/174] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:28:08 2022] PUT /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d/users/e3cc98fcaad44f8aac8763b6cd5825a7/roles/7b870ddcb3b04c6088d9d80f29ab8193 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:09.136556 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f8da9-aca0-43a8-ac78-b1fd1ba279e1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:09.158425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:09.240656 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c2870-02be-4c7f-9939-0c28dcad22b3 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:09.263509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:09.294928 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f7327-b622-487a-a6e6-88bcf1309455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:09.295895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f7327-b622-487a-a6e6-88bcf1309455 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:09.296117 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f7327-b622-487a-a6e6-88bcf1309455 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:09.296418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f7327-b622-487a-a6e6-88bcf1309455 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:09.299803 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f7327-b622-487a-a6e6-88bcf1309455 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:09.301444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f7327-b622-487a-a6e6-88bcf1309455 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:09.303698 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 96/175] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:09 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:09.311019 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f7327-b622-487a-a6e6-88bcf1309455 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:09.321469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f8da9-aca0-43a8-ac78-b1fd1ba279e1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:09.370460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd63206-518e-4c2c-ba92-8af32c377875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:09.371359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd63206-518e-4c2c-ba92-8af32c377875 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:09.371640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd63206-518e-4c2c-ba92-8af32c377875 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:09.371914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd63206-518e-4c2c-ba92-8af32c377875 None admin] PATH_INFO: `/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:09.377049 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd63206-518e-4c2c-ba92-8af32c377875 None admin] RBAC: Authorizing `identity:get_user(user_id=ff96da569f6448c9b0a3b1aebf6f10e0)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:09.378606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd63206-518e-4c2c-ba92-8af32c377875 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:09.380995 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 80/176] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:09 2022] GET /identity/v3/users/ff96da569f6448c9b0a3b1aebf6f10e0 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:09.386679 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd63206-518e-4c2c-ba92-8af32c377875 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:09.425232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:09.430040 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f7327-b622-487a-a6e6-88bcf1309455 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:09.479719 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df74c1-68c2-40f7-b100-e3c7ab49a488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:09.480428 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df74c1-68c2-40f7-b100-e3c7ab49a488 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:09.480716 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df74c1-68c2-40f7-b100-e3c7ab49a488 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:09.481017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df74c1-68c2-40f7-b100-e3c7ab49a488 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:09.481834 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df74c1-68c2-40f7-b100-e3c7ab49a488 None admin] RBAC: Adding query filter params (role.id=ff499a8782db4cce8e162a849be212ee, scope.project.id=464c75ba6b6340e4b017049e6993fe4d, user.id=ff96da569f6448c9b0a3b1aebf6f10e0) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:09.482152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df74c1-68c2-40f7-b100-e3c7ab49a488 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:09.483631 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df74c1-68c2-40f7-b100-e3c7ab49a488 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:09.490894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 97/177] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:09 2022] GET /identity/v3/role_assignments?user.id=ff96da569f6448c9b0a3b1aebf6f10e0&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=ff499a8782db4cce8e162a849be212ee => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:09.554624 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd63206-518e-4c2c-ba92-8af32c377875 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:09.594113 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:09.604206 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e741574-eb7e-4151-b525-9013e0505deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:09.605036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e741574-eb7e-4151-b525-9013e0505deb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:09.605323 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e741574-eb7e-4151-b525-9013e0505deb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:09.605602 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e741574-eb7e-4151-b525-9013e0505deb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:09.707930 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e741574-eb7e-4151-b525-9013e0505deb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:09.710126 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e741574-eb7e-4151-b525-9013e0505deb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:09.820992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 81/178] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:28:09.826330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df74c1-68c2-40f7-b100-e3c7ab49a488 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:09.973205 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df74c1-68c2-40f7-b100-e3c7ab49a488 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:10.014741 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e156767-66da-4cf3-92e5-1131efd7f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:10.015628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e156767-66da-4cf3-92e5-1131efd7f0c0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:10.015909 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e156767-66da-4cf3-92e5-1131efd7f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:10.016185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e156767-66da-4cf3-92e5-1131efd7f0c0 None admin] PATH_INFO: `/v3/projects/6f73be14e6644cbf87eb6b64d960dcaa` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:10.017555 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e156767-66da-4cf3-92e5-1131efd7f0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=6f73be14e6644cbf87eb6b64d960dcaa)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:10.019157 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e156767-66da-4cf3-92e5-1131efd7f0c0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:10.021283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 98/179] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:09 2022] GET /identity/v3/projects/6f73be14e6644cbf87eb6b64d960dcaa => generated 312 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:10.028045 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e741574-eb7e-4151-b525-9013e0505deb None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:10.072384 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:10.160007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e741574-eb7e-4151-b525-9013e0505deb None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:10.208778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5ecbd-a345-4f49-ae18-61ea0b6611cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:10.209457 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5ecbd-a345-4f49-ae18-61ea0b6611cb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:10.209735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5ecbd-a345-4f49-ae18-61ea0b6611cb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:10.210005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5ecbd-a345-4f49-ae18-61ea0b6611cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:10.210857 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5ecbd-a345-4f49-ae18-61ea0b6611cb None admin] RBAC: Adding query filter params (role.id=c8be6a897df546ca9b0e7b9541fb94d5, scope.project.id=6f73be14e6644cbf87eb6b64d960dcaa, user.id=ff96da569f6448c9b0a3b1aebf6f10e0) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:10.211172 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5ecbd-a345-4f49-ae18-61ea0b6611cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:10.212659 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5ecbd-a345-4f49-ae18-61ea0b6611cb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:10.219449 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 82/180] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:10 2022] GET /identity/v3/role_assignments?user.id=ff96da569f6448c9b0a3b1aebf6f10e0&scope.project.id=6f73be14e6644cbf87eb6b64d960dcaa&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:10.500059 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b1e17-5717-4e0c-8421-87f9e2d38386 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:10.500612 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b1e17-5717-4e0c-8421-87f9e2d38386 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:10.500694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b1e17-5717-4e0c-8421-87f9e2d38386 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:10.502207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 99/181] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:10.507850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd177e-b079-4351-8d08-555a847419ed None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:10.508361 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd177e-b079-4351-8d08-555a847419ed None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:10.508452 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd177e-b079-4351-8d08-555a847419ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:10.528358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-aedd177e-b079-4351-8d08-555a847419ed None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:10.529871 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-aedd177e-b079-4351-8d08-555a847419ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:10.616513 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 83/182] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:10.639906 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b1e17-5717-4e0c-8421-87f9e2d38386 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:10.680515 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:10.778565 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b1e17-5717-4e0c-8421-87f9e2d38386 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:10.801578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fd162-2650-47eb-8af0-0028af8c8704 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:10.801918 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fd162-2650-47eb-8af0-0028af8c8704 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:10.802149 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fd162-2650-47eb-8af0-0028af8c8704 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:10.803595 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 84/183] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:10.809741 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad1466-675a-4f39-89cd-9f4a6ada690e None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:10.810034 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad1466-675a-4f39-89cd-9f4a6ada690e None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:10.810326 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad1466-675a-4f39-89cd-9f4a6ada690e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:10.821470 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344a87f-8d40-4b74-aa45-c261deba77ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:10.822339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344a87f-8d40-4b74-aa45-c261deba77ca None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:10.822624 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344a87f-8d40-4b74-aa45-c261deba77ca None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:10.822898 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344a87f-8d40-4b74-aa45-c261deba77ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:10.835295 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344a87f-8d40-4b74-aa45-c261deba77ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:10.836685 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-25ad1466-675a-4f39-89cd-9f4a6ada690e None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:10.838110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-25ad1466-675a-4f39-89cd-9f4a6ada690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-3344a87f-8d40-4b74-aa45-c261deba77ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:10.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:10.846254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:10.847172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 100/184] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:28:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:10.852789 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344a87f-8d40-4b74-aa45-c261deba77ca None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:10.869665 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:10.929820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 85/185] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:10.931320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da3406-4a09-4d9e-8fa6-9c110975882f None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:10.931618 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da3406-4a09-4d9e-8fa6-9c110975882f None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:10.931900 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da3406-4a09-4d9e-8fa6-9c110975882f None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:10.933214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 86/186] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:10.938694 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd5bde-0042-4077-b8f4-96d1a9005531 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:10.938984 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd5bde-0042-4077-b8f4-96d1a9005531 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:10.939267 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd5bde-0042-4077-b8f4-96d1a9005531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:10.956694 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-32dd5bde-0042-4077-b8f4-96d1a9005531 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:10.957993 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-32dd5bde-0042-4077-b8f4-96d1a9005531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:10.962612 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:10.972812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344a87f-8d40-4b74-aa45-c261deba77ca None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:11.013277 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96497c8-ec1c-4347-a691-77d4fa75a60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:11.014090 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96497c8-ec1c-4347-a691-77d4fa75a60d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:11.014380 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96497c8-ec1c-4347-a691-77d4fa75a60d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:11.014843 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96497c8-ec1c-4347-a691-77d4fa75a60d None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:11.015600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96497c8-ec1c-4347-a691-77d4fa75a60d None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:11.015914 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96497c8-ec1c-4347-a691-77d4fa75a60d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:11.017369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96497c8-ec1c-4347-a691-77d4fa75a60d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:11.027388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 101/187] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:11.028175 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96497c8-ec1c-4347-a691-77d4fa75a60d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:11.032403 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 87/188] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:11.033268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd5bde-0042-4077-b8f4-96d1a9005531 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:11.128664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:11.146104 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd5bde-0042-4077-b8f4-96d1a9005531 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:11.146475 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96497c8-ec1c-4347-a691-77d4fa75a60d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:11.182420 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2eb96-3a4f-4936-9d23-cf8e370faf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:11.183298 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2eb96-3a4f-4936-9d23-cf8e370faf9c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:11.183410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcc933-1e10-4e8f-a46e-35b49773d229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:11.183617 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2eb96-3a4f-4936-9d23-cf8e370faf9c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:11.183849 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2eb96-3a4f-4936-9d23-cf8e370faf9c None admin] PATH_INFO: `/v3/users/8cb66d6e449f4da48b01b8976f5e0e80` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:11.184150 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcc933-1e10-4e8f-a46e-35b49773d229 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:11.184474 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcc933-1e10-4e8f-a46e-35b49773d229 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:11.184752 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdcc933-1e10-4e8f-a46e-35b49773d229 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:11.185936 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdcc933-1e10-4e8f-a46e-35b49773d229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:11.187801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdcc933-1e10-4e8f-a46e-35b49773d229 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:11.193701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 102/189] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:11.193701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcc933-1e10-4e8f-a46e-35b49773d229 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:11.200123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2eb96-3a4f-4936-9d23-cf8e370faf9c None admin] RBAC: Authorizing `identity:get_user(user_id=8cb66d6e449f4da48b01b8976f5e0e80)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:11.201629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2eb96-3a4f-4936-9d23-cf8e370faf9c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:11.203486 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 88/190] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:11 2022] GET /identity/v3/users/8cb66d6e449f4da48b01b8976f5e0e80 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:11.204095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2eb96-3a4f-4936-9d23-cf8e370faf9c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:11.220231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:11.310948 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2eb96-3a4f-4936-9d23-cf8e370faf9c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:11.317221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdcc933-1e10-4e8f-a46e-35b49773d229 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:11.345368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287db77-b4c2-4a48-8740-1f98e1a7012d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:11.346033 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287db77-b4c2-4a48-8740-1f98e1a7012d None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:11.346323 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287db77-b4c2-4a48-8740-1f98e1a7012d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:11.346600 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7287db77-b4c2-4a48-8740-1f98e1a7012d None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:11.347315 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287db77-b4c2-4a48-8740-1f98e1a7012d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:11.348783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7287db77-b4c2-4a48-8740-1f98e1a7012d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:11.355058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8ea99-6c05-4032-941f-799616d8556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:11.356241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8ea99-6c05-4032-941f-799616d8556c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:11.356718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8ea99-6c05-4032-941f-799616d8556c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:11.357238 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8ea99-6c05-4032-941f-799616d8556c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:11.368453 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8ea99-6c05-4032-941f-799616d8556c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-d8b8ea99-6c05-4032-941f-799616d8556c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:11.376607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:11.380126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 103/191] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:11.381285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8ea99-6c05-4032-941f-799616d8556c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:11.455394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:11.468377 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 89/192] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:28:11 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:11.468377 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287db77-b4c2-4a48-8740-1f98e1a7012d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:11.476078 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:11.503589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8ea99-6c05-4032-941f-799616d8556c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:11.548357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5edb85-ee45-4695-9469-d00b590a2f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:11.549049 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5edb85-ee45-4695-9469-d00b590a2f0a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:11.549334 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5edb85-ee45-4695-9469-d00b590a2f0a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:11.549611 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5edb85-ee45-4695-9469-d00b590a2f0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:11.588878 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7287db77-b4c2-4a48-8740-1f98e1a7012d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:11.642907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a733eb-c62c-425c-9cd5-b3b9dbb5f5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:11.643134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5edb85-ee45-4695-9469-d00b590a2f0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:11.643654 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a733eb-c62c-425c-9cd5-b3b9dbb5f5d6 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:11.643947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a733eb-c62c-425c-9cd5-b3b9dbb5f5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:11.644226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a733eb-c62c-425c-9cd5-b3b9dbb5f5d6 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:11.644926 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5edb85-ee45-4695-9469-d00b590a2f0a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:11.645081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a733eb-c62c-425c-9cd5-b3b9dbb5f5d6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:11.645396 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a733eb-c62c-425c-9cd5-b3b9dbb5f5d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:11.647444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a733eb-c62c-425c-9cd5-b3b9dbb5f5d6 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:11.660813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 90/193] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:11.666892 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a733eb-c62c-425c-9cd5-b3b9dbb5f5d6 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:11.756326 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:11.764763 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:11.798390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 104/194] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:11.804375 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5edb85-ee45-4695-9469-d00b590a2f0a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:11.829784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a733eb-c62c-425c-9cd5-b3b9dbb5f5d6 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:11.888257 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc22da-7ed6-4938-8605-edeb74ac2a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:11.889502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc22da-7ed6-4938-8605-edeb74ac2a91 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:11.889900 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc22da-7ed6-4938-8605-edeb74ac2a91 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:11.890322 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc22da-7ed6-4938-8605-edeb74ac2a91 None admin] PATH_INFO: `/v3/users/8cb66d6e449f4da48b01b8976f5e0e80` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:11.892561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc22da-7ed6-4938-8605-edeb74ac2a91 None admin] RBAC: Authorizing `identity:get_user(user_id=8cb66d6e449f4da48b01b8976f5e0e80)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:11.894216 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc22da-7ed6-4938-8605-edeb74ac2a91 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:11.896598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 91/195] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:11 2022] GET /identity/v3/users/8cb66d6e449f4da48b01b8976f5e0e80 => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:11.905495 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc22da-7ed6-4938-8605-edeb74ac2a91 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:11.980363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5edb85-ee45-4695-9469-d00b590a2f0a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:12.018065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd834a-cd3e-46c6-a554-1a9b1268dba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:12.018952 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd834a-cd3e-46c6-a554-1a9b1268dba3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:12.019234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd834a-cd3e-46c6-a554-1a9b1268dba3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:12.019511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd834a-cd3e-46c6-a554-1a9b1268dba3 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:12.020789 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd834a-cd3e-46c6-a554-1a9b1268dba3 None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:12.022295 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd834a-cd3e-46c6-a554-1a9b1268dba3 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:12.024388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 105/196] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:11 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:12.030052 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd834a-cd3e-46c6-a554-1a9b1268dba3 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:12.036226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc22da-7ed6-4938-8605-edeb74ac2a91 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:12.067561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:12.080247 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13071be-02d3-4e92-8ff2-b3482fd120cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:12.083356 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13071be-02d3-4e92-8ff2-b3482fd120cf None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:12.083356 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13071be-02d3-4e92-8ff2-b3482fd120cf None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:12.083356 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13071be-02d3-4e92-8ff2-b3482fd120cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:12.111916 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:12.182467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd834a-cd3e-46c6-a554-1a9b1268dba3 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:12.191191 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13071be-02d3-4e92-8ff2-b3482fd120cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:12.192724 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13071be-02d3-4e92-8ff2-b3482fd120cf None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:12.234289 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1084825-7f83-478b-9dc1-8b900d95db31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:12.234890 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1084825-7f83-478b-9dc1-8b900d95db31 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:12.235240 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1084825-7f83-478b-9dc1-8b900d95db31 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:12.235524 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1084825-7f83-478b-9dc1-8b900d95db31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:12.236581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1084825-7f83-478b-9dc1-8b900d95db31 None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=c9a4c742053041558d2d857f27eedf48, user.id=8cb66d6e449f4da48b01b8976f5e0e80) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:12.236903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1084825-7f83-478b-9dc1-8b900d95db31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:12.238377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1084825-7f83-478b-9dc1-8b900d95db31 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:12.252492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 106/197] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:12 2022] GET /identity/v3/role_assignments?user.id=8cb66d6e449f4da48b01b8976f5e0e80&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:12.254104 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdabcd-24e1-4ddd-97a5-2cac788b5c18 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:12.254399 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdabcd-24e1-4ddd-97a5-2cac788b5c18 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:12.254681 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdabcd-24e1-4ddd-97a5-2cac788b5c18 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:12.264503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 107/198] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:12.270350 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4ba8-a64e-4cd1-bec4-19d78e29c43a None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:12.270644 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4ba8-a64e-4cd1-bec4-19d78e29c43a None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:12.270961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4ba8-a64e-4cd1-bec4-19d78e29c43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:12.308892 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-6dbd4ba8-a64e-4cd1-bec4-19d78e29c43a None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:12.310032 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-6dbd4ba8-a64e-4cd1-bec4-19d78e29c43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:12.339125 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 92/199] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:12.346313 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13071be-02d3-4e92-8ff2-b3482fd120cf None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:12.382599 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:12.484258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13071be-02d3-4e92-8ff2-b3482fd120cf None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:12.528363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 108/200] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:12.528904 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ea055-079d-47d6-a4b6-35bdb72f304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:12.528904 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ea055-079d-47d6-a4b6-35bdb72f304f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:12.528904 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ea055-079d-47d6-a4b6-35bdb72f304f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:12.528904 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ea055-079d-47d6-a4b6-35bdb72f304f None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:12.529290 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ea055-079d-47d6-a4b6-35bdb72f304f None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:12.530254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ea055-079d-47d6-a4b6-35bdb72f304f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:12.532313 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 93/201] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:12 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:12.538736 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd4ba8-a64e-4cd1-bec4-19d78e29c43a None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:12.553348 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ea055-079d-47d6-a4b6-35bdb72f304f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:12.651092 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:12.761048 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd4ba8-a64e-4cd1-bec4-19d78e29c43a None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:12.770848 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:12.781488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ea055-079d-47d6-a4b6-35bdb72f304f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:12.828156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494476d9-1065-4408-abcf-9c91e5ab7dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:12.828854 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494476d9-1065-4408-abcf-9c91e5ab7dd4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:12.829135 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494476d9-1065-4408-abcf-9c91e5ab7dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:12.829406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494476d9-1065-4408-abcf-9c91e5ab7dd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:12.830248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494476d9-1065-4408-abcf-9c91e5ab7dd4 None admin] RBAC: Adding query filter params (role.id=ff499a8782db4cce8e162a849be212ee, scope.project.id=c9a4c742053041558d2d857f27eedf48, user.id=8cb66d6e449f4da48b01b8976f5e0e80) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:12.830618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494476d9-1065-4408-abcf-9c91e5ab7dd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:12.832086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494476d9-1065-4408-abcf-9c91e5ab7dd4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:12.846014 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4affd8-2f4c-476a-8084-916bfffb12e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:12.846828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4affd8-2f4c-476a-8084-916bfffb12e0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:12.847108 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4affd8-2f4c-476a-8084-916bfffb12e0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:12.847384 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4affd8-2f4c-476a-8084-916bfffb12e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:12.850517 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 109/202] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:12 2022] GET /identity/v3/role_assignments?user.id=8cb66d6e449f4da48b01b8976f5e0e80&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=ff499a8782db4cce8e162a849be212ee => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:12.868468 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4affd8-2f4c-476a-8084-916bfffb12e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-2e4affd8-2f4c-476a-8084-916bfffb12e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:12.876423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:12.877505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:12.878593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 94/203] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:12.885540 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494476d9-1065-4408-abcf-9c91e5ab7dd4 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:13.026715 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494476d9-1065-4408-abcf-9c91e5ab7dd4 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:13.034432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:13.066104 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c09a4c-8446-46bf-86fe-736c2c97c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:13.066844 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c09a4c-8446-46bf-86fe-736c2c97c91e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:13.067129 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c09a4c-8446-46bf-86fe-736c2c97c91e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:13.067405 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c09a4c-8446-46bf-86fe-736c2c97c91e None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:13.068224 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c09a4c-8446-46bf-86fe-736c2c97c91e None admin] RBAC: Adding query filter params (name=reader) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:13.068563 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c09a4c-8446-46bf-86fe-736c2c97c91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:13.069961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c09a4c-8446-46bf-86fe-736c2c97c91e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:13.085278 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 110/204] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:13.091122 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4affd8-2f4c-476a-8084-916bfffb12e0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:13.247346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4affd8-2f4c-476a-8084-916bfffb12e0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:13.284247 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569686c-7598-4115-af5c-3890f950170d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:13.285435 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569686c-7598-4115-af5c-3890f950170d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:13.285873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569686c-7598-4115-af5c-3890f950170d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:13.286393 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569686c-7598-4115-af5c-3890f950170d None admin] PATH_INFO: `/v3/users/e3cc98fcaad44f8aac8763b6cd5825a7` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:13.288240 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569686c-7598-4115-af5c-3890f950170d None admin] RBAC: Authorizing `identity:get_user(user_id=e3cc98fcaad44f8aac8763b6cd5825a7)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:13.289970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569686c-7598-4115-af5c-3890f950170d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:13.292206 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 95/205] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:13 2022] GET /identity/v3/users/e3cc98fcaad44f8aac8763b6cd5825a7 => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:13.300251 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c09a4c-8446-46bf-86fe-736c2c97c91e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:13.448759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c09a4c-8446-46bf-86fe-736c2c97c91e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:13.500201 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b94b2-ba08-48dd-b318-72cffbd15366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:13.500905 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b94b2-ba08-48dd-b318-72cffbd15366 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:13.501187 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b94b2-ba08-48dd-b318-72cffbd15366 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:13.501481 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b94b2-ba08-48dd-b318-72cffbd15366 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:13.559864 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:13.589771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b94b2-ba08-48dd-b318-72cffbd15366 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:13.591254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b94b2-ba08-48dd-b318-72cffbd15366 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:13.700167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 111/206] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:13.705584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569686c-7598-4115-af5c-3890f950170d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:13.781602 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:13.902133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569686c-7598-4115-af5c-3890f950170d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:13.967071 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89914041-2d09-4c15-bb72-bbd63c578572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:13.968616 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89914041-2d09-4c15-bb72-bbd63c578572 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:13.969199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89914041-2d09-4c15-bb72-bbd63c578572 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:13.969781 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89914041-2d09-4c15-bb72-bbd63c578572 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:13.977406 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89914041-2d09-4c15-bb72-bbd63c578572 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:13.979704 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89914041-2d09-4c15-bb72-bbd63c578572 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:13.982780 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 96/207] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:13 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:13.988170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b94b2-ba08-48dd-b318-72cffbd15366 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:14.094984 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:14.154567 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b94b2-ba08-48dd-b318-72cffbd15366 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:14.220075 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acce6f-83d9-442d-8220-552be1e7afce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:14.220793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acce6f-83d9-442d-8220-552be1e7afce None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:14.221079 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acce6f-83d9-442d-8220-552be1e7afce None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:14.221356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acce6f-83d9-442d-8220-552be1e7afce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:14.222211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acce6f-83d9-442d-8220-552be1e7afce None admin] RBAC: Adding query filter params (role.id=7b870ddcb3b04c6088d9d80f29ab8193, scope.project.id=464c75ba6b6340e4b017049e6993fe4d, user.id=e3cc98fcaad44f8aac8763b6cd5825a7) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:14.222525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acce6f-83d9-442d-8220-552be1e7afce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:14.224041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acce6f-83d9-442d-8220-552be1e7afce None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:14.231201 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 112/208] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:13 2022] GET /identity/v3/role_assignments?user.id=e3cc98fcaad44f8aac8763b6cd5825a7&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=7b870ddcb3b04c6088d9d80f29ab8193 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:14.709822 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce273f-d116-41c6-ba19-df370a37be66 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:14.710025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce273f-d116-41c6-ba19-df370a37be66 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:14.710296 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce273f-d116-41c6-ba19-df370a37be66 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:14.711692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 97/209] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:14.718437 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93283a-2f1e-4970-8935-c2c5c2e49256 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:14.719414 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93283a-2f1e-4970-8935-c2c5c2e49256 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:14.719907 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d93283a-2f1e-4970-8935-c2c5c2e49256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:14.748366 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-6d93283a-2f1e-4970-8935-c2c5c2e49256 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:14.751017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-6d93283a-2f1e-4970-8935-c2c5c2e49256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:14.767665 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f700a-7ab1-46d9-800c-b10cb7a7dedf None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:14.768668 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f700a-7ab1-46d9-800c-b10cb7a7dedf None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:14.769460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75f700a-7ab1-46d9-800c-b10cb7a7dedf None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:14.772577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 98/210] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:14 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:14.780579 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30985943-6f18-4d8b-b002-faec4aecf52f None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:14.781271 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30985943-6f18-4d8b-b002-faec4aecf52f None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:14.781787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30985943-6f18-4d8b-b002-faec4aecf52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:14.810256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-30985943-6f18-4d8b-b002-faec4aecf52f None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:14.812951 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-30985943-6f18-4d8b-b002-faec4aecf52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:14.877316 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 113/211] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:14.912170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93283a-2f1e-4970-8935-c2c5c2e49256 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:14.920995 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 99/212] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:14.960050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30985943-6f18-4d8b-b002-faec4aecf52f None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:14.988956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:14.990219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:15.031654 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d93283a-2f1e-4970-8935-c2c5c2e49256 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:15.073531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a56d5f-e2a8-4099-bb2c-dd38ad780cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:15.074658 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a56d5f-e2a8-4099-bb2c-dd38ad780cc9 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.075111 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a56d5f-e2a8-4099-bb2c-dd38ad780cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.075543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a56d5f-e2a8-4099-bb2c-dd38ad780cc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:15.077024 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a56d5f-e2a8-4099-bb2c-dd38ad780cc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:15.077704 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30985943-6f18-4d8b-b002-faec4aecf52f None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:15.078879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a56d5f-e2a8-4099-bb2c-dd38ad780cc9 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:15.082202 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 114/213] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:15.084398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a88399-8d42-456f-9cee-e72afca904d9 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.085264 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a88399-8d42-456f-9cee-e72afca904d9 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.085700 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a88399-8d42-456f-9cee-e72afca904d9 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:15.087351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 115/214] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:15.091858 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a88399-8d42-456f-9cee-e72afca904d9 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:15.131302 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21ab05-288c-44d8-81d3-ead2a215b73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:15.132583 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21ab05-288c-44d8-81d3-ead2a215b73c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.133854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21ab05-288c-44d8-81d3-ead2a215b73c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.134274 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21ab05-288c-44d8-81d3-ead2a215b73c None admin] PATH_INFO: `/v3/roles/member` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:15.150332 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21ab05-288c-44d8-81d3-ead2a215b73c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-5b21ab05-288c-44d8-81d3-ead2a215b73c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:15.161484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:15.162840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:15.163787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 100/215] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:15.165460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23ef60-9894-43d8-b395-2adfc98c4a60 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.165667 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23ef60-9894-43d8-b395-2adfc98c4a60 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.165949 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac23ef60-9894-43d8-b395-2adfc98c4a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:15.187315 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-ac23ef60-9894-43d8-b395-2adfc98c4a60 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:15.188580 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-ac23ef60-9894-43d8-b395-2adfc98c4a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:15.220581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a88399-8d42-456f-9cee-e72afca904d9 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:15.252724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:15.258788 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2f146-ffcf-4a71-b8a1-377b9ca97533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:15.259503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2f146-ffcf-4a71-b8a1-377b9ca97533 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.259886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2f146-ffcf-4a71-b8a1-377b9ca97533 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.260168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2f146-ffcf-4a71-b8a1-377b9ca97533 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:15.260923 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2f146-ffcf-4a71-b8a1-377b9ca97533 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:15.262379 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2f146-ffcf-4a71-b8a1-377b9ca97533 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:15.275328 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 101/216] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:15.276254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23ef60-9894-43d8-b395-2adfc98c4a60 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:15.299532 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:15.391588 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac23ef60-9894-43d8-b395-2adfc98c4a60 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:15.392413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 116/217] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:28:15 2022] POST /identity/v3/users => generated 297 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:15.401494 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2f146-ffcf-4a71-b8a1-377b9ca97533 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:15.448194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64debce-51a4-44ad-9df0-6cfe38e762df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:15.449363 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64debce-51a4-44ad-9df0-6cfe38e762df None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.450020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64debce-51a4-44ad-9df0-6cfe38e762df None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.450504 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64debce-51a4-44ad-9df0-6cfe38e762df None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:15.451631 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64debce-51a4-44ad-9df0-6cfe38e762df None admin] RBAC: Adding query filter params (name=member) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:15.453735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64debce-51a4-44ad-9df0-6cfe38e762df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:15.455842 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64debce-51a4-44ad-9df0-6cfe38e762df None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:15.470416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 102/218] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:15.477123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64debce-51a4-44ad-9df0-6cfe38e762df None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:15.565542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2f146-ffcf-4a71-b8a1-377b9ca97533 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:15.592778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:15.607345 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64debce-51a4-44ad-9df0-6cfe38e762df None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:15.607567 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09886e-b72d-4744-b56d-c2678243f0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:15.608277 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09886e-b72d-4744-b56d-c2678243f0ab None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.608593 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09886e-b72d-4744-b56d-c2678243f0ab None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.608869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09886e-b72d-4744-b56d-c2678243f0ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:15.626585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09886e-b72d-4744-b56d-c2678243f0ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-ec09886e-b72d-4744-b56d-c2678243f0ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:15.635774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:15.637256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:15.638568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 117/219] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:28:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:15.643270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09886e-b72d-4744-b56d-c2678243f0ab None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:15.653910 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e4794-a8e1-4513-8fa8-5de0b8ad39a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:15.654397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e4794-a8e1-4513-8fa8-5de0b8ad39a8 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.654685 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e4794-a8e1-4513-8fa8-5de0b8ad39a8 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.655304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e4794-a8e1-4513-8fa8-5de0b8ad39a8 None admin] PATH_INFO: `/v3/users/8107fbaddf514ec09825c4b1886aedb1` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:15.673582 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e4794-a8e1-4513-8fa8-5de0b8ad39a8 None admin] RBAC: Authorizing `identity:get_user(user_id=8107fbaddf514ec09825c4b1886aedb1)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:15.675060 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e4794-a8e1-4513-8fa8-5de0b8ad39a8 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:15.677664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 103/220] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:15 2022] GET /identity/v3/users/8107fbaddf514ec09825c4b1886aedb1 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:15.678885 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bbfb8-28e3-4a65-acfc-65775be92555 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.679187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bbfb8-28e3-4a65-acfc-65775be92555 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.679468 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bbfb8-28e3-4a65-acfc-65775be92555 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:15.680815 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 104/221] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:15.686118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bbfb8-28e3-4a65-acfc-65775be92555 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:15.707081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:15.784218 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09886e-b72d-4744-b56d-c2678243f0ab None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:15.833986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95534d0-a187-4042-9f6b-c8589a02ac91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:15.834797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95534d0-a187-4042-9f6b-c8589a02ac91 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.835410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95534d0-a187-4042-9f6b-c8589a02ac91 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.835868 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95534d0-a187-4042-9f6b-c8589a02ac91 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:15.837216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95534d0-a187-4042-9f6b-c8589a02ac91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:15.837735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95534d0-a187-4042-9f6b-c8589a02ac91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:15.840436 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95534d0-a187-4042-9f6b-c8589a02ac91 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:15.856057 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bbfb8-28e3-4a65-acfc-65775be92555 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:15.857478 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 118/222] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:15.867038 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce858bc-74dd-4f1d-a5be-e69593271aef None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.867646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce858bc-74dd-4f1d-a5be-e69593271aef None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.868127 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce858bc-74dd-4f1d-a5be-e69593271aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:15.894537 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:15.895529 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-cce858bc-74dd-4f1d-a5be-e69593271aef None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:15.897883 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-cce858bc-74dd-4f1d-a5be-e69593271aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:15.902406 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb9306-ae1a-49cd-a231-79c760e3ffbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:15.904424 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9306-ae1a-49cd-a231-79c760e3ffbc None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:15.904901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9306-ae1a-49cd-a231-79c760e3ffbc None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:15.912221 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb9306-ae1a-49cd-a231-79c760e3ffbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:16.007859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb9306-ae1a-49cd-a231-79c760e3ffbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:16.008938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 119/223] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:16.010029 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce858bc-74dd-4f1d-a5be-e69593271aef None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:16.010859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb9306-ae1a-49cd-a231-79c760e3ffbc None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:16.062684 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:16.133501 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce858bc-74dd-4f1d-a5be-e69593271aef None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:16.153240 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 105/224] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:16.154736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb9306-ae1a-49cd-a231-79c760e3ffbc None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:16.171306 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb252a06-4e95-41fc-b9a8-42e394f58a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:16.172196 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252a06-4e95-41fc-b9a8-42e394f58a98 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:16.172532 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252a06-4e95-41fc-b9a8-42e394f58a98 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:16.172810 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb252a06-4e95-41fc-b9a8-42e394f58a98 None admin] PATH_INFO: `/v3/users/8cb66d6e449f4da48b01b8976f5e0e80` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:16.174454 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb252a06-4e95-41fc-b9a8-42e394f58a98 None admin] RBAC: Authorizing `identity:get_user(user_id=8cb66d6e449f4da48b01b8976f5e0e80)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:16.176380 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb252a06-4e95-41fc-b9a8-42e394f58a98 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:16.178828 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 120/225] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:16 2022] GET /identity/v3/users/8cb66d6e449f4da48b01b8976f5e0e80 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:16.184778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb252a06-4e95-41fc-b9a8-42e394f58a98 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:16.194765 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:16.288373 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb9306-ae1a-49cd-a231-79c760e3ffbc None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:16.315107 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb252a06-4e95-41fc-b9a8-42e394f58a98 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:16.323581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:16.332844 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ac9f8-3702-4b38-824f-209e511110b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:16.336477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ac9f8-3702-4b38-824f-209e511110b7 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:16.336477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ac9f8-3702-4b38-824f-209e511110b7 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:16.336477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ac9f8-3702-4b38-824f-209e511110b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:16.351143 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ac9f8-3702-4b38-824f-209e511110b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:16.352233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c925a50-ee5b-4837-8e69-6bf56a4d0953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:16.353198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c925a50-ee5b-4837-8e69-6bf56a4d0953 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:16.353531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c925a50-ee5b-4837-8e69-6bf56a4d0953 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:16.353922 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c925a50-ee5b-4837-8e69-6bf56a4d0953 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:16.355180 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c925a50-ee5b-4837-8e69-6bf56a4d0953 None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:16.356718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c925a50-ee5b-4837-8e69-6bf56a4d0953 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:16.359216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 121/226] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:16 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:16.360383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c925a50-ee5b-4837-8e69-6bf56a4d0953 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-670ac9f8-3702-4b38-824f-209e511110b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:16.362628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:16.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:16.364850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:16.366135 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 106/227] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:16.369190 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ac9f8-3702-4b38-824f-209e511110b7 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:16.507767 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c925a50-ee5b-4837-8e69-6bf56a4d0953 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:16.515418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:16.530126 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ac9f8-3702-4b38-824f-209e511110b7 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:16.550327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43e7ad-f9f6-4e93-bfdc-eb61edccdcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:16.551731 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43e7ad-f9f6-4e93-bfdc-eb61edccdcd4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:16.552802 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43e7ad-f9f6-4e93-bfdc-eb61edccdcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:16.553310 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43e7ad-f9f6-4e93-bfdc-eb61edccdcd4 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:16.554990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43e7ad-f9f6-4e93-bfdc-eb61edccdcd4 None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:16.556741 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43e7ad-f9f6-4e93-bfdc-eb61edccdcd4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:16.563516 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 122/228] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:16 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:16.564815 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43e7ad-f9f6-4e93-bfdc-eb61edccdcd4 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:16.573635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6505bda-2864-434e-8cca-26e917676fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:16.574223 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6505bda-2864-434e-8cca-26e917676fe2 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:16.574512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6505bda-2864-434e-8cca-26e917676fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:16.574787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6505bda-2864-434e-8cca-26e917676fe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:16.575636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6505bda-2864-434e-8cca-26e917676fe2 None admin] RBAC: Adding query filter params (role.id=c8be6a897df546ca9b0e7b9541fb94d5, scope.project.id=c9a4c742053041558d2d857f27eedf48, user.id=8107fbaddf514ec09825c4b1886aedb1) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:16.575954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6505bda-2864-434e-8cca-26e917676fe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:16.577488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6505bda-2864-434e-8cca-26e917676fe2 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:16.585506 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 107/229] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:16 2022] GET /identity/v3/role_assignments?user.id=8107fbaddf514ec09825c4b1886aedb1&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:16.586758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6505bda-2864-434e-8cca-26e917676fe2 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:16.623485 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:16.765807 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43e7ad-f9f6-4e93-bfdc-eb61edccdcd4 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:16.817177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6505bda-2864-434e-8cca-26e917676fe2 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:16.829259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e3d11-c3a2-4200-80af-ba2e89f81fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:16.830685 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3d11-c3a2-4200-80af-ba2e89f81fbf None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:16.831207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3d11-c3a2-4200-80af-ba2e89f81fbf None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:16.831490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e3d11-c3a2-4200-80af-ba2e89f81fbf None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:16.832289 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e3d11-c3a2-4200-80af-ba2e89f81fbf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:16.832635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e3d11-c3a2-4200-80af-ba2e89f81fbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:16.834176 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e3d11-c3a2-4200-80af-ba2e89f81fbf None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:16.852259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 123/230] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:16.859979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e3d11-c3a2-4200-80af-ba2e89f81fbf None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:16.870510 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40397a3f-efcc-4a0a-85d2-fc0b97a8460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:16.872486 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397a3f-efcc-4a0a-85d2-fc0b97a8460d None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:16.873162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397a3f-efcc-4a0a-85d2-fc0b97a8460d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:16.874340 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397a3f-efcc-4a0a-85d2-fc0b97a8460d None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48/users/8cb66d6e449f4da48b01b8976f5e0e80/roles/8eaff9cd613a49b29548e90d7e3e9215` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:16.883303 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40397a3f-efcc-4a0a-85d2-fc0b97a8460d None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a4c742053041558d2d857f27eedf48, user_id=8cb66d6e449f4da48b01b8976f5e0e80, role_id=8eaff9cd613a49b29548e90d7e3e9215)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:16.885815 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40397a3f-efcc-4a0a-85d2-fc0b97a8460d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:16.962481 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 108/231] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:28:16 2022] PUT /identity/v3/projects/c9a4c742053041558d2d857f27eedf48/users/8cb66d6e449f4da48b01b8976f5e0e80/roles/8eaff9cd613a49b29548e90d7e3e9215 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:17.001387 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e3d11-c3a2-4200-80af-ba2e89f81fbf None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:17.004467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:17.043759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a35289-94cb-4d6c-b2ef-2cbfa89f4e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:17.045223 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a35289-94cb-4d6c-b2ef-2cbfa89f4e25 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:17.045511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a35289-94cb-4d6c-b2ef-2cbfa89f4e25 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:17.045790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a35289-94cb-4d6c-b2ef-2cbfa89f4e25 None admin] PATH_INFO: `/v3/users/8cb66d6e449f4da48b01b8976f5e0e80` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:17.047254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a35289-94cb-4d6c-b2ef-2cbfa89f4e25 None admin] RBAC: Authorizing `identity:get_user(user_id=8cb66d6e449f4da48b01b8976f5e0e80)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:17.048936 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a35289-94cb-4d6c-b2ef-2cbfa89f4e25 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:17.051189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 124/232] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:16 2022] GET /identity/v3/users/8cb66d6e449f4da48b01b8976f5e0e80 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:17.057284 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40397a3f-efcc-4a0a-85d2-fc0b97a8460d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:17.097308 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:17.237849 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40397a3f-efcc-4a0a-85d2-fc0b97a8460d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:17.284864 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991fc1c-d960-48ad-a74b-867ed4ca834e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:17.285764 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991fc1c-d960-48ad-a74b-867ed4ca834e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:17.286050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991fc1c-d960-48ad-a74b-867ed4ca834e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:17.286381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991fc1c-d960-48ad-a74b-867ed4ca834e None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:17.288162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991fc1c-d960-48ad-a74b-867ed4ca834e None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:17.289883 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991fc1c-d960-48ad-a74b-867ed4ca834e None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:17.292180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 109/233] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:17 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:17.298526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a35289-94cb-4d6c-b2ef-2cbfa89f4e25 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:17.454181 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a35289-94cb-4d6c-b2ef-2cbfa89f4e25 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:17.514371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36415bc7-0380-4a01-b106-ad955c2fe7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:17.515279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36415bc7-0380-4a01-b106-ad955c2fe7f1 None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:17.515564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36415bc7-0380-4a01-b106-ad955c2fe7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:17.515875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36415bc7-0380-4a01-b106-ad955c2fe7f1 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48/users/8cb66d6e449f4da48b01b8976f5e0e80/roles/ff499a8782db4cce8e162a849be212ee` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:17.526658 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36415bc7-0380-4a01-b106-ad955c2fe7f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a4c742053041558d2d857f27eedf48, user_id=8cb66d6e449f4da48b01b8976f5e0e80, role_id=ff499a8782db4cce8e162a849be212ee)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:17.526658 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36415bc7-0380-4a01-b106-ad955c2fe7f1 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:17.661033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 125/234] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:28:17 2022] PUT /identity/v3/projects/c9a4c742053041558d2d857f27eedf48/users/8cb66d6e449f4da48b01b8976f5e0e80/roles/ff499a8782db4cce8e162a849be212ee => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:28:18.211431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deacf8-fc5f-4557-a144-368152f1c73c None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:18.211958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deacf8-fc5f-4557-a144-368152f1c73c None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:18.212040 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deacf8-fc5f-4557-a144-368152f1c73c None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:18.213901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 110/235] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:18.234196 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e562233-9b59-467e-b121-31e008d05167 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:18.235212 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e562233-9b59-467e-b121-31e008d05167 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:18.235701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e562233-9b59-467e-b121-31e008d05167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:18.261381 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-4e562233-9b59-467e-b121-31e008d05167 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:18.262967 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-4e562233-9b59-467e-b121-31e008d05167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:18.295710 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:18.357863 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 126/236] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:18.402313 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deacf8-fc5f-4557-a144-368152f1c73c None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:18.534779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:18.563741 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deacf8-fc5f-4557-a144-368152f1c73c None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:18.595163 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dadf7-9b4d-4be5-bdc0-2918fedebcf8 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:18.595600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dadf7-9b4d-4be5-bdc0-2918fedebcf8 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:18.596006 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431dadf7-9b4d-4be5-bdc0-2918fedebcf8 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:18.599711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 127/237] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:18.604577 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cbb74-a378-47d8-8e80-1743f0d4b0d1 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:18.604995 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cbb74-a378-47d8-8e80-1743f0d4b0d1 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:18.605382 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cbb74-a378-47d8-8e80-1743f0d4b0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:18.619156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963a281-2611-48e8-9a6f-537c5a26581f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:18.620003 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963a281-2611-48e8-9a6f-537c5a26581f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:18.620283 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963a281-2611-48e8-9a6f-537c5a26581f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:18.620581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963a281-2611-48e8-9a6f-537c5a26581f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:18.633285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-a64cbb74-a378-47d8-8e80-1743f0d4b0d1 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:18.634599 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-a64cbb74-a378-47d8-8e80-1743f0d4b0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:18.648454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3963a281-2611-48e8-9a6f-537c5a26581f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-3963a281-2611-48e8-9a6f-537c5a26581f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:18.656365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:18.657411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:18.658294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 111/238] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:18.667969 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963a281-2611-48e8-9a6f-537c5a26581f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:18.745232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 128/239] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:18.809170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cbb74-a378-47d8-8e80-1743f0d4b0d1 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:18.886841 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963a281-2611-48e8-9a6f-537c5a26581f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:18.948710 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65cf2a-b890-45cf-8e0c-3eee762ca028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:18.949395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65cf2a-b890-45cf-8e0c-3eee762ca028 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:18.949679 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65cf2a-b890-45cf-8e0c-3eee762ca028 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:18.949958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65cf2a-b890-45cf-8e0c-3eee762ca028 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:18.950721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65cf2a-b890-45cf-8e0c-3eee762ca028 None admin] RBAC: Adding query filter params (name=reader) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:18.951026 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65cf2a-b890-45cf-8e0c-3eee762ca028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:18.952437 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a65cf2a-b890-45cf-8e0c-3eee762ca028 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:18.966349 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 112/240] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:18.967650 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65cf2a-b890-45cf-8e0c-3eee762ca028 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:18.977392 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:19.033962 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:19.069294 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cbb74-a378-47d8-8e80-1743f0d4b0d1 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:19.111117 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67067b4b-a92b-446f-bc42-804a79545e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:19.112193 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67067b4b-a92b-446f-bc42-804a79545e59 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:19.112651 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67067b4b-a92b-446f-bc42-804a79545e59 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:19.113616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67067b4b-a92b-446f-bc42-804a79545e59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:19.115781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67067b4b-a92b-446f-bc42-804a79545e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:19.117434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67067b4b-a92b-446f-bc42-804a79545e59 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:19.123983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 129/241] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:19.125486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67067b4b-a92b-446f-bc42-804a79545e59 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:19.125893 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65cf2a-b890-45cf-8e0c-3eee762ca028 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:19.233196 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02378eff-afeb-4388-befa-970b8c5ed620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:19.234104 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02378eff-afeb-4388-befa-970b8c5ed620 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:19.234392 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02378eff-afeb-4388-befa-970b8c5ed620 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:19.234673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02378eff-afeb-4388-befa-970b8c5ed620 None admin] PATH_INFO: `/v3/users/33e37a49b3f340ee8c92713a3b600de1` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:19.261045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67067b4b-a92b-446f-bc42-804a79545e59 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:19.266951 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02378eff-afeb-4388-befa-970b8c5ed620 None admin] RBAC: Authorizing `identity:get_user(user_id=33e37a49b3f340ee8c92713a3b600de1)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:19.268484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02378eff-afeb-4388-befa-970b8c5ed620 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:19.274856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 113/242] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:18 2022] GET /identity/v3/users/33e37a49b3f340ee8c92713a3b600de1 => generated 297 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:19.289662 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02378eff-afeb-4388-befa-970b8c5ed620 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:19.310311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5560264-4509-4768-b94b-5548fa0880d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:19.311349 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5560264-4509-4768-b94b-5548fa0880d9 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:19.311791 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5560264-4509-4768-b94b-5548fa0880d9 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:19.312223 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5560264-4509-4768-b94b-5548fa0880d9 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:19.313925 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5560264-4509-4768-b94b-5548fa0880d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:19.315749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5560264-4509-4768-b94b-5548fa0880d9 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:19.383463 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:19.409899 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 130/243] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:28:19 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:28:19.427456 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02378eff-afeb-4388-befa-970b8c5ed620 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:19.478160 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9101919-96e5-4fa8-a0cc-a9fadd7c66b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:19.478854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9101919-96e5-4fa8-a0cc-a9fadd7c66b1 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:19.479142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9101919-96e5-4fa8-a0cc-a9fadd7c66b1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:19.479419 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9101919-96e5-4fa8-a0cc-a9fadd7c66b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:19.578016 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce773854-61de-4946-aa14-5b8cf5f07d25 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:19.578361 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce773854-61de-4946-aa14-5b8cf5f07d25 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:19.578585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce773854-61de-4946-aa14-5b8cf5f07d25 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:19.580245 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 131/244] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:19.584012 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9101919-96e5-4fa8-a0cc-a9fadd7c66b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:19.585646 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9101919-96e5-4fa8-a0cc-a9fadd7c66b1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:19.588478 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e9046-9fdc-4544-aa41-e19b00dfb3e6 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:19.589185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e9046-9fdc-4544-aa41-e19b00dfb3e6 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:19.589713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e9046-9fdc-4544-aa41-e19b00dfb3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:19.621160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-456e9046-9fdc-4544-aa41-e19b00dfb3e6 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:19.622874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-456e9046-9fdc-4544-aa41-e19b00dfb3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:19.643664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:19.701212 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:19.722810 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 132/245] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:19.747538 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e9046-9fdc-4544-aa41-e19b00dfb3e6 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:19.749758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 114/246] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:19.756975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9101919-96e5-4fa8-a0cc-a9fadd7c66b1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:19.901843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9101919-96e5-4fa8-a0cc-a9fadd7c66b1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:19.916907 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e9046-9fdc-4544-aa41-e19b00dfb3e6 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:19.962808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b073c-d3bf-4ad4-91ec-2ad151ec6b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:19.964298 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b073c-d3bf-4ad4-91ec-2ad151ec6b17 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:19.964617 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b073c-d3bf-4ad4-91ec-2ad151ec6b17 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:19.964896 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b073c-d3bf-4ad4-91ec-2ad151ec6b17 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:19.966497 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b073c-d3bf-4ad4-91ec-2ad151ec6b17 None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:19.967991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4b073c-d3bf-4ad4-91ec-2ad151ec6b17 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:19.970804 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593049a-3bb8-4651-bc8c-872d2f267030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:19.971638 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593049a-3bb8-4651-bc8c-872d2f267030 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:19.971924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593049a-3bb8-4651-bc8c-872d2f267030 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:19.972199 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593049a-3bb8-4651-bc8c-872d2f267030 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:19.976979 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 115/247] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:19 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:19.984044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b073c-d3bf-4ad4-91ec-2ad151ec6b17 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:19.993250 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593049a-3bb8-4651-bc8c-872d2f267030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-a593049a-3bb8-4651-bc8c-872d2f267030 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:20.008692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:20.009783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:20.010694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 133/248] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:20.018900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593049a-3bb8-4651-bc8c-872d2f267030 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:20.030155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:20.034919 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:20.132688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4b073c-d3bf-4ad4-91ec-2ad151ec6b17 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:20.169208 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593049a-3bb8-4651-bc8c-872d2f267030 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:20.181497 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52300bf-23e6-4533-b938-ddb7682f8e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:20.182559 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52300bf-23e6-4533-b938-ddb7682f8e54 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:20.182853 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52300bf-23e6-4533-b938-ddb7682f8e54 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:20.183447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52300bf-23e6-4533-b938-ddb7682f8e54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:20.184322 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52300bf-23e6-4533-b938-ddb7682f8e54 None admin] RBAC: Adding query filter params (role.id=7b870ddcb3b04c6088d9d80f29ab8193, scope.project.id=c9a4c742053041558d2d857f27eedf48, user.id=33e37a49b3f340ee8c92713a3b600de1) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:20.184675 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52300bf-23e6-4533-b938-ddb7682f8e54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:20.186406 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52300bf-23e6-4533-b938-ddb7682f8e54 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:20.194866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 116/249] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:19 2022] GET /identity/v3/role_assignments?user.id=33e37a49b3f340ee8c92713a3b600de1&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=7b870ddcb3b04c6088d9d80f29ab8193 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:20.216746 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fff4a2-81fa-44ac-92d9-8400163b1e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:20.217568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fff4a2-81fa-44ac-92d9-8400163b1e0e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:20.217929 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fff4a2-81fa-44ac-92d9-8400163b1e0e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:20.218289 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fff4a2-81fa-44ac-92d9-8400163b1e0e None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:20.219168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fff4a2-81fa-44ac-92d9-8400163b1e0e None admin] RBAC: Adding query filter params (name=member) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:20.219557 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fff4a2-81fa-44ac-92d9-8400163b1e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:20.221226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fff4a2-81fa-44ac-92d9-8400163b1e0e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:20.224735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0747-55fe-4f73-b7cd-5d2e8998cf46 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:20.225031 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0747-55fe-4f73-b7cd-5d2e8998cf46 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:20.225320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d0747-55fe-4f73-b7cd-5d2e8998cf46 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:20.227032 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 117/250] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:28:20.233245 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 134/251] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:20.233912 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed67ab-9ae8-4c43-b1d8-c4896247c336 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:20.234448 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed67ab-9ae8-4c43-b1d8-c4896247c336 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:20.234619 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed67ab-9ae8-4c43-b1d8-c4896247c336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:20.241626 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fff4a2-81fa-44ac-92d9-8400163b1e0e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:20.273659 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-f5ed67ab-9ae8-4c43-b1d8-c4896247c336 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:20.274942 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-f5ed67ab-9ae8-4c43-b1d8-c4896247c336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:20.351336 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:20.402617 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 118/252] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:20.409779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fff4a2-81fa-44ac-92d9-8400163b1e0e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:20.449011 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed67ab-9ae8-4c43-b1d8-c4896247c336 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:20.464207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14861e72-51dc-469b-a894-6a20af9ffa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:20.465152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14861e72-51dc-469b-a894-6a20af9ffa4d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:20.465437 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14861e72-51dc-469b-a894-6a20af9ffa4d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:20.465713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14861e72-51dc-469b-a894-6a20af9ffa4d None admin] PATH_INFO: `/v3/users/8107fbaddf514ec09825c4b1886aedb1` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:20.473459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14861e72-51dc-469b-a894-6a20af9ffa4d None admin] RBAC: Authorizing `identity:get_user(user_id=8107fbaddf514ec09825c4b1886aedb1)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:20.475072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14861e72-51dc-469b-a894-6a20af9ffa4d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:20.477292 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 135/253] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:20 2022] GET /identity/v3/users/8107fbaddf514ec09825c4b1886aedb1 => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:20.482902 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14861e72-51dc-469b-a894-6a20af9ffa4d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:20.483377 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:20.575646 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed67ab-9ae8-4c43-b1d8-c4896247c336 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:20.618415 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204f30e-a42c-4ea0-8932-28551f7d2c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:20.619250 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204f30e-a42c-4ea0-8932-28551f7d2c2a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:20.619531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204f30e-a42c-4ea0-8932-28551f7d2c2a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:20.619811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204f30e-a42c-4ea0-8932-28551f7d2c2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:20.643228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204f30e-a42c-4ea0-8932-28551f7d2c2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-7204f30e-a42c-4ea0-8932-28551f7d2c2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:20.651509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:20.654133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:20.655778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 119/254] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:28:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:20.659302 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204f30e-a42c-4ea0-8932-28551f7d2c2a None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:20.686436 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14861e72-51dc-469b-a894-6a20af9ffa4d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:20.761629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:20.779853 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204f30e-a42c-4ea0-8932-28551f7d2c2a None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:20.792141 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e5ec7-cf98-4a2a-9aa0-0bb03a66befe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:20.793347 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e5ec7-cf98-4a2a-9aa0-0bb03a66befe None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:20.794220 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e5ec7-cf98-4a2a-9aa0-0bb03a66befe None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:20.794534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e5ec7-cf98-4a2a-9aa0-0bb03a66befe None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:20.795957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e5ec7-cf98-4a2a-9aa0-0bb03a66befe None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:20.797427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e5ec7-cf98-4a2a-9aa0-0bb03a66befe None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:20.799461 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 136/255] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:20 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:20.800606 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292689f-0970-4044-a459-547677d1d46a None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:20.800836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292689f-0970-4044-a459-547677d1d46a None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:20.801120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292689f-0970-4044-a459-547677d1d46a None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:20.809314 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 137/256] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:20.810435 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292689f-0970-4044-a459-547677d1d46a None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:20.818797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:20.827593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf842c-9d33-4899-a88a-e6b1fbc7e818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:20.828571 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf842c-9d33-4899-a88a-e6b1fbc7e818 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:20.836106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf842c-9d33-4899-a88a-e6b1fbc7e818 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:20.836106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf842c-9d33-4899-a88a-e6b1fbc7e818 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:20.836106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf842c-9d33-4899-a88a-e6b1fbc7e818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:20.836106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf842c-9d33-4899-a88a-e6b1fbc7e818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:20.836106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf842c-9d33-4899-a88a-e6b1fbc7e818 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:20.843261 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 120/257] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:20.849717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8c1e3-cb28-47f7-8ea8-3abae0822bbb None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:20.850110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8c1e3-cb28-47f7-8ea8-3abae0822bbb None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:20.850763 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8c1e3-cb28-47f7-8ea8-3abae0822bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:20.894178 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-1ee8c1e3-cb28-47f7-8ea8-3abae0822bbb None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:20.896301 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-1ee8c1e3-cb28-47f7-8ea8-3abae0822bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:20.966159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292689f-0970-4044-a459-547677d1d46a None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:21.006035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d80804-9b5c-4d11-a010-cfeef2518f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:21.007691 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d80804-9b5c-4d11-a010-cfeef2518f9e None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:21.008230 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d80804-9b5c-4d11-a010-cfeef2518f9e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:21.008762 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d80804-9b5c-4d11-a010-cfeef2518f9e None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48/users/8107fbaddf514ec09825c4b1886aedb1/roles/c8be6a897df546ca9b0e7b9541fb94d5` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:21.012127 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d80804-9b5c-4d11-a010-cfeef2518f9e None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a4c742053041558d2d857f27eedf48, user_id=8107fbaddf514ec09825c4b1886aedb1, role_id=c8be6a897df546ca9b0e7b9541fb94d5)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:21.014121 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d80804-9b5c-4d11-a010-cfeef2518f9e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:21.067179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 121/258] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:21.068298 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8c1e3-cb28-47f7-8ea8-3abae0822bbb None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:21.093982 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 138/259] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:28:20 2022] PUT /identity/v3/projects/c9a4c742053041558d2d857f27eedf48/users/8107fbaddf514ec09825c4b1886aedb1/roles/c8be6a897df546ca9b0e7b9541fb94d5 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:21.103435 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d80804-9b5c-4d11-a010-cfeef2518f9e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:21.187640 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:21.259264 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d80804-9b5c-4d11-a010-cfeef2518f9e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:21.261411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:21.309152 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8c1e3-cb28-47f7-8ea8-3abae0822bbb None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:21.338630 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87609fe8-4715-4a70-8c77-f890dc61006e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:21.339487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87609fe8-4715-4a70-8c77-f890dc61006e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:21.339808 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87609fe8-4715-4a70-8c77-f890dc61006e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:21.340083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87609fe8-4715-4a70-8c77-f890dc61006e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:21.368427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87609fe8-4715-4a70-8c77-f890dc61006e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-87609fe8-4715-4a70-8c77-f890dc61006e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:21.379646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:21.381254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:21.382234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:21.385682 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 139/260] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:21.387886 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552ffe7-e632-420a-bf7d-fbc1e96054e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:21.388953 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552ffe7-e632-420a-bf7d-fbc1e96054e6 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:21.389243 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552ffe7-e632-420a-bf7d-fbc1e96054e6 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:21.389523 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552ffe7-e632-420a-bf7d-fbc1e96054e6 None admin] PATH_INFO: `/v3/users/8cb66d6e449f4da48b01b8976f5e0e80` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:21.392978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552ffe7-e632-420a-bf7d-fbc1e96054e6 None admin] RBAC: Authorizing `identity:get_user(user_id=8cb66d6e449f4da48b01b8976f5e0e80)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:21.394488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e552ffe7-e632-420a-bf7d-fbc1e96054e6 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:21.396375 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 122/261] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:21 2022] GET /identity/v3/users/8cb66d6e449f4da48b01b8976f5e0e80 => generated 283 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:21.399727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87609fe8-4715-4a70-8c77-f890dc61006e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:21.404727 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552ffe7-e632-420a-bf7d-fbc1e96054e6 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:21.597604 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552ffe7-e632-420a-bf7d-fbc1e96054e6 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:21.647409 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87609fe8-4715-4a70-8c77-f890dc61006e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:21.648051 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c5ded-584d-4ead-9129-1b3bba4b8b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:21.649073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c5ded-584d-4ead-9129-1b3bba4b8b05 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:21.649625 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c5ded-584d-4ead-9129-1b3bba4b8b05 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:21.650076 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c5ded-584d-4ead-9129-1b3bba4b8b05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:21.668815 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:21.713521 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:21.731585 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c5ded-584d-4ead-9129-1b3bba4b8b05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:21.733446 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c5ded-584d-4ead-9129-1b3bba4b8b05 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:21.749105 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e5ded-d2f4-4ebe-b3fd-35d365da61f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:21.749808 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e5ded-d2f4-4ebe-b3fd-35d365da61f1 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:21.750094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e5ded-d2f4-4ebe-b3fd-35d365da61f1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:21.750370 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e5ded-d2f4-4ebe-b3fd-35d365da61f1 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:21.751142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e5ded-d2f4-4ebe-b3fd-35d365da61f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:21.751453 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e5ded-d2f4-4ebe-b3fd-35d365da61f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:21.752951 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e5ded-d2f4-4ebe-b3fd-35d365da61f1 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:21.778408 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 140/262] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:21.789524 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e5ded-d2f4-4ebe-b3fd-35d365da61f1 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:21.854643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 123/263] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:21.860170 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c5ded-584d-4ead-9129-1b3bba4b8b05 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:21.953612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:22.031236 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e5ded-d2f4-4ebe-b3fd-35d365da61f1 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:22.072256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c5ded-584d-4ead-9129-1b3bba4b8b05 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:22.115282 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef7f4f-32b5-4c32-b87d-3746aa121e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:22.116152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef7f4f-32b5-4c32-b87d-3746aa121e2e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:22.116654 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef7f4f-32b5-4c32-b87d-3746aa121e2e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:22.116839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef7f4f-32b5-4c32-b87d-3746aa121e2e None admin] PATH_INFO: `/v3/users/8cb66d6e449f4da48b01b8976f5e0e80` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:22.118133 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef7f4f-32b5-4c32-b87d-3746aa121e2e None admin] RBAC: Authorizing `identity:get_user(user_id=8cb66d6e449f4da48b01b8976f5e0e80)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:22.119657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef7f4f-32b5-4c32-b87d-3746aa121e2e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:22.121977 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 141/264] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:21 2022] GET /identity/v3/users/8cb66d6e449f4da48b01b8976f5e0e80 => generated 283 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:22.132362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef7f4f-32b5-4c32-b87d-3746aa121e2e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:22.136361 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b41ce-4439-4677-af1e-a61d6d5c540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:22.137642 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b41ce-4439-4677-af1e-a61d6d5c540d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:22.138172 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b41ce-4439-4677-af1e-a61d6d5c540d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:22.138784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b41ce-4439-4677-af1e-a61d6d5c540d None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:22.144368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b41ce-4439-4677-af1e-a61d6d5c540d None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:22.144368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b41ce-4439-4677-af1e-a61d6d5c540d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:22.145863 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 124/265] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:21 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:22.152383 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b41ce-4439-4677-af1e-a61d6d5c540d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:22.234036 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:22.268862 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b41ce-4439-4677-af1e-a61d6d5c540d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:22.284819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:22.319496 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef7f4f-32b5-4c32-b87d-3746aa121e2e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:22.321623 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a74683-4af8-4682-834d-ae8525180481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:22.322584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74683-4af8-4682-834d-ae8525180481 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:22.323202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74683-4af8-4682-834d-ae8525180481 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:22.323713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74683-4af8-4682-834d-ae8525180481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:22.324819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a74683-4af8-4682-834d-ae8525180481 None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=c9a4c742053041558d2d857f27eedf48, user.id=8cb66d6e449f4da48b01b8976f5e0e80) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:22.325370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a74683-4af8-4682-834d-ae8525180481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:22.327112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a74683-4af8-4682-834d-ae8525180481 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:22.341799 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 125/266] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:22 2022] GET /identity/v3/role_assignments?user.id=8cb66d6e449f4da48b01b8976f5e0e80&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:22.358954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc18b0d-eeb9-4614-acd2-da3451b99904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:22.359633 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18b0d-eeb9-4614-acd2-da3451b99904 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:22.359912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18b0d-eeb9-4614-acd2-da3451b99904 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:22.360186 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc18b0d-eeb9-4614-acd2-da3451b99904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:22.470371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc18b0d-eeb9-4614-acd2-da3451b99904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:22.471868 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc18b0d-eeb9-4614-acd2-da3451b99904 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:22.543754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:22.606379 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 142/267] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:28:22.612205 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a74683-4af8-4682-834d-ae8525180481 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:22.772068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a74683-4af8-4682-834d-ae8525180481 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:22.809884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cf303-539e-4a86-bc75-cc8fa4697eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:22.810778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cf303-539e-4a86-bc75-cc8fa4697eba None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:22.811061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cf303-539e-4a86-bc75-cc8fa4697eba None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:22.811333 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cf303-539e-4a86-bc75-cc8fa4697eba None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:22.812651 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cf303-539e-4a86-bc75-cc8fa4697eba None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:22.814127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cf303-539e-4a86-bc75-cc8fa4697eba None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:22.816253 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 126/268] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:22 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:22.822242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc18b0d-eeb9-4614-acd2-da3451b99904 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:22.941819 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc18b0d-eeb9-4614-acd2-da3451b99904 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:22.984240 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d2f93-2338-4333-ac74-9f905487927e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:22.984958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d2f93-2338-4333-ac74-9f905487927e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:22.985237 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d2f93-2338-4333-ac74-9f905487927e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:22.985508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d2f93-2338-4333-ac74-9f905487927e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:22.986364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d2f93-2338-4333-ac74-9f905487927e None admin] RBAC: Adding query filter params (role.id=ff499a8782db4cce8e162a849be212ee, scope.project.id=c9a4c742053041558d2d857f27eedf48, user.id=8cb66d6e449f4da48b01b8976f5e0e80) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:22.986686 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d2f93-2338-4333-ac74-9f905487927e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:22.988175 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d2f93-2338-4333-ac74-9f905487927e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:22.995628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 143/269] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:22 2022] GET /identity/v3/role_assignments?user.id=8cb66d6e449f4da48b01b8976f5e0e80&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=ff499a8782db4cce8e162a849be212ee => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:23.218907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210adb4-88d1-4cef-91c1-29528e3ef43c None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:23.219169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210adb4-88d1-4cef-91c1-29528e3ef43c None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:23.219461 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210adb4-88d1-4cef-91c1-29528e3ef43c None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:23.220964 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 127/270] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:23.228488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208af8b-03dc-41ac-b6c9-53c2379f6925 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:23.228777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208af8b-03dc-41ac-b6c9-53c2379f6925 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:23.229130 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208af8b-03dc-41ac-b6c9-53c2379f6925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:23.254617 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-f208af8b-03dc-41ac-b6c9-53c2379f6925 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:23.256136 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-f208af8b-03dc-41ac-b6c9-53c2379f6925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:23.277748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:23.347000 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 144/271] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:23.354261 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c109b9-a28b-474a-b027-5f5d297d6bad None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:23.355212 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c109b9-a28b-474a-b027-5f5d297d6bad None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:23.355708 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c109b9-a28b-474a-b027-5f5d297d6bad None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:23.357796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 128/272] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:23.364291 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040122e-9026-4d22-97df-73f9b4d88b10 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:23.364652 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040122e-9026-4d22-97df-73f9b4d88b10 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:23.364918 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040122e-9026-4d22-97df-73f9b4d88b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:23.374083 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c109b9-a28b-474a-b027-5f5d297d6bad None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:23.388397 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-2040122e-9026-4d22-97df-73f9b4d88b10 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:23.389698 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-2040122e-9026-4d22-97df-73f9b4d88b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:23.446226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:23.494774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 145/273] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:23.517770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c109b9-a28b-474a-b027-5f5d297d6bad None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:23.520018 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040122e-9026-4d22-97df-73f9b4d88b10 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:23.559252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930ae4a-fda3-4b93-a8d3-fc796909f514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:23.560066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930ae4a-fda3-4b93-a8d3-fc796909f514 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:23.560370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930ae4a-fda3-4b93-a8d3-fc796909f514 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:23.560653 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0930ae4a-fda3-4b93-a8d3-fc796909f514 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:23.561918 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930ae4a-fda3-4b93-a8d3-fc796909f514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:23.563700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0930ae4a-fda3-4b93-a8d3-fc796909f514 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:23.565350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 129/274] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:23.574823 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930ae4a-fda3-4b93-a8d3-fc796909f514 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:23.674069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:23.713314 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040122e-9026-4d22-97df-73f9b4d88b10 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:23.721443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0930ae4a-fda3-4b93-a8d3-fc796909f514 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:23.747929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:23.752481 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705032a-2041-41e4-9f7c-b2edab7dce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:23.753474 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705032a-2041-41e4-9f7c-b2edab7dce1f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:23.753749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705032a-2041-41e4-9f7c-b2edab7dce1f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:23.754022 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705032a-2041-41e4-9f7c-b2edab7dce1f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:23.768456 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705032a-2041-41e4-9f7c-b2edab7dce1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:23.774354 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d005c-9ac4-4e4d-ab9d-72c3e81bb0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:23.775319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d005c-9ac4-4e4d-ab9d-72c3e81bb0c3 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:23.775793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d005c-9ac4-4e4d-ab9d-72c3e81bb0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:23.776255 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d005c-9ac4-4e4d-ab9d-72c3e81bb0c3 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:23.777240 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d005c-9ac4-4e4d-ab9d-72c3e81bb0c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-1705032a-2041-41e4-9f7c-b2edab7dce1f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:23.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:23.778687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:23.779956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 146/275] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:23.780859 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba750bb-8b41-413f-bb09-3cbbe901840d None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:23.780859 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba750bb-8b41-413f-bb09-3cbbe901840d None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:23.781157 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba750bb-8b41-413f-bb09-3cbbe901840d None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:23.782540 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 147/276] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:23.783042 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d005c-9ac4-4e4d-ab9d-72c3e81bb0c3 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:23.785101 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba750bb-8b41-413f-bb09-3cbbe901840d None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:23.893253 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba750bb-8b41-413f-bb09-3cbbe901840d None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:23.906180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 130/277] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:28:23 2022] POST /identity/v3/users => generated 293 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:23.912931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64521-0f83-4ff1-a3db-3432f01a5142 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:23.912931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64521-0f83-4ff1-a3db-3432f01a5142 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:23.912931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64521-0f83-4ff1-a3db-3432f01a5142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:23.933832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-3ac64521-0f83-4ff1-a3db-3432f01a5142 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:23.935601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-3ac64521-0f83-4ff1-a3db-3432f01a5142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:23.958823 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2925d00-13d7-4683-9690-b6ed8bf71263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:23.960083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2925d00-13d7-4683-9690-b6ed8bf71263 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:23.960938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2925d00-13d7-4683-9690-b6ed8bf71263 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:23.961568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2925d00-13d7-4683-9690-b6ed8bf71263 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:23.963314 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2925d00-13d7-4683-9690-b6ed8bf71263 None admin] RBAC: Adding query filter params (name=reader) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:23.963993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2925d00-13d7-4683-9690-b6ed8bf71263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:23.966792 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2925d00-13d7-4683-9690-b6ed8bf71263 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:23.978339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:23.992395 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 148/278] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:23.995044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2925d00-13d7-4683-9690-b6ed8bf71263 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:24.035899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 131/279] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:24.062954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac64521-0f83-4ff1-a3db-3432f01a5142 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:24.138968 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2925d00-13d7-4683-9690-b6ed8bf71263 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:24.190855 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924bf8b-b5ff-4a42-bad5-31c6d7bc31cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:24.194382 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924bf8b-b5ff-4a42-bad5-31c6d7bc31cc None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:24.194382 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924bf8b-b5ff-4a42-bad5-31c6d7bc31cc None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:24.194382 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924bf8b-b5ff-4a42-bad5-31c6d7bc31cc None admin] PATH_INFO: `/v3/users/33e37a49b3f340ee8c92713a3b600de1` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:24.194874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924bf8b-b5ff-4a42-bad5-31c6d7bc31cc None admin] RBAC: Authorizing `identity:get_user(user_id=33e37a49b3f340ee8c92713a3b600de1)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:24.197303 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c924bf8b-b5ff-4a42-bad5-31c6d7bc31cc None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:24.201514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:24.201726 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 149/280] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:23 2022] GET /identity/v3/users/33e37a49b3f340ee8c92713a3b600de1 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:24.207119 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924bf8b-b5ff-4a42-bad5-31c6d7bc31cc None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:24.208869 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac64521-0f83-4ff1-a3db-3432f01a5142 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:24.256274 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d23aea-8105-4c29-aab9-05d32eeeb87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:24.256778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d23aea-8105-4c29-aab9-05d32eeeb87e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:24.258351 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d23aea-8105-4c29-aab9-05d32eeeb87e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:24.258631 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d23aea-8105-4c29-aab9-05d32eeeb87e None admin] PATH_INFO: `/v3/roles/member` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:24.275733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d23aea-8105-4c29-aab9-05d32eeeb87e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-60d23aea-8105-4c29-aab9-05d32eeeb87e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:24.286204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:24.287277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:24.288180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:24.292894 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 132/281] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:24.294104 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d23aea-8105-4c29-aab9-05d32eeeb87e None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:24.304632 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:24.361445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c924bf8b-b5ff-4a42-bad5-31c6d7bc31cc None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:24.409802 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09e3e2-2826-4581-9750-7da478b92b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:24.411412 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09e3e2-2826-4581-9750-7da478b92b60 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:24.411883 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09e3e2-2826-4581-9750-7da478b92b60 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:24.412368 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de09e3e2-2826-4581-9750-7da478b92b60 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:24.414599 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09e3e2-2826-4581-9750-7da478b92b60 None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:24.416729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de09e3e2-2826-4581-9750-7da478b92b60 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:24.419938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 150/282] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:24 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:24.428743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09e3e2-2826-4581-9750-7da478b92b60 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:24.462256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d23aea-8105-4c29-aab9-05d32eeeb87e None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:24.513425 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742d080-b439-4e40-a8a7-177c6bb9a409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:24.514385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742d080-b439-4e40-a8a7-177c6bb9a409 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:24.514744 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742d080-b439-4e40-a8a7-177c6bb9a409 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:24.515097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742d080-b439-4e40-a8a7-177c6bb9a409 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:24.515990 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3742d080-b439-4e40-a8a7-177c6bb9a409 None admin] RBAC: Adding query filter params (name=member) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:24.516388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3742d080-b439-4e40-a8a7-177c6bb9a409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:24.518198 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3742d080-b439-4e40-a8a7-177c6bb9a409 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:24.529490 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 133/283] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:24.535438 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742d080-b439-4e40-a8a7-177c6bb9a409 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:24.573620 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:24.607568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de09e3e2-2826-4581-9750-7da478b92b60 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:24.654838 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf7fd5-601d-4c36-bb2b-975e7b95e79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:24.656048 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf7fd5-601d-4c36-bb2b-975e7b95e79b None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:24.656360 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf7fd5-601d-4c36-bb2b-975e7b95e79b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:24.657216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf7fd5-601d-4c36-bb2b-975e7b95e79b None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48/users/33e37a49b3f340ee8c92713a3b600de1/roles/7b870ddcb3b04c6088d9d80f29ab8193` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:24.659686 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf7fd5-601d-4c36-bb2b-975e7b95e79b None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a4c742053041558d2d857f27eedf48, user_id=33e37a49b3f340ee8c92713a3b600de1, role_id=7b870ddcb3b04c6088d9d80f29ab8193)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:24.662480 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf7fd5-601d-4c36-bb2b-975e7b95e79b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:24.667062 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:24.681211 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3742d080-b439-4e40-a8a7-177c6bb9a409 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:24.731375 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47c6ce-a738-4729-9453-1721ab692af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:24.733309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47c6ce-a738-4729-9453-1721ab692af7 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:24.734098 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47c6ce-a738-4729-9453-1721ab692af7 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:24.734944 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47c6ce-a738-4729-9453-1721ab692af7 None admin] PATH_INFO: `/v3/users/2520db4539d44b2f8b56209f8643c64f` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:24.765415 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47c6ce-a738-4729-9453-1721ab692af7 None admin] RBAC: Authorizing `identity:get_user(user_id=2520db4539d44b2f8b56209f8643c64f)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:24.768915 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47c6ce-a738-4729-9453-1721ab692af7 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:24.776839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 134/284] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:24 2022] GET /identity/v3/users/2520db4539d44b2f8b56209f8643c64f => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:24.779895 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076be0a-d735-477a-926a-deeab5bde47c None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:24.797781 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076be0a-d735-477a-926a-deeab5bde47c None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:24.799144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076be0a-d735-477a-926a-deeab5bde47c None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:24.803512 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 151/285] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:28:24 2022] PUT /identity/v3/projects/c9a4c742053041558d2d857f27eedf48/users/33e37a49b3f340ee8c92713a3b600de1/roles/7b870ddcb3b04c6088d9d80f29ab8193 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:24.810072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf7fd5-601d-4c36-bb2b-975e7b95e79b None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:24.814443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 135/286] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:24 2022] GET /identity => generated 270 bytes in 37 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:24.840382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacf610-b5cb-4957-b4b6-6757814622f0 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:24.843766 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacf610-b5cb-4957-b4b6-6757814622f0 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:24.854556 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacf610-b5cb-4957-b4b6-6757814622f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:24.912829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-6cacf610-b5cb-4957-b4b6-6757814622f0 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:24.916709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-6cacf610-b5cb-4957-b4b6-6757814622f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:25.121878 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 136/287] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:25.147577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacf610-b5cb-4957-b4b6-6757814622f0 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:25.187468 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf7fd5-601d-4c36-bb2b-975e7b95e79b None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:25.207219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:25.270877 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9762e-c3ea-4816-9129-cb5c2e405947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:25.272562 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9762e-c3ea-4816-9129-cb5c2e405947 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:25.272948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9762e-c3ea-4816-9129-cb5c2e405947 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:25.275886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9762e-c3ea-4816-9129-cb5c2e405947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:25.276708 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9762e-c3ea-4816-9129-cb5c2e405947 None admin] RBAC: Adding query filter params (role.id=c8be6a897df546ca9b0e7b9541fb94d5, user.id=2520db4539d44b2f8b56209f8643c64f, scope.system=all) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:25.277029 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9762e-c3ea-4816-9129-cb5c2e405947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:25.278536 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9762e-c3ea-4816-9129-cb5c2e405947 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:25.303619 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 152/288] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:28:24 2022] GET /identity/v3/role_assignments?user.id=2520db4539d44b2f8b56209f8643c64f&scope.system=all&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 226 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:25.374378 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacf610-b5cb-4957-b4b6-6757814622f0 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:25.414009 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c9969-e4d0-4ecc-bba7-9576806511e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:25.415220 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c9969-e4d0-4ecc-bba7-9576806511e5 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:25.415728 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c9969-e4d0-4ecc-bba7-9576806511e5 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:25.416229 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c9969-e4d0-4ecc-bba7-9576806511e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:25.429861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c9969-e4d0-4ecc-bba7-9576806511e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-7d8c9969-e4d0-4ecc-bba7-9576806511e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:25.438439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:25.439551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:25.440491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:25.443594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 137/289] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:25.449733 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9762e-c3ea-4816-9129-cb5c2e405947 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:25.584913 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:25.646441 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9762e-c3ea-4816-9129-cb5c2e405947 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:25.708602 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b198a-58e1-4657-b022-2a9481f454ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:25.709681 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b198a-58e1-4657-b022-2a9481f454ad None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:25.710197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b198a-58e1-4657-b022-2a9481f454ad None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:25.710688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b198a-58e1-4657-b022-2a9481f454ad None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:25.711665 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b198a-58e1-4657-b022-2a9481f454ad None admin] RBAC: Adding query filter params (name=member) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:25.712187 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b198a-58e1-4657-b022-2a9481f454ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:25.713844 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069b198a-58e1-4657-b022-2a9481f454ad None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:25.729879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 153/290] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:25.735853 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c9969-e4d0-4ecc-bba7-9576806511e5 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:25.852747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:25.881524 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c9969-e4d0-4ecc-bba7-9576806511e5 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:25.926145 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a933265-f0b9-4e25-b215-db8b051fae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:25.927481 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a933265-f0b9-4e25-b215-db8b051fae18 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:25.927999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a933265-f0b9-4e25-b215-db8b051fae18 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:25.928521 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a933265-f0b9-4e25-b215-db8b051fae18 None admin] PATH_INFO: `/v3/users/8107fbaddf514ec09825c4b1886aedb1` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:25.933178 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a933265-f0b9-4e25-b215-db8b051fae18 None admin] RBAC: Authorizing `identity:get_user(user_id=8107fbaddf514ec09825c4b1886aedb1)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:25.935293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a933265-f0b9-4e25-b215-db8b051fae18 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:25.937308 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 138/291] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:25 2022] GET /identity/v3/users/8107fbaddf514ec09825c4b1886aedb1 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:25.945521 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b198a-58e1-4657-b022-2a9481f454ad None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:26.090984 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b198a-58e1-4657-b022-2a9481f454ad None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:26.115314 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:26.130796 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fdc39-f1a1-4bf8-a49d-3615c680f08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:26.131464 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdc39-f1a1-4bf8-a49d-3615c680f08a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:26.131753 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdc39-f1a1-4bf8-a49d-3615c680f08a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:26.132029 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fdc39-f1a1-4bf8-a49d-3615c680f08a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:26.229300 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fdc39-f1a1-4bf8-a49d-3615c680f08a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:26.230702 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fdc39-f1a1-4bf8-a49d-3615c680f08a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:26.369351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 154/292] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:26.375499 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a933265-f0b9-4e25-b215-db8b051fae18 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:26.526165 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a933265-f0b9-4e25-b215-db8b051fae18 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:26.582459 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49b9c2-c7b3-468c-8099-1396e2428e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:26.583907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49b9c2-c7b3-468c-8099-1396e2428e76 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:26.584197 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49b9c2-c7b3-468c-8099-1396e2428e76 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:26.584509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49b9c2-c7b3-468c-8099-1396e2428e76 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:26.585817 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49b9c2-c7b3-468c-8099-1396e2428e76 None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:26.587269 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49b9c2-c7b3-468c-8099-1396e2428e76 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:26.589043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 139/293] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:26 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:26.594967 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fdc39-f1a1-4bf8-a49d-3615c680f08a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:26.653922 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:26.717739 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fdc39-f1a1-4bf8-a49d-3615c680f08a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:26.756285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a12f6e-0564-4e24-8718-41ec10f80a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:26.756990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12f6e-0564-4e24-8718-41ec10f80a4d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:26.757280 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12f6e-0564-4e24-8718-41ec10f80a4d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:26.757563 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a12f6e-0564-4e24-8718-41ec10f80a4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:26.758413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a12f6e-0564-4e24-8718-41ec10f80a4d None admin] RBAC: Adding query filter params (role.id=c8be6a897df546ca9b0e7b9541fb94d5, scope.project.id=c9a4c742053041558d2d857f27eedf48, user.id=8107fbaddf514ec09825c4b1886aedb1) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:26.758732 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a12f6e-0564-4e24-8718-41ec10f80a4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:26.760203 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a12f6e-0564-4e24-8718-41ec10f80a4d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:26.767026 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 155/294] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:26 2022] GET /identity/v3/role_assignments?user.id=8107fbaddf514ec09825c4b1886aedb1&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:26.847924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cfd2a-71ff-4ce9-b7f7-74bff36d878d None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:26.848663 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cfd2a-71ff-4ce9-b7f7-74bff36d878d None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:26.848663 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cfd2a-71ff-4ce9-b7f7-74bff36d878d None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:26.850226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 140/295] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:26.855901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244bec4-a858-4c72-81e1-092af84d1f5b None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:26.856234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244bec4-a858-4c72-81e1-092af84d1f5b None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:26.856488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244bec4-a858-4c72-81e1-092af84d1f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:26.860491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822a305-18c1-41b2-b410-3d9aa20350e6 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:26.860800 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822a305-18c1-41b2-b410-3d9aa20350e6 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:26.861084 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822a305-18c1-41b2-b410-3d9aa20350e6 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:26.862918 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 141/296] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:28:26.869352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf3166-93a7-4bec-b737-a89b6e771943 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:26.869682 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf3166-93a7-4bec-b737-a89b6e771943 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:26.869900 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf3166-93a7-4bec-b737-a89b6e771943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:26.877628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-1244bec4-a858-4c72-81e1-092af84d1f5b None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:26.878862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-1244bec4-a858-4c72-81e1-092af84d1f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:26.888841 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:26.905561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-fddf3166-93a7-4bec-b737-a89b6e771943 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:26.906826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-fddf3166-93a7-4bec-b737-a89b6e771943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:27.014439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 156/297] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:27.039135 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244bec4-a858-4c72-81e1-092af84d1f5b None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:27.039676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 142/298] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:27.068742 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:27.069407 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf3166-93a7-4bec-b737-a89b6e771943 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:27.172302 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244bec4-a858-4c72-81e1-092af84d1f5b None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:27.206029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf3166-93a7-4bec-b737-a89b6e771943 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:27.216852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d2c16-a488-4ac1-9dab-5e9d4381910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:27.218532 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d2c16-a488-4ac1-9dab-5e9d4381910e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:27.218532 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d2c16-a488-4ac1-9dab-5e9d4381910e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:27.218532 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d2c16-a488-4ac1-9dab-5e9d4381910e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:27.233803 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d2c16-a488-4ac1-9dab-5e9d4381910e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-475d2c16-a488-4ac1-9dab-5e9d4381910e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:27.244493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:27.245537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:27.246445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 157/299] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:27.251554 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d2c16-a488-4ac1-9dab-5e9d4381910e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:27.264149 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cef13-24b6-4657-9e6a-988ef769f68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:27.265354 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cef13-24b6-4657-9e6a-988ef769f68d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:27.265847 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cef13-24b6-4657-9e6a-988ef769f68d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:27.266340 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cef13-24b6-4657-9e6a-988ef769f68d None admin] PATH_INFO: `/v3/domains/default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:27.267743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cef13-24b6-4657-9e6a-988ef769f68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:27.269537 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cef13-24b6-4657-9e6a-988ef769f68d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:27.273510 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 143/300] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:27.280211 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cef13-24b6-4657-9e6a-988ef769f68d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:27.285498 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:27.362287 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:27.394305 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d2c16-a488-4ac1-9dab-5e9d4381910e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:27.401963 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cef13-24b6-4657-9e6a-988ef769f68d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:27.436452 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713376-1349-474d-904e-faa005627224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:27.437580 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713376-1349-474d-904e-faa005627224 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:27.438062 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713376-1349-474d-904e-faa005627224 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:27.438580 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11713376-1349-474d-904e-faa005627224 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:27.439819 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713376-1349-474d-904e-faa005627224 None admin] RBAC: Adding query filter params (name=reader) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:27.440328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713376-1349-474d-904e-faa005627224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:27.440457 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec7246-aa95-420c-b40d-65e2df59c298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:27.441303 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec7246-aa95-420c-b40d-65e2df59c298 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:27.441656 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec7246-aa95-420c-b40d-65e2df59c298 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:27.442025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec7246-aa95-420c-b40d-65e2df59c298 None admin] PATH_INFO: `/v3/groups` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:27.442720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11713376-1349-474d-904e-faa005627224 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:27.442848 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec7246-aa95-420c-b40d-65e2df59c298 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:27.453225 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 158/301] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:27.454778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec7246-aa95-420c-b40d-65e2df59c298 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:27.458256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713376-1349-474d-904e-faa005627224 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:27.536474 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 144/302] 10.222.0.88 () {60 vars in 1134 bytes} [Thu May 19 20:28:27 2022] POST /identity/v3/groups => generated 229 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:27.586940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11713376-1349-474d-904e-faa005627224 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:27.631485 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2071d48-ba03-4b5d-91ec-c9cce36252a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:27.632438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2071d48-ba03-4b5d-91ec-c9cce36252a0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:27.632725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2071d48-ba03-4b5d-91ec-c9cce36252a0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:27.633004 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2071d48-ba03-4b5d-91ec-c9cce36252a0 None admin] PATH_INFO: `/v3/users/e1f5282a6792425ea98f9ea8fd96e1a9` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:27.643616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:27.652514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2071d48-ba03-4b5d-91ec-c9cce36252a0 None admin] RBAC: Authorizing `identity:get_user(user_id=e1f5282a6792425ea98f9ea8fd96e1a9)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:27.659546 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2071d48-ba03-4b5d-91ec-c9cce36252a0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:27.659546 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 159/303] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:27 2022] GET /identity/v3/users/e1f5282a6792425ea98f9ea8fd96e1a9 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:27.663071 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec7246-aa95-420c-b40d-65e2df59c298 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:27.745847 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:27.791814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec7246-aa95-420c-b40d-65e2df59c298 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:27.829460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7a4d1-22a6-481e-80f1-c8d008806d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:27.830593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7a4d1-22a6-481e-80f1-c8d008806d15 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:27.831275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7a4d1-22a6-481e-80f1-c8d008806d15 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:27.832267 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7a4d1-22a6-481e-80f1-c8d008806d15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:27.833726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7a4d1-22a6-481e-80f1-c8d008806d15 None admin] RBAC: Adding query filter params (role.id=7b870ddcb3b04c6088d9d80f29ab8193, user.id=e1f5282a6792425ea98f9ea8fd96e1a9, scope.system=all) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:27.834055 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7a4d1-22a6-481e-80f1-c8d008806d15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:27.835561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7a4d1-22a6-481e-80f1-c8d008806d15 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:27.842299 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 145/304] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:28:27 2022] GET /identity/v3/role_assignments?user.id=e1f5282a6792425ea98f9ea8fd96e1a9&scope.system=all&role.id=7b870ddcb3b04c6088d9d80f29ab8193 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:28.055803 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b54311-4e2f-4950-9501-3497fc84f587 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:28.056793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b54311-4e2f-4950-9501-3497fc84f587 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:28.057499 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b54311-4e2f-4950-9501-3497fc84f587 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:28.059860 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 160/305] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:28.066780 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e18d7-4040-465b-88a2-a2dbb7389f03 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:28.067802 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e18d7-4040-465b-88a2-a2dbb7389f03 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:28.068302 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e18d7-4040-465b-88a2-a2dbb7389f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:28.086841 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-391e18d7-4040-465b-88a2-a2dbb7389f03 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:28.088520 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-391e18d7-4040-465b-88a2-a2dbb7389f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:28.168370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 146/306] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:28.193582 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b54311-4e2f-4950-9501-3497fc84f587 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:28.230463 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d53f6-8463-415a-a72d-e6a17dc1531f None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:28.231120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d53f6-8463-415a-a72d-e6a17dc1531f None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:28.231627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d53f6-8463-415a-a72d-e6a17dc1531f None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:28.233585 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 147/307] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:28.239506 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b8eeb-b147-4b77-8a0a-c5b1216ef2db None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:28.240076 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b8eeb-b147-4b77-8a0a-c5b1216ef2db None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:28.240387 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b8eeb-b147-4b77-8a0a-c5b1216ef2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:28.268512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-b38b8eeb-b147-4b77-8a0a-c5b1216ef2db None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:28.270132 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-b38b8eeb-b147-4b77-8a0a-c5b1216ef2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:28.335527 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:28.338785 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b54311-4e2f-4950-9501-3497fc84f587 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:28.356857 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 148/308] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:28.375089 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b8eeb-b147-4b77-8a0a-c5b1216ef2db None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:28.381339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49390cd4-5968-4af1-8102-473d13deadf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:28.382252 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49390cd4-5968-4af1-8102-473d13deadf7 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:28.382543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49390cd4-5968-4af1-8102-473d13deadf7 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:28.382823 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49390cd4-5968-4af1-8102-473d13deadf7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:28.395586 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49390cd4-5968-4af1-8102-473d13deadf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-49390cd4-5968-4af1-8102-473d13deadf7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:28.404248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:28.405364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:28.406440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 161/309] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:28.412308 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49390cd4-5968-4af1-8102-473d13deadf7 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:28.497356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:28.505558 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b8eeb-b147-4b77-8a0a-c5b1216ef2db None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:28.541320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49390cd4-5968-4af1-8102-473d13deadf7 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:28.543981 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb8d6a-29d4-43d0-b486-ca817ba5300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:28.545067 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb8d6a-29d4-43d0-b486-ca817ba5300f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:28.545573 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb8d6a-29d4-43d0-b486-ca817ba5300f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:28.546041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb8d6a-29d4-43d0-b486-ca817ba5300f None admin] PATH_INFO: `/v3/roles/member` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:28.558197 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb8d6a-29d4-43d0-b486-ca817ba5300f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-24cb8d6a-29d4-43d0-b486-ca817ba5300f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:28.566676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:28.568594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:28.569612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:28.574695 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 149/310] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:28.576005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb8d6a-29d4-43d0-b486-ca817ba5300f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:28.578846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc38d5-b1df-4ec0-91d6-a7b51e2ebf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:28.579648 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc38d5-b1df-4ec0-91d6-a7b51e2ebf82 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:28.579939 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc38d5-b1df-4ec0-91d6-a7b51e2ebf82 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:28.580274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc38d5-b1df-4ec0-91d6-a7b51e2ebf82 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:28.581940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc38d5-b1df-4ec0-91d6-a7b51e2ebf82 None admin] RBAC: Adding query filter params (name=reader) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:28.581940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc38d5-b1df-4ec0-91d6-a7b51e2ebf82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:28.583553 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc38d5-b1df-4ec0-91d6-a7b51e2ebf82 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:28.601823 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 162/311] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:28.601823 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc38d5-b1df-4ec0-91d6-a7b51e2ebf82 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:28.633597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:28.690723 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb8d6a-29d4-43d0-b486-ca817ba5300f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:28.719677 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc38d5-b1df-4ec0-91d6-a7b51e2ebf82 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:28.729058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8741a-a0ed-4454-8442-cfd01cf28f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:28.730088 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8741a-a0ed-4454-8442-cfd01cf28f37 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:28.730579 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8741a-a0ed-4454-8442-cfd01cf28f37 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:28.731074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b8741a-a0ed-4454-8442-cfd01cf28f37 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:28.732073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8741a-a0ed-4454-8442-cfd01cf28f37 None admin] RBAC: Adding query filter params (name=member) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:28.732637 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8741a-a0ed-4454-8442-cfd01cf28f37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:28.734247 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b8741a-a0ed-4454-8442-cfd01cf28f37 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:28.742212 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:28.744151 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 150/312] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:28.750138 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8741a-a0ed-4454-8442-cfd01cf28f37 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:28.762704 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a518df-971b-43ea-9328-55f6adf8d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:28.763990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a518df-971b-43ea-9328-55f6adf8d1dd None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:28.763990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a518df-971b-43ea-9328-55f6adf8d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:28.764248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a518df-971b-43ea-9328-55f6adf8d1dd None admin] PATH_INFO: `/v3/users/33e37a49b3f340ee8c92713a3b600de1` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:28.765749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a518df-971b-43ea-9328-55f6adf8d1dd None admin] RBAC: Authorizing `identity:get_user(user_id=33e37a49b3f340ee8c92713a3b600de1)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:28.767497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a518df-971b-43ea-9328-55f6adf8d1dd None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:28.773055 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 163/313] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:28 2022] GET /identity/v3/users/33e37a49b3f340ee8c92713a3b600de1 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:28.776623 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a518df-971b-43ea-9328-55f6adf8d1dd None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:28.874663 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b8741a-a0ed-4454-8442-cfd01cf28f37 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:28.887600 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:28.911751 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a518df-971b-43ea-9328-55f6adf8d1dd None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:28.916081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b3e24-395c-4b1b-a2d6-6714efdad921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:28.917579 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b3e24-395c-4b1b-a2d6-6714efdad921 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:28.918070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b3e24-395c-4b1b-a2d6-6714efdad921 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:28.918389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b3e24-395c-4b1b-a2d6-6714efdad921 None admin] PATH_INFO: `/v3/users/2520db4539d44b2f8b56209f8643c64f` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:28.919791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b3e24-395c-4b1b-a2d6-6714efdad921 None admin] RBAC: Authorizing `identity:get_user(user_id=2520db4539d44b2f8b56209f8643c64f)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:28.921980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b3e24-395c-4b1b-a2d6-6714efdad921 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:28.924964 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 151/314] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:28 2022] GET /identity/v3/users/2520db4539d44b2f8b56209f8643c64f => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:28.933470 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b3e24-395c-4b1b-a2d6-6714efdad921 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:28.956155 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31497fa-8e85-4481-ab82-39eaaaf54fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:28.956873 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31497fa-8e85-4481-ab82-39eaaaf54fc3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:28.957155 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31497fa-8e85-4481-ab82-39eaaaf54fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:28.957437 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31497fa-8e85-4481-ab82-39eaaaf54fc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:29.019393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:29.038936 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31497fa-8e85-4481-ab82-39eaaaf54fc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:29.041469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31497fa-8e85-4481-ab82-39eaaaf54fc3 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:29.073357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b3e24-395c-4b1b-a2d6-6714efdad921 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:29.124300 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7f6d1-7269-4d03-8cc9-fa4fb6099fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:29.125251 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7f6d1-7269-4d03-8cc9-fa4fb6099fa5 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:29.125469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7f6d1-7269-4d03-8cc9-fa4fb6099fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:29.125747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7f6d1-7269-4d03-8cc9-fa4fb6099fa5 None admin] PATH_INFO: `/v3/system/users/2520db4539d44b2f8b56209f8643c64f/roles/c8be6a897df546ca9b0e7b9541fb94d5` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:29.127285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7f6d1-7269-4d03-8cc9-fa4fb6099fa5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2520db4539d44b2f8b56209f8643c64f, role_id=c8be6a897df546ca9b0e7b9541fb94d5)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:29.128722 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7f6d1-7269-4d03-8cc9-fa4fb6099fa5 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:29.185132 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 164/315] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:29.191333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31497fa-8e85-4481-ab82-39eaaaf54fc3 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:29.214943 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 152/316] 10.222.0.88 () {58 vars in 1334 bytes} [Thu May 19 20:28:28 2022] PUT /identity/v3/system/users/2520db4539d44b2f8b56209f8643c64f/roles/c8be6a897df546ca9b0e7b9541fb94d5 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:29.309734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:29.327685 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31497fa-8e85-4481-ab82-39eaaaf54fc3 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:29.369339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a7c5ea-3749-48c0-b841-a11d4ffbd6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:29.370237 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7c5ea-3749-48c0-b841-a11d4ffbd6e8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:29.370522 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7c5ea-3749-48c0-b841-a11d4ffbd6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:29.370794 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a7c5ea-3749-48c0-b841-a11d4ffbd6e8 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:29.372144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7c5ea-3749-48c0-b841-a11d4ffbd6e8 None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:29.373671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a7c5ea-3749-48c0-b841-a11d4ffbd6e8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:29.375923 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 165/317] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:29 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:29.382323 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7f6d1-7269-4d03-8cc9-fa4fb6099fa5 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:29.455414 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:29.519813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7f6d1-7269-4d03-8cc9-fa4fb6099fa5 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:29.559621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d6b58-0c4f-45a7-b14d-b4d33ba4ec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:29.560708 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d6b58-0c4f-45a7-b14d-b4d33ba4ec47 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:29.561231 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d6b58-0c4f-45a7-b14d-b4d33ba4ec47 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:29.561698 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d6b58-0c4f-45a7-b14d-b4d33ba4ec47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:29.562740 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d6b58-0c4f-45a7-b14d-b4d33ba4ec47 None admin] RBAC: Adding query filter params (role.id=7b870ddcb3b04c6088d9d80f29ab8193, scope.project.id=c9a4c742053041558d2d857f27eedf48, user.id=33e37a49b3f340ee8c92713a3b600de1) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:29.563232 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d6b58-0c4f-45a7-b14d-b4d33ba4ec47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:29.564942 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d6b58-0c4f-45a7-b14d-b4d33ba4ec47 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:29.573254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 153/318] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:29 2022] GET /identity/v3/role_assignments?user.id=33e37a49b3f340ee8c92713a3b600de1&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=7b870ddcb3b04c6088d9d80f29ab8193 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:29.989983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dff8ca-aa4c-4a3b-8712-06ae6c5d2e10 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:29.990580 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dff8ca-aa4c-4a3b-8712-06ae6c5d2e10 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:29.990892 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dff8ca-aa4c-4a3b-8712-06ae6c5d2e10 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:29.993252 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 166/319] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:28:29.999311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514524bb-90f6-4076-8d1e-914227deaadb None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:29.999635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514524bb-90f6-4076-8d1e-914227deaadb None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:29.999880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514524bb-90f6-4076-8d1e-914227deaadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.018104 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-514524bb-90f6-4076-8d1e-914227deaadb None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:30.019512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-514524bb-90f6-4076-8d1e-914227deaadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:30.102431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 154/320] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:30.129580 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dff8ca-aa4c-4a3b-8712-06ae6c5d2e10 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:30.242375 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dff8ca-aa4c-4a3b-8712-06ae6c5d2e10 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:30.279927 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b2734-34c8-4ad6-9e50-eaee3bffc844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:30.280765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b2734-34c8-4ad6-9e50-eaee3bffc844 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:30.281050 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b2734-34c8-4ad6-9e50-eaee3bffc844 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:30.281326 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b2734-34c8-4ad6-9e50-eaee3bffc844 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.282525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b2734-34c8-4ad6-9e50-eaee3bffc844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:30.284017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b2734-34c8-4ad6-9e50-eaee3bffc844 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:30.286477 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 167/321] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:30.292013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514524bb-90f6-4076-8d1e-914227deaadb None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:30.297147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8d2e6-eae8-462a-a120-59aa8308a02f None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:30.297439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8d2e6-eae8-462a-a120-59aa8308a02f None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:30.297714 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8d2e6-eae8-462a-a120-59aa8308a02f None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.299248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 168/322] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:28:30.304742 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda88bd-3b18-4805-afa1-ecc3e58226da None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:30.304921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda88bd-3b18-4805-afa1-ecc3e58226da None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:30.305097 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda88bd-3b18-4805-afa1-ecc3e58226da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.323050 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-beda88bd-3b18-4805-afa1-ecc3e58226da None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:30.324632 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-beda88bd-3b18-4805-afa1-ecc3e58226da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:30.334888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:30.344301 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:30.405988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 169/323] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:30.415846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514524bb-90f6-4076-8d1e-914227deaadb None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:30.428588 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda88bd-3b18-4805-afa1-ecc3e58226da None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:30.459818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4356084-d7ed-49cf-bb47-4699087cc312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:30.460377 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4356084-d7ed-49cf-bb47-4699087cc312 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:30.460667 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4356084-d7ed-49cf-bb47-4699087cc312 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:30.460943 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4356084-d7ed-49cf-bb47-4699087cc312 None admin] PATH_INFO: `/v3/groups` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.461649 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4356084-d7ed-49cf-bb47-4699087cc312 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:30.463099 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4356084-d7ed-49cf-bb47-4699087cc312 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:30.550568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda88bd-3b18-4805-afa1-ecc3e58226da None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:30.595494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 155/324] 10.222.0.88 () {60 vars in 1134 bytes} [Thu May 19 20:28:30 2022] POST /identity/v3/groups => generated 226 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:28:30.599672 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb08a3-e3f1-4bb3-8f5c-5f36bfa10898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:30.600630 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb08a3-e3f1-4bb3-8f5c-5f36bfa10898 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:30.601055 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb08a3-e3f1-4bb3-8f5c-5f36bfa10898 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:30.601376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb08a3-e3f1-4bb3-8f5c-5f36bfa10898 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.615048 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb08a3-e3f1-4bb3-8f5c-5f36bfa10898 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-86fb08a3-e3f1-4bb3-8f5c-5f36bfa10898 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:30.623413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:30.631339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:30.639330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 170/325] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:30.640293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4356084-d7ed-49cf-bb47-4699087cc312 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:30.757317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:30.765028 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4356084-d7ed-49cf-bb47-4699087cc312 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:30.816183 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc5cdb-c9fc-447a-9b0b-63b04883fe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:30.817709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc5cdb-c9fc-447a-9b0b-63b04883fe4c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:30.818154 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc5cdb-c9fc-447a-9b0b-63b04883fe4c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:30.818579 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc5cdb-c9fc-447a-9b0b-63b04883fe4c None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:30.820393 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc5cdb-c9fc-447a-9b0b-63b04883fe4c None admin] RBAC: Adding query filter params (name=reader) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:30.820393 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc5cdb-c9fc-447a-9b0b-63b04883fe4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:30.822682 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cc5cdb-c9fc-447a-9b0b-63b04883fe4c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:30.835547 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 156/326] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:30.841939 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb08a3-e3f1-4bb3-8f5c-5f36bfa10898 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:30.862999 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:31.006238 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb08a3-e3f1-4bb3-8f5c-5f36bfa10898 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:31.058575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0be98-1d8e-44e0-8540-2a475bb0cf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:31.059802 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0be98-1d8e-44e0-8540-2a475bb0cf87 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:31.060279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0be98-1d8e-44e0-8540-2a475bb0cf87 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:31.060782 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0be98-1d8e-44e0-8540-2a475bb0cf87 None admin] PATH_INFO: `/v3/users/e1f5282a6792425ea98f9ea8fd96e1a9` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:31.062232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0be98-1d8e-44e0-8540-2a475bb0cf87 None admin] RBAC: Authorizing `identity:get_user(user_id=e1f5282a6792425ea98f9ea8fd96e1a9)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:31.063906 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0be98-1d8e-44e0-8540-2a475bb0cf87 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:31.066110 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 171/327] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:30 2022] GET /identity/v3/users/e1f5282a6792425ea98f9ea8fd96e1a9 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:31.072095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc5cdb-c9fc-447a-9b0b-63b04883fe4c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:31.190834 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc5cdb-c9fc-447a-9b0b-63b04883fe4c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:31.231615 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64183fd7-139f-417f-afb5-e14815c0ed2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:31.232972 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183fd7-139f-417f-afb5-e14815c0ed2b None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:31.233475 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183fd7-139f-417f-afb5-e14815c0ed2b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:31.233954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183fd7-139f-417f-afb5-e14815c0ed2b None admin] PATH_INFO: `/v3/system/users/e1f5282a6792425ea98f9ea8fd96e1a9/roles/7b870ddcb3b04c6088d9d80f29ab8193` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:31.236316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64183fd7-139f-417f-afb5-e14815c0ed2b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e1f5282a6792425ea98f9ea8fd96e1a9, role_id=7b870ddcb3b04c6088d9d80f29ab8193)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:31.237944 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64183fd7-139f-417f-afb5-e14815c0ed2b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:31.307158 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 157/328] 10.222.0.88 () {58 vars in 1334 bytes} [Thu May 19 20:28:31 2022] PUT /identity/v3/system/users/e1f5282a6792425ea98f9ea8fd96e1a9/roles/7b870ddcb3b04c6088d9d80f29ab8193 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:28:31.723002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2bad8-0a29-4866-9352-cca819013ef9 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:31.723286 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2bad8-0a29-4866-9352-cca819013ef9 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:31.723582 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2bad8-0a29-4866-9352-cca819013ef9 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:31.725197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 172/329] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:31.735089 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba5e53-37e1-4872-bb58-034a7aaf5edf None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:31.735826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba5e53-37e1-4872-bb58-034a7aaf5edf None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:31.736315 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba5e53-37e1-4872-bb58-034a7aaf5edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:31.755401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-2dba5e53-37e1-4872-bb58-034a7aaf5edf None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:31.757165 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-2dba5e53-37e1-4872-bb58-034a7aaf5edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:31.789651 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:31.852603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 158/330] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:31.877579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2bad8-0a29-4866-9352-cca819013ef9 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:31.977200 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:32.002097 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2bad8-0a29-4866-9352-cca819013ef9 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:32.043701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982f16f-89e9-4927-8954-01344125c3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:32.044568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f16f-89e9-4927-8954-01344125c3f7 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:32.044853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f16f-89e9-4927-8954-01344125c3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:32.045203 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f16f-89e9-4927-8954-01344125c3f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:32.065025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982f16f-89e9-4927-8954-01344125c3f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-a982f16f-89e9-4927-8954-01344125c3f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:32.073351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:32.074411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:32.075734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 173/331] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:32.082762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba5e53-37e1-4872-bb58-034a7aaf5edf None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:32.231505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba5e53-37e1-4872-bb58-034a7aaf5edf None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:32.273381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27146403-9858-4f0f-86b4-42a002fa5d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:32.274447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146403-9858-4f0f-86b4-42a002fa5d6a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:32.274940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146403-9858-4f0f-86b4-42a002fa5d6a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:32.275449 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27146403-9858-4f0f-86b4-42a002fa5d6a None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:32.276483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27146403-9858-4f0f-86b4-42a002fa5d6a None admin] RBAC: Adding query filter params (name=member) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:32.277002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27146403-9858-4f0f-86b4-42a002fa5d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:32.278679 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27146403-9858-4f0f-86b4-42a002fa5d6a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:32.293819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 159/332] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:32.299797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982f16f-89e9-4927-8954-01344125c3f7 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:32.484807 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982f16f-89e9-4927-8954-01344125c3f7 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:32.529572 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15776a22-b2a7-4df5-b9d5-71f1214db72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:32.530845 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15776a22-b2a7-4df5-b9d5-71f1214db72c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:32.531339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15776a22-b2a7-4df5-b9d5-71f1214db72c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:32.531841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15776a22-b2a7-4df5-b9d5-71f1214db72c None admin] PATH_INFO: `/v3/users/2520db4539d44b2f8b56209f8643c64f` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:32.536630 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15776a22-b2a7-4df5-b9d5-71f1214db72c None admin] RBAC: Authorizing `identity:get_user(user_id=2520db4539d44b2f8b56209f8643c64f)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:32.538395 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15776a22-b2a7-4df5-b9d5-71f1214db72c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:32.541180 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 174/333] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:32 2022] GET /identity/v3/users/2520db4539d44b2f8b56209f8643c64f => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:32.547848 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27146403-9858-4f0f-86b4-42a002fa5d6a None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:32.567567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:32.666397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27146403-9858-4f0f-86b4-42a002fa5d6a None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:32.710075 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855016e-a99c-4694-aeba-82b408b7c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:32.710764 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855016e-a99c-4694-aeba-82b408b7c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:32.711051 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855016e-a99c-4694-aeba-82b408b7c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:32.711330 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855016e-a99c-4694-aeba-82b408b7c0c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:32.712201 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855016e-a99c-4694-aeba-82b408b7c0c1 None admin] RBAC: Adding query filter params (role.id=c8be6a897df546ca9b0e7b9541fb94d5, user.id=2520db4539d44b2f8b56209f8643c64f, scope.system=all) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:32.712555 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855016e-a99c-4694-aeba-82b408b7c0c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:32.714009 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855016e-a99c-4694-aeba-82b408b7c0c1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:32.731408 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 160/334] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:28:32 2022] GET /identity/v3/role_assignments?user.id=2520db4539d44b2f8b56209f8643c64f&scope.system=all&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:33.987517 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4d8f-a32b-462f-8a20-78d73697b2dc None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:33.988080 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4d8f-a32b-462f-8a20-78d73697b2dc None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:33.988080 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff4d8f-a32b-462f-8a20-78d73697b2dc None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:33.990131 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 175/335] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:33.995465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d391a-8fd7-43fa-8a4f-9e8f22e2251d None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:33.995966 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d391a-8fd7-43fa-8a4f-9e8f22e2251d None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:33.996050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d391a-8fd7-43fa-8a4f-9e8f22e2251d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:34.022652 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-198d391a-8fd7-43fa-8a4f-9e8f22e2251d None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:34.023863 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-198d391a-8fd7-43fa-8a4f-9e8f22e2251d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:34.120479 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 161/336] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:34.158176 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff4d8f-a32b-462f-8a20-78d73697b2dc None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:34.187298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:34.327378 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff4d8f-a32b-462f-8a20-78d73697b2dc None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:34.374859 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50496faf-3df6-4792-ae02-7e1b4576e843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:34.376089 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50496faf-3df6-4792-ae02-7e1b4576e843 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:34.376629 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50496faf-3df6-4792-ae02-7e1b4576e843 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:34.377130 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50496faf-3df6-4792-ae02-7e1b4576e843 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:34.391097 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50496faf-3df6-4792-ae02-7e1b4576e843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-50496faf-3df6-4792-ae02-7e1b4576e843 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:34.400568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:34.401655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:34.402639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:34.405654 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 176/337] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:34.411326 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d391a-8fd7-43fa-8a4f-9e8f22e2251d None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:34.414199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:34.496393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc688db-e29b-4d84-9a9e-1357c11dc286 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:34.501437 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc688db-e29b-4d84-9a9e-1357c11dc286 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:34.502446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc688db-e29b-4d84-9a9e-1357c11dc286 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:34.505051 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 177/338] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:34.510900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73169105-f76e-4ff1-8b28-7a917f57d26f None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:34.511519 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73169105-f76e-4ff1-8b28-7a917f57d26f None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:34.512003 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73169105-f76e-4ff1-8b28-7a917f57d26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:34.545742 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-73169105-f76e-4ff1-8b28-7a917f57d26f None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:34.547346 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-73169105-f76e-4ff1-8b28-7a917f57d26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:34.589086 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d391a-8fd7-43fa-8a4f-9e8f22e2251d None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:34.630229 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056789ad-3a9d-4043-bcec-50e8b0fedbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:34.631272 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056789ad-3a9d-4043-bcec-50e8b0fedbfd None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:34.631750 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056789ad-3a9d-4043-bcec-50e8b0fedbfd None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:34.632206 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056789ad-3a9d-4043-bcec-50e8b0fedbfd None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:34.633170 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056789ad-3a9d-4043-bcec-50e8b0fedbfd None admin] RBAC: Adding query filter params (name=member) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:34.633662 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056789ad-3a9d-4043-bcec-50e8b0fedbfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:34.635572 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056789ad-3a9d-4043-bcec-50e8b0fedbfd None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:34.647032 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 162/339] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:34.648188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:34.652365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056789ad-3a9d-4043-bcec-50e8b0fedbfd None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:34.658058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 178/340] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:34.700116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73169105-f76e-4ff1-8b28-7a917f57d26f None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:34.728810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:34.789767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056789ad-3a9d-4043-bcec-50e8b0fedbfd None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:34.846534 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b28447-5ebe-4665-a36c-cf1c20983f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:34.847131 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b28447-5ebe-4665-a36c-cf1c20983f50 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:34.847419 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b28447-5ebe-4665-a36c-cf1c20983f50 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:34.847691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b28447-5ebe-4665-a36c-cf1c20983f50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:34.943420 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b28447-5ebe-4665-a36c-cf1c20983f50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:34.944922 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b28447-5ebe-4665-a36c-cf1c20983f50 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:34.962642 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73169105-f76e-4ff1-8b28-7a917f57d26f None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:35.021488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:35.071207 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 163/341] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:35.073333 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07fea-ae03-491c-ac63-9ccb619f738c None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.074231 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07fea-ae03-491c-ac63-9ccb619f738c None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.074983 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07fea-ae03-491c-ac63-9ccb619f738c None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.077048 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 164/342] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:35.077847 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87152e2-b5e6-404c-99cb-4733948e61cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:35.079200 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87152e2-b5e6-404c-99cb-4733948e61cc None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.079628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87152e2-b5e6-404c-99cb-4733948e61cc None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.080057 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87152e2-b5e6-404c-99cb-4733948e61cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.081706 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d07fea-ae03-491c-ac63-9ccb619f738c None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:35.107354 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87152e2-b5e6-404c-99cb-4733948e61cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-a87152e2-b5e6-404c-99cb-4733948e61cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:35.119755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:35.121182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:35.122328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:35.131843 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 179/343] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:35.135607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110695b-9fc6-422f-b243-002c2fa785b2 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.136078 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110695b-9fc6-422f-b243-002c2fa785b2 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.138121 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110695b-9fc6-422f-b243-002c2fa785b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.194820 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-4110695b-9fc6-422f-b243-002c2fa785b2 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:35.196029 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-4110695b-9fc6-422f-b243-002c2fa785b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:35.259907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d07fea-ae03-491c-ac63-9ccb619f738c None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:35.309207 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de831c-bd26-4c01-813b-0adc7a0e16b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:35.310527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:35.311393 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de831c-bd26-4c01-813b-0adc7a0e16b8 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.311839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de831c-bd26-4c01-813b-0adc7a0e16b8 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.312269 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de831c-bd26-4c01-813b-0adc7a0e16b8 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.313781 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de831c-bd26-4c01-813b-0adc7a0e16b8 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:35.315451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de831c-bd26-4c01-813b-0adc7a0e16b8 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:35.318006 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 165/344] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:35 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:35.318965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de831c-bd26-4c01-813b-0adc7a0e16b8 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:35.338231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 180/345] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:35.339534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110695b-9fc6-422f-b243-002c2fa785b2 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:35.382963 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:35.436445 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de831c-bd26-4c01-813b-0adc7a0e16b8 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:35.479009 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4a7b8-c2bf-46a5-bfed-28b6eb955e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:35.480023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4a7b8-c2bf-46a5-bfed-28b6eb955e58 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.480683 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4a7b8-c2bf-46a5-bfed-28b6eb955e58 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.481150 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4a7b8-c2bf-46a5-bfed-28b6eb955e58 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.482079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4a7b8-c2bf-46a5-bfed-28b6eb955e58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:35.482551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4a7b8-c2bf-46a5-bfed-28b6eb955e58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:35.484129 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4a7b8-c2bf-46a5-bfed-28b6eb955e58 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:35.497001 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 166/346] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:35.498132 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4a7b8-c2bf-46a5-bfed-28b6eb955e58 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:35.620719 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:35.624436 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110695b-9fc6-422f-b243-002c2fa785b2 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:35.624983 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4a7b8-c2bf-46a5-bfed-28b6eb955e58 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:35.673239 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244a063-7ad6-4424-9d11-24fa4e58b537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:35.673982 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244a063-7ad6-4424-9d11-24fa4e58b537 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.674268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244a063-7ad6-4424-9d11-24fa4e58b537 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.674543 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244a063-7ad6-4424-9d11-24fa4e58b537 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.682679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae07a5d-5ef2-4e85-975c-c43c980c2316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:35.684239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae07a5d-5ef2-4e85-975c-c43c980c2316 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.684765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae07a5d-5ef2-4e85-975c-c43c980c2316 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.685235 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae07a5d-5ef2-4e85-975c-c43c980c2316 None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.694950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae07a5d-5ef2-4e85-975c-c43c980c2316 None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:35.696936 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae07a5d-5ef2-4e85-975c-c43c980c2316 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:35.699579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 181/347] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:35 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:35.700661 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae07a5d-5ef2-4e85-975c-c43c980c2316 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:35.709019 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244a063-7ad6-4424-9d11-24fa4e58b537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-7244a063-7ad6-4424-9d11-24fa4e58b537 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:35.716899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:35.717978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:35.718887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 167/348] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:35.719816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244a063-7ad6-4424-9d11-24fa4e58b537 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:35.784937 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:35.851081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae07a5d-5ef2-4e85-975c-c43c980c2316 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:35.874510 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244a063-7ad6-4424-9d11-24fa4e58b537 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:35.903565 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0fade2-745e-4586-8a76-ac1d62d95c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:35.904862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fade2-745e-4586-8a76-ac1d62d95c42 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.905156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fade2-745e-4586-8a76-ac1d62d95c42 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.905437 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0fade2-745e-4586-8a76-ac1d62d95c42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.937020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b35b6-3deb-43e0-843c-c7234d5ddcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:35.938614 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b35b6-3deb-43e0-843c-c7234d5ddcb1 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:35.939451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b35b6-3deb-43e0-843c-c7234d5ddcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:35.940296 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b35b6-3deb-43e0-843c-c7234d5ddcb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:35.942311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b35b6-3deb-43e0-843c-c7234d5ddcb1 None admin] RBAC: Adding query filter params (group.id=78659048a89146daaf540a2065bb80f0, role.id=c8be6a897df546ca9b0e7b9541fb94d5, scope.project.id=464c75ba6b6340e4b017049e6993fe4d) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:35.943212 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b35b6-3deb-43e0-843c-c7234d5ddcb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:35.946488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b35b6-3deb-43e0-843c-c7234d5ddcb1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:35.959390 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 168/349] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:28:35 2022] GET /identity/v3/role_assignments?group.id=78659048a89146daaf540a2065bb80f0&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 260 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:35.961328 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b35b6-3deb-43e0-843c-c7234d5ddcb1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:35.992977 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:36.000021 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fade2-745e-4586-8a76-ac1d62d95c42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:36.002329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0fade2-745e-4586-8a76-ac1d62d95c42 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:36.080749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:36.099259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b35b6-3deb-43e0-843c-c7234d5ddcb1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:36.143617 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc7ee8-42a4-45a6-bb33-e2bfa60b1a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:36.144578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7ee8-42a4-45a6-bb33-e2bfa60b1a65 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.145875 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7ee8-42a4-45a6-bb33-e2bfa60b1a65 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.146175 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7ee8-42a4-45a6-bb33-e2bfa60b1a65 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.147160 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc7ee8-42a4-45a6-bb33-e2bfa60b1a65 None admin] RBAC: Adding query filter params (name=member) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:36.147481 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc7ee8-42a4-45a6-bb33-e2bfa60b1a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:36.149244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc7ee8-42a4-45a6-bb33-e2bfa60b1a65 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:36.151728 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 182/350] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:36.154789 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068595fc-1456-458e-9731-89e5af5d8382 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.155389 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068595fc-1456-458e-9731-89e5af5d8382 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.155879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068595fc-1456-458e-9731-89e5af5d8382 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.161260 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 183/351] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:36.161519 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 169/352] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:36.162642 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b8601-c25e-41f4-983d-dd5f7327bc42 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.162767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36db33b-2de7-429d-ae79-58222cac9bcd None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.162950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b8601-c25e-41f4-983d-dd5f7327bc42 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.163056 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36db33b-2de7-429d-ae79-58222cac9bcd None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.163217 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b8601-c25e-41f4-983d-dd5f7327bc42 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.163321 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36db33b-2de7-429d-ae79-58222cac9bcd None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.164949 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 184/353] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:36.165157 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 170/354] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:36.165628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b8601-c25e-41f4-983d-dd5f7327bc42 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:36.168770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f819cfd-15a6-4ee7-ac51-e9930e8555c2 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.169788 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f819cfd-15a6-4ee7-ac51-e9930e8555c2 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.170287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f819cfd-15a6-4ee7-ac51-e9930e8555c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.193587 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-9f819cfd-15a6-4ee7-ac51-e9930e8555c2 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:36.195043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-9f819cfd-15a6-4ee7-ac51-e9930e8555c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:36.279449 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 171/355] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:36.280941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be6ffc-134d-4983-b6fe-3a5daf0e553e None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.281242 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be6ffc-134d-4983-b6fe-3a5daf0e553e None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.281522 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be6ffc-134d-4983-b6fe-3a5daf0e553e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.303539 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-46be6ffc-134d-4983-b6fe-3a5daf0e553e None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:36.304796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-46be6ffc-134d-4983-b6fe-3a5daf0e553e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:36.312774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:36.334992 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b8601-c25e-41f4-983d-dd5f7327bc42 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:36.383037 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5a215-feb9-4264-8a5c-4d4bd1c98276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:36.384270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5a215-feb9-4264-8a5c-4d4bd1c98276 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.384805 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5a215-feb9-4264-8a5c-4d4bd1c98276 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.385085 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5a215-feb9-4264-8a5c-4d4bd1c98276 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.386368 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf5a215-feb9-4264-8a5c-4d4bd1c98276 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:36.387832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf5a215-feb9-4264-8a5c-4d4bd1c98276 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:36.389805 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 185/356] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:36 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:36.390512 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5a215-feb9-4264-8a5c-4d4bd1c98276 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:36.396717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 172/357] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:36.397995 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5a9d6-3932-4fbd-8cbe-067b0e29da71 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.398292 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5a9d6-3932-4fbd-8cbe-067b0e29da71 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.398577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5a9d6-3932-4fbd-8cbe-067b0e29da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.416867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:36.427188 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-c5a5a9d6-3932-4fbd-8cbe-067b0e29da71 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:36.428506 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-c5a5a9d6-3932-4fbd-8cbe-067b0e29da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:36.525576 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 173/358] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:36.526794 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5a9d6-3932-4fbd-8cbe-067b0e29da71 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:36.539152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5a215-feb9-4264-8a5c-4d4bd1c98276 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:36.586460 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22e916-3245-4b06-8633-baaa6953a4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:36.587135 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22e916-3245-4b06-8633-baaa6953a4cb None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.587414 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22e916-3245-4b06-8633-baaa6953a4cb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.587685 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22e916-3245-4b06-8633-baaa6953a4cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.623953 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:36.668800 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5a9d6-3932-4fbd-8cbe-067b0e29da71 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:36.681621 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22e916-3245-4b06-8633-baaa6953a4cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:36.683439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22e916-3245-4b06-8633-baaa6953a4cb None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:36.708067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:36.715361 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f0e8e-d989-43de-94cd-03642cf742c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:36.716174 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f0e8e-d989-43de-94cd-03642cf742c3 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.716529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f0e8e-d989-43de-94cd-03642cf742c3 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.716775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f0e8e-d989-43de-94cd-03642cf742c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.731731 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f0e8e-d989-43de-94cd-03642cf742c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-dc8f0e8e-d989-43de-94cd-03642cf742c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:36.739688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:36.740774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:36.741709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 174/359] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:36.743823 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f0e8e-d989-43de-94cd-03642cf742c3 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:36.833760 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 186/360] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:36.835072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22e916-3245-4b06-8633-baaa6953a4cb None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:36.863137 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f0e8e-d989-43de-94cd-03642cf742c3 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:36.900801 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:36.910026 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ba161-da3b-4212-b1af-0a94d33d1712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:36.914534 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ba161-da3b-4212-b1af-0a94d33d1712 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:36.914534 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ba161-da3b-4212-b1af-0a94d33d1712 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:36.914534 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ba161-da3b-4212-b1af-0a94d33d1712 None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:36.914534 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ba161-da3b-4212-b1af-0a94d33d1712 None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:36.914865 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ba161-da3b-4212-b1af-0a94d33d1712 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:36.917672 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 175/361] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:36 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:36.918089 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ba161-da3b-4212-b1af-0a94d33d1712 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:36.965087 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22e916-3245-4b06-8633-baaa6953a4cb None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:36.991195 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:37.008894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fb4b6-8f98-4a6a-a359-ec2b03233980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:37.009719 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fb4b6-8f98-4a6a-a359-ec2b03233980 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:37.010028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fb4b6-8f98-4a6a-a359-ec2b03233980 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:37.010390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fb4b6-8f98-4a6a-a359-ec2b03233980 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:37.027866 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fb4b6-8f98-4a6a-a359-ec2b03233980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-242fb4b6-8f98-4a6a-a359-ec2b03233980 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:37.039083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:37.040152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:37.041112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 187/362] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:37.043583 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fb4b6-8f98-4a6a-a359-ec2b03233980 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:37.072318 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ba161-da3b-4212-b1af-0a94d33d1712 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:37.118059 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b142dc-6d26-48cd-87ec-bd0ae433dd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:37.119581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b142dc-6d26-48cd-87ec-bd0ae433dd05 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:37.120045 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b142dc-6d26-48cd-87ec-bd0ae433dd05 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:37.120548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b142dc-6d26-48cd-87ec-bd0ae433dd05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:37.135192 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b142dc-6d26-48cd-87ec-bd0ae433dd05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-c1b142dc-6d26-48cd-87ec-bd0ae433dd05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:37.143162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:37.144245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:37.145816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:37.146586 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 176/363] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:28:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:37.147162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b142dc-6d26-48cd-87ec-bd0ae433dd05 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:37.176312 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fb4b6-8f98-4a6a-a359-ec2b03233980 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:37.210897 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:37.229415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b1a1b-2e1d-43b0-bfe4-bff7bacf39a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:37.229415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b1a1b-2e1d-43b0-bfe4-bff7bacf39a9 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:37.229415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b1a1b-2e1d-43b0-bfe4-bff7bacf39a9 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:37.229415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b1a1b-2e1d-43b0-bfe4-bff7bacf39a9 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:37.229887 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b1a1b-2e1d-43b0-bfe4-bff7bacf39a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:37.229887 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b1a1b-2e1d-43b0-bfe4-bff7bacf39a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:37.229887 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b1a1b-2e1d-43b0-bfe4-bff7bacf39a9 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:37.245672 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 188/364] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:37.245672 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b1a1b-2e1d-43b0-bfe4-bff7bacf39a9 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:37.278519 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b142dc-6d26-48cd-87ec-bd0ae433dd05 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:37.310543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:37.326588 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6debc-24e6-42f5-8ba7-b97a7c550b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:37.327568 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6debc-24e6-42f5-8ba7-b97a7c550b8f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:37.327852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6debc-24e6-42f5-8ba7-b97a7c550b8f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:37.328133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6debc-24e6-42f5-8ba7-b97a7c550b8f None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:37.329413 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6debc-24e6-42f5-8ba7-b97a7c550b8f None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:37.331026 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6debc-24e6-42f5-8ba7-b97a7c550b8f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:37.333318 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 177/365] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:37 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:37.334097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6debc-24e6-42f5-8ba7-b97a7c550b8f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:37.402923 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b1a1b-2e1d-43b0-bfe4-bff7bacf39a9 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:37.461591 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ebb69-b174-4093-8319-391e65104a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:37.462907 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ebb69-b174-4093-8319-391e65104a7f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:37.463219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ebb69-b174-4093-8319-391e65104a7f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:37.463482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ebb69-b174-4093-8319-391e65104a7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:37.464362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ebb69-b174-4093-8319-391e65104a7f None admin] RBAC: Adding query filter params (group.id=78659048a89146daaf540a2065bb80f0, role.id=ff499a8782db4cce8e162a849be212ee, scope.project.id=464c75ba6b6340e4b017049e6993fe4d) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:37.464738 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ebb69-b174-4093-8319-391e65104a7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:37.465035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6debc-24e6-42f5-8ba7-b97a7c550b8f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:37.466218 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ebb69-b174-4093-8319-391e65104a7f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:37.474785 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 189/366] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:28:37 2022] GET /identity/v3/role_assignments?group.id=78659048a89146daaf540a2065bb80f0&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=ff499a8782db4cce8e162a849be212ee => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:37.475751 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ebb69-b174-4093-8319-391e65104a7f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:37.508473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:37.521702 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ec3b1-86f9-4d9e-8dd2-e2cfd226a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:37.524020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ec3b1-86f9-4d9e-8dd2-e2cfd226a7ac None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:37.524587 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ec3b1-86f9-4d9e-8dd2-e2cfd226a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:37.525216 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4ec3b1-86f9-4d9e-8dd2-e2cfd226a7ac None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:37.526322 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ec3b1-86f9-4d9e-8dd2-e2cfd226a7ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:37.527385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ec3b1-86f9-4d9e-8dd2-e2cfd226a7ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:37.529932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4ec3b1-86f9-4d9e-8dd2-e2cfd226a7ac None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:37.542852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 178/367] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:37.545410 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ec3b1-86f9-4d9e-8dd2-e2cfd226a7ac None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:37.626715 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ebb69-b174-4093-8319-391e65104a7f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:37.636368 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:37.669869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3708a9-a7a0-416c-9a39-1e8fa658ce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:37.671029 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3708a9-a7a0-416c-9a39-1e8fa658ce8b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:37.671528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3708a9-a7a0-416c-9a39-1e8fa658ce8b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:37.672148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3708a9-a7a0-416c-9a39-1e8fa658ce8b None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:37.673319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3708a9-a7a0-416c-9a39-1e8fa658ce8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:37.673859 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3708a9-a7a0-416c-9a39-1e8fa658ce8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:37.675756 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3708a9-a7a0-416c-9a39-1e8fa658ce8b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:37.679619 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4ec3b1-86f9-4d9e-8dd2-e2cfd226a7ac None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:37.687527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 190/368] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:37.689023 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3708a9-a7a0-416c-9a39-1e8fa658ce8b None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:37.727602 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6384-d40b-4119-857c-76f3a9c92973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:37.728536 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6384-d40b-4119-857c-76f3a9c92973 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:37.728823 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6384-d40b-4119-857c-76f3a9c92973 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:37.729102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6384-d40b-4119-857c-76f3a9c92973 None admin] PATH_INFO: `/v3/users/e1f5282a6792425ea98f9ea8fd96e1a9` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:37.730398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a6384-d40b-4119-857c-76f3a9c92973 None admin] RBAC: Authorizing `identity:get_user(user_id=e1f5282a6792425ea98f9ea8fd96e1a9)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:37.732044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a6384-d40b-4119-857c-76f3a9c92973 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:37.734477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 179/369] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:28:37 2022] GET /identity/v3/users/e1f5282a6792425ea98f9ea8fd96e1a9 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:37.735291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6384-d40b-4119-857c-76f3a9c92973 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:37.818093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:37.831528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3708a9-a7a0-416c-9a39-1e8fa658ce8b None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:37.888666 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9278122-af16-432f-be20-61479aa602d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:37.889579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278122-af16-432f-be20-61479aa602d7 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:37.889858 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278122-af16-432f-be20-61479aa602d7 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:37.890128 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9278122-af16-432f-be20-61479aa602d7 None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:37.891546 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9278122-af16-432f-be20-61479aa602d7 None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:37.893375 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9278122-af16-432f-be20-61479aa602d7 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:37.899937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6384-d40b-4119-857c-76f3a9c92973 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:37.902186 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 191/370] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:37 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:37.902186 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9278122-af16-432f-be20-61479aa602d7 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:37.946324 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:37.950693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eaed69-d4c5-45f4-a3ea-3bae453cb82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:37.951847 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaed69-d4c5-45f4-a3ea-3bae453cb82f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:37.952379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaed69-d4c5-45f4-a3ea-3bae453cb82f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:37.952859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaed69-d4c5-45f4-a3ea-3bae453cb82f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:38.033104 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9278122-af16-432f-be20-61479aa602d7 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:38.049234 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eaed69-d4c5-45f4-a3ea-3bae453cb82f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:38.051662 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eaed69-d4c5-45f4-a3ea-3bae453cb82f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:38.075892 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af6d5b-6c89-40b2-a39e-0a5763381f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:38.076541 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6d5b-6c89-40b2-a39e-0a5763381f5e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:38.076831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6d5b-6c89-40b2-a39e-0a5763381f5e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:38.077116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6d5b-6c89-40b2-a39e-0a5763381f5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:38.142210 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:38.189748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af6d5b-6c89-40b2-a39e-0a5763381f5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:38.192496 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af6d5b-6c89-40b2-a39e-0a5763381f5e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:38.214548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 180/371] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:38.215542 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eaed69-d4c5-45f4-a3ea-3bae453cb82f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:38.239881 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:38.337066 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 192/372] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:38.338176 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af6d5b-6c89-40b2-a39e-0a5763381f5e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:38.348704 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eaed69-d4c5-45f4-a3ea-3bae453cb82f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:38.387909 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7524d71-76c6-4cdd-b720-ddd3691e5fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:38.388425 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7524d71-76c6-4cdd-b720-ddd3691e5fc7 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:38.388714 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7524d71-76c6-4cdd-b720-ddd3691e5fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:38.388989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7524d71-76c6-4cdd-b720-ddd3691e5fc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:38.389839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7524d71-76c6-4cdd-b720-ddd3691e5fc7 None admin] RBAC: Adding query filter params (role.id=7b870ddcb3b04c6088d9d80f29ab8193, user.id=e1f5282a6792425ea98f9ea8fd96e1a9, scope.system=all) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:38.390155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7524d71-76c6-4cdd-b720-ddd3691e5fc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:38.391603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7524d71-76c6-4cdd-b720-ddd3691e5fc7 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:38.405518 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 181/373] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:28:38 2022] GET /identity/v3/role_assignments?user.id=e1f5282a6792425ea98f9ea8fd96e1a9&scope.system=all&role.id=7b870ddcb3b04c6088d9d80f29ab8193 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:38.406452 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7524d71-76c6-4cdd-b720-ddd3691e5fc7 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:38.425311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:38.457911 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af6d5b-6c89-40b2-a39e-0a5763381f5e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:38.497827 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f4b3e-b23f-4c65-86e3-df7086d303ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:38.498561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f4b3e-b23f-4c65-86e3-df7086d303ec None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:38.498902 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f4b3e-b23f-4c65-86e3-df7086d303ec None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:38.499181 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f4b3e-b23f-4c65-86e3-df7086d303ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:38.500024 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f4b3e-b23f-4c65-86e3-df7086d303ec None admin] RBAC: Adding query filter params (group.id=78659048a89146daaf540a2065bb80f0, role.id=c8be6a897df546ca9b0e7b9541fb94d5, scope.project.id=c9a4c742053041558d2d857f27eedf48) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:38.500361 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f4b3e-b23f-4c65-86e3-df7086d303ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:38.501827 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f4b3e-b23f-4c65-86e3-df7086d303ec None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:38.507908 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:38.512017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 193/374] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:28:38 2022] GET /identity/v3/role_assignments?group.id=78659048a89146daaf540a2065bb80f0&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:38.512921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f4b3e-b23f-4c65-86e3-df7086d303ec None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:38.527800 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7524d71-76c6-4cdd-b720-ddd3691e5fc7 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:38.574346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc53f33-3cc0-4113-b0a8-89ade32a39af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:38.575407 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53f33-3cc0-4113-b0a8-89ade32a39af None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:38.575706 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53f33-3cc0-4113-b0a8-89ade32a39af None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:38.575979 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc53f33-3cc0-4113-b0a8-89ade32a39af None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:38.577486 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc53f33-3cc0-4113-b0a8-89ade32a39af None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:38.579049 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc53f33-3cc0-4113-b0a8-89ade32a39af None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:38.580954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 182/375] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:38 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:38.586934 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc53f33-3cc0-4113-b0a8-89ade32a39af None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:38.692224 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f4b3e-b23f-4c65-86e3-df7086d303ec None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:38.719049 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:38.739131 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827fc89-9b04-4fa1-9ec6-7b9dbc63f7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:38.740676 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827fc89-9b04-4fa1-9ec6-7b9dbc63f7df None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:38.741253 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827fc89-9b04-4fa1-9ec6-7b9dbc63f7df None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:38.741543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827fc89-9b04-4fa1-9ec6-7b9dbc63f7df None admin] PATH_INFO: `/v3/projects/e556dd7d9fbd4a6e9dbb4191eee92638` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:38.742889 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827fc89-9b04-4fa1-9ec6-7b9dbc63f7df None admin] RBAC: Authorizing `identity:get_project(project_id=e556dd7d9fbd4a6e9dbb4191eee92638)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:38.743118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc53f33-3cc0-4113-b0a8-89ade32a39af None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:38.744462 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827fc89-9b04-4fa1-9ec6-7b9dbc63f7df None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:38.748018 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 194/376] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:38 2022] GET /identity/v3/projects/e556dd7d9fbd4a6e9dbb4191eee92638 => generated 344 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:38.749462 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023047b-4b50-4db1-9554-bc7de7f920bc None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:38.749749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023047b-4b50-4db1-9554-bc7de7f920bc None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:38.750026 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0023047b-4b50-4db1-9554-bc7de7f920bc None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:38.751932 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 195/377] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:38.753155 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023047b-4b50-4db1-9554-bc7de7f920bc None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:38.799793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c5b00-d805-4484-ac30-d1dd7db14c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:38.800881 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c5b00-d805-4484-ac30-d1dd7db14c1a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:38.801886 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c5b00-d805-4484-ac30-d1dd7db14c1a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:38.802235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c5b00-d805-4484-ac30-d1dd7db14c1a None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:38.803682 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c5b00-d805-4484-ac30-d1dd7db14c1a None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:38.805253 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c5b00-d805-4484-ac30-d1dd7db14c1a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:38.808929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 183/378] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:38 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:38.810288 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf10ac6-ee0a-4911-9c8d-d22bf3f54e2e None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:38.810579 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf10ac6-ee0a-4911-9c8d-d22bf3f54e2e None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:38.810851 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf10ac6-ee0a-4911-9c8d-d22bf3f54e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:38.835543 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-2bf10ac6-ee0a-4911-9c8d-d22bf3f54e2e None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:38.837154 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-2bf10ac6-ee0a-4911-9c8d-d22bf3f54e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:38.848506 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:38.907853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0023047b-4b50-4db1-9554-bc7de7f920bc None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:38.932248 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 184/379] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:38.933254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf10ac6-ee0a-4911-9c8d-d22bf3f54e2e None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:38.975529 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a4089-a244-4bbd-82db-b7dbeda18c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:38.976362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a4089-a244-4bbd-82db-b7dbeda18c70 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:38.976650 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a4089-a244-4bbd-82db-b7dbeda18c70 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:38.976925 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a4089-a244-4bbd-82db-b7dbeda18c70 None admin] PATH_INFO: `/v3/groups/22d8794c67914d558453bf62870e87ed` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:38.984976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a4089-a244-4bbd-82db-b7dbeda18c70 None admin] RBAC: Authorizing `identity:get_group(group_id=22d8794c67914d558453bf62870e87ed)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:38.986492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a4089-a244-4bbd-82db-b7dbeda18c70 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:38.988697 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 196/380] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:38 2022] GET /identity/v3/groups/22d8794c67914d558453bf62870e87ed => generated 229 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:38.989649 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a4089-a244-4bbd-82db-b7dbeda18c70 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:39.028088 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:39.053662 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf10ac6-ee0a-4911-9c8d-d22bf3f54e2e None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:39.096610 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10458416-c6af-407f-ad13-ec3d5ed41b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:39.097381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10458416-c6af-407f-ad13-ec3d5ed41b17 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.098294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10458416-c6af-407f-ad13-ec3d5ed41b17 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.099137 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10458416-c6af-407f-ad13-ec3d5ed41b17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.100142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10458416-c6af-407f-ad13-ec3d5ed41b17 None admin] RBAC: Adding query filter params (group.id=78659048a89146daaf540a2065bb80f0, role.id=ff499a8782db4cce8e162a849be212ee, scope.project.id=c9a4c742053041558d2d857f27eedf48) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:39.100494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10458416-c6af-407f-ad13-ec3d5ed41b17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:39.102719 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10458416-c6af-407f-ad13-ec3d5ed41b17 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:39.110822 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 185/381] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:28:38 2022] GET /identity/v3/role_assignments?group.id=78659048a89146daaf540a2065bb80f0&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=ff499a8782db4cce8e162a849be212ee => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:39.112011 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10458416-c6af-407f-ad13-ec3d5ed41b17 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:39.147678 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a4089-a244-4bbd-82db-b7dbeda18c70 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:39.195165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:39.203949 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586cedb-f88f-4e4b-bd5b-62af1c4443eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:39.204865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586cedb-f88f-4e4b-bd5b-62af1c4443eb None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.205439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586cedb-f88f-4e4b-bd5b-62af1c4443eb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.208402 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586cedb-f88f-4e4b-bd5b-62af1c4443eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.222321 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0586cedb-f88f-4e4b-bd5b-62af1c4443eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-0586cedb-f88f-4e4b-bd5b-62af1c4443eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:39.237120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:39.238945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:39.240351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 197/382] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:39.247999 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586cedb-f88f-4e4b-bd5b-62af1c4443eb None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:39.266445 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10458416-c6af-407f-ad13-ec3d5ed41b17 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:39.309341 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bf106-cf3a-4c31-b45f-a759a6fcba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:39.310039 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bf106-cf3a-4c31-b45f-a759a6fcba58 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.310327 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bf106-cf3a-4c31-b45f-a759a6fcba58 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.310605 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bf106-cf3a-4c31-b45f-a759a6fcba58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.311532 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bf106-cf3a-4c31-b45f-a759a6fcba58 None admin] RBAC: Adding query filter params (group.id=22d8794c67914d558453bf62870e87ed, role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=e556dd7d9fbd4a6e9dbb4191eee92638) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:39.311856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bf106-cf3a-4c31-b45f-a759a6fcba58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:39.313491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bf106-cf3a-4c31-b45f-a759a6fcba58 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:39.320784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 186/383] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:28:39 2022] GET /identity/v3/role_assignments?group.id=22d8794c67914d558453bf62870e87ed&scope.project.id=e556dd7d9fbd4a6e9dbb4191eee92638&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:39.409707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586cedb-f88f-4e4b-bd5b-62af1c4443eb None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:39.462390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892341a8-bb1f-43cc-97e0-9e112a2a3a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:39.463096 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892341a8-bb1f-43cc-97e0-9e112a2a3a43 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.463377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892341a8-bb1f-43cc-97e0-9e112a2a3a43 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.463651 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892341a8-bb1f-43cc-97e0-9e112a2a3a43 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.464451 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892341a8-bb1f-43cc-97e0-9e112a2a3a43 None admin] RBAC: Adding query filter params (name=member) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:39.464760 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892341a8-bb1f-43cc-97e0-9e112a2a3a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:39.466216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892341a8-bb1f-43cc-97e0-9e112a2a3a43 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:39.476905 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 198/384] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:39.483673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bf106-cf3a-4c31-b45f-a759a6fcba58 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:39.524896 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:39.635924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bf106-cf3a-4c31-b45f-a759a6fcba58 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:39.676377 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e771b9e-3241-405f-8cc5-431391581bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:39.677292 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e771b9e-3241-405f-8cc5-431391581bc4 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.677643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e771b9e-3241-405f-8cc5-431391581bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.677935 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e771b9e-3241-405f-8cc5-431391581bc4 None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.679593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e771b9e-3241-405f-8cc5-431391581bc4 None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:39.681235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e771b9e-3241-405f-8cc5-431391581bc4 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:39.682932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 187/385] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:39 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:39.688875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892341a8-bb1f-43cc-97e0-9e112a2a3a43 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:39.764602 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:39.813656 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892341a8-bb1f-43cc-97e0-9e112a2a3a43 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:39.849659 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0509ed-e201-49cc-92c8-43d7547b2bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:39.850545 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0509ed-e201-49cc-92c8-43d7547b2bbb None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:39.850826 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0509ed-e201-49cc-92c8-43d7547b2bbb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:39.851096 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0509ed-e201-49cc-92c8-43d7547b2bbb None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:39.852319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0509ed-e201-49cc-92c8-43d7547b2bbb None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:39.853805 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0509ed-e201-49cc-92c8-43d7547b2bbb None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:39.855581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 199/386] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:39 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:39.865447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e771b9e-3241-405f-8cc5-431391581bc4 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:40.005777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:40.033539 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e771b9e-3241-405f-8cc5-431391581bc4 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:40.084920 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc448f-a09e-4d8a-a598-2be7278a8b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:40.086082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc448f-a09e-4d8a-a598-2be7278a8b52 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:40.086532 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc448f-a09e-4d8a-a598-2be7278a8b52 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:40.086958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc448f-a09e-4d8a-a598-2be7278a8b52 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d/groups/78659048a89146daaf540a2065bb80f0/roles/c8be6a897df546ca9b0e7b9541fb94d5` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:40.089215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc448f-a09e-4d8a-a598-2be7278a8b52 None admin] RBAC: Authorizing `identity:create_grant(project_id=464c75ba6b6340e4b017049e6993fe4d, group_id=78659048a89146daaf540a2065bb80f0, role_id=c8be6a897df546ca9b0e7b9541fb94d5)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:40.090859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc448f-a09e-4d8a-a598-2be7278a8b52 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:40.174483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 188/387] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:28:39 2022] PUT /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d/groups/78659048a89146daaf540a2065bb80f0/roles/c8be6a897df546ca9b0e7b9541fb94d5 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:40.552045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9122e-e7c4-4b0e-8d24-028c42fdcf0d None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:40.552629 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9122e-e7c4-4b0e-8d24-028c42fdcf0d None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:40.552629 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9122e-e7c4-4b0e-8d24-028c42fdcf0d None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:40.554210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 200/388] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:40.560375 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf087a9-46ec-4882-b011-d20075dac7ea None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:40.561300 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf087a9-46ec-4882-b011-d20075dac7ea None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:40.561881 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf087a9-46ec-4882-b011-d20075dac7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:40.582428 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-2cf087a9-46ec-4882-b011-d20075dac7ea None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:40.584025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-2cf087a9-46ec-4882-b011-d20075dac7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:40.727991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 189/389] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:40.763780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9122e-e7c4-4b0e-8d24-028c42fdcf0d None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:40.936284 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9122e-e7c4-4b0e-8d24-028c42fdcf0d None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:40.978508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:40.992485 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aca2a7-d97d-4b37-a533-8461066a9e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:40.993343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aca2a7-d97d-4b37-a533-8461066a9e4c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:40.993696 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aca2a7-d97d-4b37-a533-8461066a9e4c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:40.994463 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aca2a7-d97d-4b37-a533-8461066a9e4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:41.012929 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aca2a7-d97d-4b37-a533-8461066a9e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-e1aca2a7-d97d-4b37-a533-8461066a9e4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:41.022231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:41.023327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:41.024239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:41.028175 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 201/390] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:41.030077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf087a9-46ec-4882-b011-d20075dac7ea None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:41.156322 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:41.198108 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf087a9-46ec-4882-b011-d20075dac7ea None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:41.246915 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997020d-b65f-40be-9d79-63ac75b709df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:41.247526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997020d-b65f-40be-9d79-63ac75b709df None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:41.247813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997020d-b65f-40be-9d79-63ac75b709df None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:41.248092 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997020d-b65f-40be-9d79-63ac75b709df None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:41.248908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997020d-b65f-40be-9d79-63ac75b709df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:41.249219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997020d-b65f-40be-9d79-63ac75b709df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:41.250596 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997020d-b65f-40be-9d79-63ac75b709df None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:41.266954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 190/391] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:41.272261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aca2a7-d97d-4b37-a533-8461066a9e4c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:41.334441 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45781eee-b267-495a-bbcb-98d9bd94b5c1 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:41.334765 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45781eee-b267-495a-bbcb-98d9bd94b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:41.335027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45781eee-b267-495a-bbcb-98d9bd94b5c1 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:41.336835 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 191/392] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:41.349756 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32628f9-8a23-472d-ab27-78f9dc12ddb0 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:41.350190 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32628f9-8a23-472d-ab27-78f9dc12ddb0 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:41.350612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32628f9-8a23-472d-ab27-78f9dc12ddb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:41.395986 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-e32628f9-8a23-472d-ab27-78f9dc12ddb0 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:41.397265 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-e32628f9-8a23-472d-ab27-78f9dc12ddb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:41.409749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aca2a7-d97d-4b37-a533-8461066a9e4c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:41.462796 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af9d5c-7960-4485-97a9-bd16bff74373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:41.463679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af9d5c-7960-4485-97a9-bd16bff74373 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:41.464018 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af9d5c-7960-4485-97a9-bd16bff74373 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:41.464297 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af9d5c-7960-4485-97a9-bd16bff74373 None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:41.465673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af9d5c-7960-4485-97a9-bd16bff74373 None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:41.467239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af9d5c-7960-4485-97a9-bd16bff74373 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:41.469237 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 202/393] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:41 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:41.474867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af9d5c-7960-4485-97a9-bd16bff74373 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:41.530827 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:41.570983 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 192/394] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:41.586175 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af9d5c-7960-4485-97a9-bd16bff74373 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:41.619771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32628f9-8a23-472d-ab27-78f9dc12ddb0 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:41.627190 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f545e0a-74dc-4c9f-a2fb-1586c3f39262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:41.628459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545e0a-74dc-4c9f-a2fb-1586c3f39262 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:41.628951 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545e0a-74dc-4c9f-a2fb-1586c3f39262 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:41.629403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f545e0a-74dc-4c9f-a2fb-1586c3f39262 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:41.630958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f545e0a-74dc-4c9f-a2fb-1586c3f39262 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:41.632657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f545e0a-74dc-4c9f-a2fb-1586c3f39262 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:41.635217 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 203/395] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:41 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:41.641040 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f545e0a-74dc-4c9f-a2fb-1586c3f39262 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:41.750216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f545e0a-74dc-4c9f-a2fb-1586c3f39262 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:41.776870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:41.789884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f71380-c556-4f9c-83a9-06fc4d5af4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:41.791748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f71380-c556-4f9c-83a9-06fc4d5af4d6 None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:41.792776 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f71380-c556-4f9c-83a9-06fc4d5af4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:41.797165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f71380-c556-4f9c-83a9-06fc4d5af4d6 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d/groups/78659048a89146daaf540a2065bb80f0/roles/ff499a8782db4cce8e162a849be212ee` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:41.797288 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:41.797374 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f71380-c556-4f9c-83a9-06fc4d5af4d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=464c75ba6b6340e4b017049e6993fe4d, group_id=78659048a89146daaf540a2065bb80f0, role_id=ff499a8782db4cce8e162a849be212ee)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:41.799024 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f71380-c556-4f9c-83a9-06fc4d5af4d6 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:41.846991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32628f9-8a23-472d-ab27-78f9dc12ddb0 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:41.886680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 204/396] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:28:41 2022] PUT /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d/groups/78659048a89146daaf540a2065bb80f0/roles/ff499a8782db4cce8e162a849be212ee => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:28:41.897425 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8b3d7-cd92-4cf2-9df2-ec791569905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:41.898821 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8b3d7-cd92-4cf2-9df2-ec791569905d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:41.899338 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8b3d7-cd92-4cf2-9df2-ec791569905d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:41.899806 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8b3d7-cd92-4cf2-9df2-ec791569905d None admin] PATH_INFO: `/v3/roles/member` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:41.922843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa8b3d7-cd92-4cf2-9df2-ec791569905d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-2fa8b3d7-cd92-4cf2-9df2-ec791569905d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:41.932010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:41.933155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:41.934061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:41.940889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 193/397] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:41.947001 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f71380-c556-4f9c-83a9-06fc4d5af4d6 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:42.102420 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f71380-c556-4f9c-83a9-06fc4d5af4d6 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:42.145272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05796f33-0395-4c03-9bd9-a60006e7d841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:42.145995 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05796f33-0395-4c03-9bd9-a60006e7d841 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:42.146285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05796f33-0395-4c03-9bd9-a60006e7d841 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:42.146558 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05796f33-0395-4c03-9bd9-a60006e7d841 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:42.147342 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05796f33-0395-4c03-9bd9-a60006e7d841 None admin] RBAC: Adding query filter params (name=member) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:42.147654 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05796f33-0395-4c03-9bd9-a60006e7d841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:42.149148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05796f33-0395-4c03-9bd9-a60006e7d841 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:42.159811 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 205/398] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:42.169504 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8b3d7-cd92-4cf2-9df2-ec791569905d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:42.243697 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c044e5-0ce6-4c4b-817c-d12ff44b01c0 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:42.244082 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c044e5-0ce6-4c4b-817c-d12ff44b01c0 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:42.244271 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c044e5-0ce6-4c4b-817c-d12ff44b01c0 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:42.246144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 206/399] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:42.253778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee97c67-5bd3-4280-8618-ea78c4a6a259 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:42.254178 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee97c67-5bd3-4280-8618-ea78c4a6a259 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:42.254461 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee97c67-5bd3-4280-8618-ea78c4a6a259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:42.273474 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-aee97c67-5bd3-4280-8618-ea78c4a6a259 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:42.274689 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-aee97c67-5bd3-4280-8618-ea78c4a6a259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:42.304805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa8b3d7-cd92-4cf2-9df2-ec791569905d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:42.316602 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:42.335405 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:42.353740 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b226a1f-49c3-43eb-886b-c88c2409e252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:42.354536 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b226a1f-49c3-43eb-886b-c88c2409e252 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:42.354823 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b226a1f-49c3-43eb-886b-c88c2409e252 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:42.355100 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b226a1f-49c3-43eb-886b-c88c2409e252 None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:42.355735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 207/400] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:42.356401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b226a1f-49c3-43eb-886b-c88c2409e252 None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:42.357928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b226a1f-49c3-43eb-886b-c88c2409e252 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:42.359752 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 194/401] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:42 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:42.360644 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7a2e3-b154-45cb-a8e5-c89bc486911e None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:42.360942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7a2e3-b154-45cb-a8e5-c89bc486911e None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:42.361219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7a2e3-b154-45cb-a8e5-c89bc486911e None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:42.362892 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 208/402] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:42.369312 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b226a1f-49c3-43eb-886b-c88c2409e252 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:42.371385 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d57d37-58c4-472f-8c2d-11a40b3ce4e8 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:42.371492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d57d37-58c4-472f-8c2d-11a40b3ce4e8 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:42.371701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d57d37-58c4-472f-8c2d-11a40b3ce4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:42.393127 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-67d57d37-58c4-472f-8c2d-11a40b3ce4e8 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:42.394491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-67d57d37-58c4-472f-8c2d-11a40b3ce4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:42.473196 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 209/403] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:42.474156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d57d37-58c4-472f-8c2d-11a40b3ce4e8 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:42.500595 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b226a1f-49c3-43eb-886b-c88c2409e252 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:42.549135 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a569b-cd04-400b-83f1-7c5faa0a354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:42.550059 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a569b-cd04-400b-83f1-7c5faa0a354b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:42.554624 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a569b-cd04-400b-83f1-7c5faa0a354b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:42.554624 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a569b-cd04-400b-83f1-7c5faa0a354b None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:42.554624 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a569b-cd04-400b-83f1-7c5faa0a354b None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:42.554624 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a569b-cd04-400b-83f1-7c5faa0a354b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:42.555741 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 195/404] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:42 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:42.556612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a569b-cd04-400b-83f1-7c5faa0a354b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:42.592319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:42.596214 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d57d37-58c4-472f-8c2d-11a40b3ce4e8 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:42.618613 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:42.638319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c68d64-fb3c-4458-acb1-8db6b420bc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:42.639147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c68d64-fb3c-4458-acb1-8db6b420bc43 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:42.639433 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c68d64-fb3c-4458-acb1-8db6b420bc43 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:42.639714 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c68d64-fb3c-4458-acb1-8db6b420bc43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:42.655894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c68d64-fb3c-4458-acb1-8db6b420bc43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-b0c68d64-fb3c-4458-acb1-8db6b420bc43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:42.669169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:42.671497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:42.672994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:42.673906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a569b-cd04-400b-83f1-7c5faa0a354b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:42.680362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 210/405] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:42.680362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c68d64-fb3c-4458-acb1-8db6b420bc43 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:42.713726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d5be9-bd2b-4d3a-90e6-214c6b6b218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:42.714706 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d5be9-bd2b-4d3a-90e6-214c6b6b218b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:42.715088 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d5be9-bd2b-4d3a-90e6-214c6b6b218b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:42.715531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d5be9-bd2b-4d3a-90e6-214c6b6b218b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:42.734703 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d5be9-bd2b-4d3a-90e6-214c6b6b218b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-e31d5be9-bd2b-4d3a-90e6-214c6b6b218b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:42.745757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:42.748002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 196/406] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:28:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:42.750121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d5be9-bd2b-4d3a-90e6-214c6b6b218b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:42.812720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c68d64-fb3c-4458-acb1-8db6b420bc43 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:42.850667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83c8f9-ec1a-4883-85ca-eb5c3d219834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:42.852215 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83c8f9-ec1a-4883-85ca-eb5c3d219834 None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:42.852649 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83c8f9-ec1a-4883-85ca-eb5c3d219834 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:42.852933 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83c8f9-ec1a-4883-85ca-eb5c3d219834 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48/groups/78659048a89146daaf540a2065bb80f0/roles/c8be6a897df546ca9b0e7b9541fb94d5` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:42.855043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83c8f9-ec1a-4883-85ca-eb5c3d219834 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a4c742053041558d2d857f27eedf48, group_id=78659048a89146daaf540a2065bb80f0, role_id=c8be6a897df546ca9b0e7b9541fb94d5)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:42.856552 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83c8f9-ec1a-4883-85ca-eb5c3d219834 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:42.884252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d5be9-bd2b-4d3a-90e6-214c6b6b218b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:42.899335 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:42.920686 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221377e4-1f05-4644-a0ab-834bce253ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:42.921375 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221377e4-1f05-4644-a0ab-834bce253ee3 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:42.921658 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221377e4-1f05-4644-a0ab-834bce253ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:42.921929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221377e4-1f05-4644-a0ab-834bce253ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:42.922686 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221377e4-1f05-4644-a0ab-834bce253ee3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:42.923169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221377e4-1f05-4644-a0ab-834bce253ee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:42.924618 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221377e4-1f05-4644-a0ab-834bce253ee3 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:42.936948 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 197/407] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:42.937882 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221377e4-1f05-4644-a0ab-834bce253ee3 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:42.945412 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 211/408] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:28:42 2022] PUT /identity/v3/projects/c9a4c742053041558d2d857f27eedf48/groups/78659048a89146daaf540a2065bb80f0/roles/c8be6a897df546ca9b0e7b9541fb94d5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:42.946704 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83c8f9-ec1a-4883-85ca-eb5c3d219834 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:42.985614 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:43.081143 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221377e4-1f05-4644-a0ab-834bce253ee3 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:43.127746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f9515-5535-4ba2-b34c-8ea4f32b8569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:43.129196 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f9515-5535-4ba2-b34c-8ea4f32b8569 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:43.129765 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f9515-5535-4ba2-b34c-8ea4f32b8569 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:43.130586 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f9515-5535-4ba2-b34c-8ea4f32b8569 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:43.131619 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f9515-5535-4ba2-b34c-8ea4f32b8569 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:43.132142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f9515-5535-4ba2-b34c-8ea4f32b8569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:43.133788 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f9515-5535-4ba2-b34c-8ea4f32b8569 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:43.146070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 198/409] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:43.154207 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f9515-5535-4ba2-b34c-8ea4f32b8569 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:43.157711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83c8f9-ec1a-4883-85ca-eb5c3d219834 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:43.235253 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:43.236198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b118e92-a591-4c9c-a954-0b04d3688c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:43.237137 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b118e92-a591-4c9c-a954-0b04d3688c9e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:43.237425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b118e92-a591-4c9c-a954-0b04d3688c9e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:43.237704 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b118e92-a591-4c9c-a954-0b04d3688c9e None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:43.239987 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b118e92-a591-4c9c-a954-0b04d3688c9e None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:43.242146 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b118e92-a591-4c9c-a954-0b04d3688c9e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:43.245095 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 212/410] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:42 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:43.249274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211f4ae-afd6-48b7-a033-1b6c5e8f7d89 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:43.249274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211f4ae-afd6-48b7-a033-1b6c5e8f7d89 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:43.249274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211f4ae-afd6-48b7-a033-1b6c5e8f7d89 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:43.249274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 213/411] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:28:43.252914 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211f4ae-afd6-48b7-a033-1b6c5e8f7d89 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:43.318040 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f9515-5535-4ba2-b34c-8ea4f32b8569 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:43.360809 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:43.368003 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2502d49-589c-4f4d-844d-041319dc6838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:43.369105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2502d49-589c-4f4d-844d-041319dc6838 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:43.369513 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2502d49-589c-4f4d-844d-041319dc6838 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:43.369866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2502d49-589c-4f4d-844d-041319dc6838 None admin] PATH_INFO: `/v3/groups/22d8794c67914d558453bf62870e87ed` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:43.371199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2502d49-589c-4f4d-844d-041319dc6838 None admin] RBAC: Authorizing `identity:get_group(group_id=22d8794c67914d558453bf62870e87ed)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:43.373238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2502d49-589c-4f4d-844d-041319dc6838 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:43.373711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211f4ae-afd6-48b7-a033-1b6c5e8f7d89 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:43.375250 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 199/412] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:43 2022] GET /identity/v3/groups/22d8794c67914d558453bf62870e87ed => generated 229 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:43.381269 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d1ae4-6203-4f90-88cf-0d9bf2019831 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:43.388551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d1ae4-6203-4f90-88cf-0d9bf2019831 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:43.388551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d1ae4-6203-4f90-88cf-0d9bf2019831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:43.402126 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-548d1ae4-6203-4f90-88cf-0d9bf2019831 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:43.404115 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-548d1ae4-6203-4f90-88cf-0d9bf2019831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:43.423838 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c1847-0d69-4bee-bed8-d1be90991136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:43.424800 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c1847-0d69-4bee-bed8-d1be90991136 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:43.425049 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c1847-0d69-4bee-bed8-d1be90991136 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:43.425328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c1847-0d69-4bee-bed8-d1be90991136 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:43.427273 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c1847-0d69-4bee-bed8-d1be90991136 None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:43.429255 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c1847-0d69-4bee-bed8-d1be90991136 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:43.431201 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 214/413] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:43 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:43.431979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c1847-0d69-4bee-bed8-d1be90991136 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:43.518942 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 200/414] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:43.518942 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d1ae4-6203-4f90-88cf-0d9bf2019831 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:43.639173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c1847-0d69-4bee-bed8-d1be90991136 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:43.643818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:43.690035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1790cd-775f-4d44-b648-bf7e5d829068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:43.691610 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1790cd-775f-4d44-b648-bf7e5d829068 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:43.692379 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1790cd-775f-4d44-b648-bf7e5d829068 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:43.692919 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1790cd-775f-4d44-b648-bf7e5d829068 None admin] PATH_INFO: `/v3/projects/e556dd7d9fbd4a6e9dbb4191eee92638` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:43.694853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1790cd-775f-4d44-b648-bf7e5d829068 None admin] RBAC: Authorizing `identity:get_project(project_id=e556dd7d9fbd4a6e9dbb4191eee92638)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:43.697321 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1790cd-775f-4d44-b648-bf7e5d829068 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:43.700260 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 215/415] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:43 2022] GET /identity/v3/projects/e556dd7d9fbd4a6e9dbb4191eee92638 => generated 344 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:43.701694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1790cd-775f-4d44-b648-bf7e5d829068 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:43.716592 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d1ae4-6203-4f90-88cf-0d9bf2019831 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:43.739182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:43.759733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc092d-4d5c-416a-a9b4-f6000e383dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:43.760774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc092d-4d5c-416a-a9b4-f6000e383dbf None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:43.761135 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc092d-4d5c-416a-a9b4-f6000e383dbf None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:43.761480 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc092d-4d5c-416a-a9b4-f6000e383dbf None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48/groups/78659048a89146daaf540a2065bb80f0/roles/ff499a8782db4cce8e162a849be212ee` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:43.764022 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc092d-4d5c-416a-a9b4-f6000e383dbf None admin] RBAC: Authorizing `identity:create_grant(project_id=c9a4c742053041558d2d857f27eedf48, group_id=78659048a89146daaf540a2065bb80f0, role_id=ff499a8782db4cce8e162a849be212ee)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:43.765649 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc092d-4d5c-416a-a9b4-f6000e383dbf None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:43.836041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1790cd-775f-4d44-b648-bf7e5d829068 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:43.845998 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 201/416] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:28:43 2022] PUT /identity/v3/projects/c9a4c742053041558d2d857f27eedf48/groups/78659048a89146daaf540a2065bb80f0/roles/ff499a8782db4cce8e162a849be212ee => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:43.847399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc092d-4d5c-416a-a9b4-f6000e383dbf None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:43.910730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eda89a-2837-4bdc-81a8-a00ca8c97dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:43.913711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eda89a-2837-4bdc-81a8-a00ca8c97dd4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:43.915814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eda89a-2837-4bdc-81a8-a00ca8c97dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:43.916429 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eda89a-2837-4bdc-81a8-a00ca8c97dd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:43.960828 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eda89a-2837-4bdc-81a8-a00ca8c97dd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-a4eda89a-2837-4bdc-81a8-a00ca8c97dd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:43.975852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:43.977046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:43.977948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:43.980743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 216/417] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:43.986779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eda89a-2837-4bdc-81a8-a00ca8c97dd4 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:44.033980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc092d-4d5c-416a-a9b4-f6000e383dbf None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:44.043309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:44.081124 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789493f0-ae6b-48bf-8eff-02f7016aaa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:44.082629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789493f0-ae6b-48bf-8eff-02f7016aaa03 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:44.082909 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789493f0-ae6b-48bf-8eff-02f7016aaa03 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:44.083180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789493f0-ae6b-48bf-8eff-02f7016aaa03 None admin] PATH_INFO: `/v3/projects/e556dd7d9fbd4a6e9dbb4191eee92638/groups/22d8794c67914d558453bf62870e87ed/roles/8eaff9cd613a49b29548e90d7e3e9215` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:44.089232 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789493f0-ae6b-48bf-8eff-02f7016aaa03 None admin] RBAC: Authorizing `identity:create_grant(project_id=e556dd7d9fbd4a6e9dbb4191eee92638, group_id=22d8794c67914d558453bf62870e87ed, role_id=8eaff9cd613a49b29548e90d7e3e9215)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:44.090853 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789493f0-ae6b-48bf-8eff-02f7016aaa03 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:44.150617 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:44.168627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eda89a-2837-4bdc-81a8-a00ca8c97dd4 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:44.208852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 202/418] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:28:43 2022] PUT /identity/v3/projects/e556dd7d9fbd4a6e9dbb4191eee92638/groups/22d8794c67914d558453bf62870e87ed/roles/8eaff9cd613a49b29548e90d7e3e9215 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:28:44.216835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc1baa-917b-47fa-a78a-d688e383d998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:44.218323 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc1baa-917b-47fa-a78a-d688e383d998 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:44.218856 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc1baa-917b-47fa-a78a-d688e383d998 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:44.219557 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc1baa-917b-47fa-a78a-d688e383d998 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:44.220537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc1baa-917b-47fa-a78a-d688e383d998 None admin] RBAC: Adding query filter params (name=member) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:44.220909 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc1baa-917b-47fa-a78a-d688e383d998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:44.223083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc1baa-917b-47fa-a78a-d688e383d998 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:44.233799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 217/419] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:44.240705 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789493f0-ae6b-48bf-8eff-02f7016aaa03 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:44.421510 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789493f0-ae6b-48bf-8eff-02f7016aaa03 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:44.489339 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57227be9-ff16-4cba-beed-33a90cf930eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:44.489914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57227be9-ff16-4cba-beed-33a90cf930eb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:44.490211 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57227be9-ff16-4cba-beed-33a90cf930eb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:44.490502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57227be9-ff16-4cba-beed-33a90cf930eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:44.533980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:44.596395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57227be9-ff16-4cba-beed-33a90cf930eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:44.598423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57227be9-ff16-4cba-beed-33a90cf930eb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:44.655528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb2f28-2e8f-4436-91e1-a72ea54e10ef None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:44.656093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb2f28-2e8f-4436-91e1-a72ea54e10ef None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:44.656093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb2f28-2e8f-4436-91e1-a72ea54e10ef None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:44.657915 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 218/420] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:44.663801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520433b8-d57d-4836-92bb-5d8780e0f817 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:44.664090 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520433b8-d57d-4836-92bb-5d8780e0f817 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:44.664380 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520433b8-d57d-4836-92bb-5d8780e0f817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:44.701087 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-520433b8-d57d-4836-92bb-5d8780e0f817 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:44.703279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-520433b8-d57d-4836-92bb-5d8780e0f817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:44.758961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 203/421] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:44.766819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57227be9-ff16-4cba-beed-33a90cf930eb None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:44.790944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 219/422] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:44.817062 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520433b8-d57d-4836-92bb-5d8780e0f817 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:44.837326 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:44.917738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57227be9-ff16-4cba-beed-33a90cf930eb None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:44.938274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:44.973095 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520433b8-d57d-4836-92bb-5d8780e0f817 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:45.001221 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3a6d3-0774-45a5-8317-3bb3e3a44049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:45.002116 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3a6d3-0774-45a5-8317-3bb3e3a44049 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.002403 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3a6d3-0774-45a5-8317-3bb3e3a44049 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.002681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3a6d3-0774-45a5-8317-3bb3e3a44049 None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.004066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3a6d3-0774-45a5-8317-3bb3e3a44049 None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:45.005578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3a6d3-0774-45a5-8317-3bb3e3a44049 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:45.007693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 204/423] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:44 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:45.014612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3a6d3-0774-45a5-8317-3bb3e3a44049 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:45.021122 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118bfe0-4be8-453e-b4f3-9c1177bca168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:45.021972 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118bfe0-4be8-453e-b4f3-9c1177bca168 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.022254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118bfe0-4be8-453e-b4f3-9c1177bca168 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.022528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118bfe0-4be8-453e-b4f3-9c1177bca168 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.049670 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118bfe0-4be8-453e-b4f3-9c1177bca168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-4118bfe0-4be8-453e-b4f3-9c1177bca168 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:45.059396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:45.060616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:45.061940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 220/424] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:45.066605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118bfe0-4be8-453e-b4f3-9c1177bca168 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:45.233463 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118bfe0-4be8-453e-b4f3-9c1177bca168 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:45.235704 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3a6d3-0774-45a5-8317-3bb3e3a44049 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:45.277777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11b232-707e-4051-95f0-fc8fc5521657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:45.278995 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11b232-707e-4051-95f0-fc8fc5521657 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.280652 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11b232-707e-4051-95f0-fc8fc5521657 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.281097 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac11b232-707e-4051-95f0-fc8fc5521657 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.282334 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11b232-707e-4051-95f0-fc8fc5521657 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:45.283985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11b232-707e-4051-95f0-fc8fc5521657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:45.288912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac11b232-707e-4051-95f0-fc8fc5521657 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:45.293984 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:45.301132 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:45.309004 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 221/425] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:45.314913 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11b232-707e-4051-95f0-fc8fc5521657 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:45.328487 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46921d-2cc6-4220-b080-abc310018a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:45.329703 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46921d-2cc6-4220-b080-abc310018a98 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.330148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46921d-2cc6-4220-b080-abc310018a98 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.330581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46921d-2cc6-4220-b080-abc310018a98 None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.333373 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46921d-2cc6-4220-b080-abc310018a98 None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:45.335116 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46921d-2cc6-4220-b080-abc310018a98 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:45.339392 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 205/426] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:45 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:45.346348 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46921d-2cc6-4220-b080-abc310018a98 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:45.505855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46921d-2cc6-4220-b080-abc310018a98 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:45.533810 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac11b232-707e-4051-95f0-fc8fc5521657 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:45.550995 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9d5c4-6165-466a-84aa-99c7c0952622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:45.552124 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9d5c4-6165-466a-84aa-99c7c0952622 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.552649 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9d5c4-6165-466a-84aa-99c7c0952622 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.553152 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9d5c4-6165-466a-84aa-99c7c0952622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.554224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9d5c4-6165-466a-84aa-99c7c0952622 None admin] RBAC: Adding query filter params (group.id=78659048a89146daaf540a2065bb80f0, role.id=c8be6a897df546ca9b0e7b9541fb94d5, scope.project.id=464c75ba6b6340e4b017049e6993fe4d) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:45.555252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9d5c4-6165-466a-84aa-99c7c0952622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:45.557147 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9d5c4-6165-466a-84aa-99c7c0952622 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:45.566583 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 206/427] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:28:45 2022] GET /identity/v3/role_assignments?group.id=78659048a89146daaf540a2065bb80f0&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 611 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:45.568391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cfaba-760b-4b69-be25-92a3eae3cc02 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.568897 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cfaba-760b-4b69-be25-92a3eae3cc02 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.569465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cfaba-760b-4b69-be25-92a3eae3cc02 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.571101 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 207/428] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:45.582909 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aed2db-c019-43af-903e-bfb9f536d5f6 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.583601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aed2db-c019-43af-903e-bfb9f536d5f6 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.584088 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aed2db-c019-43af-903e-bfb9f536d5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.624282 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-02aed2db-c019-43af-903e-bfb9f536d5f6 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:45.626479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b1536-b367-495c-9362-04536fe038e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:45.627466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b1536-b367-495c-9362-04536fe038e2 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.627989 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b1536-b367-495c-9362-04536fe038e2 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.628508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b1536-b367-495c-9362-04536fe038e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:45.631569 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-02aed2db-c019-43af-903e-bfb9f536d5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:45.736854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:45.740124 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 208/429] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:45.780801 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aed2db-c019-43af-903e-bfb9f536d5f6 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:45.826389 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:45.834870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b1536-b367-495c-9362-04536fe038e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:45.838807 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b1536-b367-495c-9362-04536fe038e2 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:45.933867 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aed2db-c019-43af-903e-bfb9f536d5f6 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:45.977048 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5040fd4-8f94-421f-8352-d639fa116d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:45.978081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5040fd4-8f94-421f-8352-d639fa116d05 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:45.978375 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5040fd4-8f94-421f-8352-d639fa116d05 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:45.978693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5040fd4-8f94-421f-8352-d639fa116d05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.000775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5040fd4-8f94-421f-8352-d639fa116d05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-b5040fd4-8f94-421f-8352-d639fa116d05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:46.012309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:46.013677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:28:46.014791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:46.018069 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 209/430] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:46.020685 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 222/431] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 706 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:46.024456 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5040fd4-8f94-421f-8352-d639fa116d05 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:46.028315 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b1536-b367-495c-9362-04536fe038e2 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:46.122471 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:46.174375 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:46.179222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b1536-b367-495c-9362-04536fe038e2 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:46.209731 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5040fd4-8f94-421f-8352-d639fa116d05 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:46.253750 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa492f-0a31-4de8-a8a7-d1d70f60f00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:46.254776 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa492f-0a31-4de8-a8a7-d1d70f60f00f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:46.255216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa492f-0a31-4de8-a8a7-d1d70f60f00f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:46.255502 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa492f-0a31-4de8-a8a7-d1d70f60f00f None admin] PATH_INFO: `/v3/projects/464c75ba6b6340e4b017049e6993fe4d` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.257202 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa492f-0a31-4de8-a8a7-d1d70f60f00f None admin] RBAC: Authorizing `identity:get_project(project_id=464c75ba6b6340e4b017049e6993fe4d)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:46.259538 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfa492f-0a31-4de8-a8a7-d1d70f60f00f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:46.262306 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 223/432] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:46 2022] GET /identity/v3/projects/464c75ba6b6340e4b017049e6993fe4d => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:46.267106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6433fed-8c73-4247-ba0c-428a78fdc9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:46.267581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa492f-0a31-4de8-a8a7-d1d70f60f00f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:46.268402 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6433fed-8c73-4247-ba0c-428a78fdc9f8 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:46.268867 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6433fed-8c73-4247-ba0c-428a78fdc9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:46.269306 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6433fed-8c73-4247-ba0c-428a78fdc9f8 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.270256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6433fed-8c73-4247-ba0c-428a78fdc9f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:46.270740 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6433fed-8c73-4247-ba0c-428a78fdc9f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:46.274154 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6433fed-8c73-4247-ba0c-428a78fdc9f8 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:46.287481 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 210/433] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:28:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:46.293752 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6433fed-8c73-4247-ba0c-428a78fdc9f8 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:46.421776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6433fed-8c73-4247-ba0c-428a78fdc9f8 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:46.424041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfa492f-0a31-4de8-a8a7-d1d70f60f00f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:46.433641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:46.468482 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1662f7-12d4-4eae-a59d-b75eed12c547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:46.468482 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1662f7-12d4-4eae-a59d-b75eed12c547 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:46.468821 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1662f7-12d4-4eae-a59d-b75eed12c547 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:46.468942 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1662f7-12d4-4eae-a59d-b75eed12c547 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.477754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6b00b-9fac-420a-8cb8-117ef3783546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:46.478904 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6b00b-9fac-420a-8cb8-117ef3783546 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:46.479348 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6b00b-9fac-420a-8cb8-117ef3783546 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:46.479782 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6b00b-9fac-420a-8cb8-117ef3783546 None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.481979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6b00b-9fac-420a-8cb8-117ef3783546 None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:46.484093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6b00b-9fac-420a-8cb8-117ef3783546 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:46.486760 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 224/434] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:46 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:46.493335 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6b00b-9fac-420a-8cb8-117ef3783546 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:46.550612 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:46.561264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1662f7-12d4-4eae-a59d-b75eed12c547 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:46.563065 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1662f7-12d4-4eae-a59d-b75eed12c547 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:46.643492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6b00b-9fac-420a-8cb8-117ef3783546 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:46.687941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e02d5-e129-4308-be66-dab21b4bdc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:46.688946 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e02d5-e129-4308-be66-dab21b4bdc29 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:46.689445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e02d5-e129-4308-be66-dab21b4bdc29 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:46.689919 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e02d5-e129-4308-be66-dab21b4bdc29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.690940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e02d5-e129-4308-be66-dab21b4bdc29 None admin] RBAC: Adding query filter params (group.id=78659048a89146daaf540a2065bb80f0, role.id=ff499a8782db4cce8e162a849be212ee, scope.project.id=464c75ba6b6340e4b017049e6993fe4d) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:46.691477 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e02d5-e129-4308-be66-dab21b4bdc29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:46.693136 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e02d5-e129-4308-be66-dab21b4bdc29 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:46.696006 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 211/435] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:46.701500 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1662f7-12d4-4eae-a59d-b75eed12c547 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:46.704300 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 225/436] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:28:46 2022] GET /identity/v3/role_assignments?group.id=78659048a89146daaf540a2065bb80f0&scope.project.id=464c75ba6b6340e4b017049e6993fe4d&role.id=ff499a8782db4cce8e162a849be212ee => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:46.714406 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:46.719059 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afe187-1875-467d-8e83-91036cf0457d None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:46.719495 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afe187-1875-467d-8e83-91036cf0457d None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:46.719906 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49afe187-1875-467d-8e83-91036cf0457d None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.723145 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 226/437] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:28:46.729233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3def39e-aadd-471c-8e41-fef72d5d836b None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:46.729560 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3def39e-aadd-471c-8e41-fef72d5d836b None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:46.730963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3def39e-aadd-471c-8e41-fef72d5d836b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.748545 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-c3def39e-aadd-471c-8e41-fef72d5d836b None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:46.749819 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-c3def39e-aadd-471c-8e41-fef72d5d836b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:46.825010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1662f7-12d4-4eae-a59d-b75eed12c547 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:46.836772 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 227/438] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:46.855043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3def39e-aadd-471c-8e41-fef72d5d836b None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:46.868527 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801b920-e272-4876-b3a8-1e62c1004258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:46.869961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801b920-e272-4876-b3a8-1e62c1004258 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:46.870458 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801b920-e272-4876-b3a8-1e62c1004258 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:46.870927 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801b920-e272-4876-b3a8-1e62c1004258 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:46.872537 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801b920-e272-4876-b3a8-1e62c1004258 None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:46.874289 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801b920-e272-4876-b3a8-1e62c1004258 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:46.876817 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 212/439] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:46 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:46.884193 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801b920-e272-4876-b3a8-1e62c1004258 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:46.899617 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:46.994394 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3def39e-aadd-471c-8e41-fef72d5d836b None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:46.998284 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:47.035189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801b920-e272-4876-b3a8-1e62c1004258 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:47.052039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec5066-f498-482d-b1ec-0d6be9c06e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:47.053719 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec5066-f498-482d-b1ec-0d6be9c06e85 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:47.054009 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec5066-f498-482d-b1ec-0d6be9c06e85 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:47.054287 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aec5066-f498-482d-b1ec-0d6be9c06e85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:47.073710 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aec5066-f498-482d-b1ec-0d6be9c06e85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-6aec5066-f498-482d-b1ec-0d6be9c06e85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:47.084106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:47.085283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:47.086299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 228/440] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:47.089131 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042a99b-ea10-481c-8e6e-c4b5b25fc328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:47.090252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042a99b-ea10-481c-8e6e-c4b5b25fc328 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:47.090360 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042a99b-ea10-481c-8e6e-c4b5b25fc328 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:47.090598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042a99b-ea10-481c-8e6e-c4b5b25fc328 None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:47.091247 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec5066-f498-482d-b1ec-0d6be9c06e85 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:47.093718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042a99b-ea10-481c-8e6e-c4b5b25fc328 None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:47.095408 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042a99b-ea10-481c-8e6e-c4b5b25fc328 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:47.097530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 213/441] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:46 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:47.103898 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042a99b-ea10-481c-8e6e-c4b5b25fc328 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:47.219683 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:47.231133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042a99b-ea10-481c-8e6e-c4b5b25fc328 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:47.236508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aec5066-f498-482d-b1ec-0d6be9c06e85 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:47.271076 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86d0d4-fe80-421c-b8f0-f6d1aa272998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:47.271665 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d0d4-fe80-421c-b8f0-f6d1aa272998 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:47.271944 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d0d4-fe80-421c-b8f0-f6d1aa272998 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:47.272214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86d0d4-fe80-421c-b8f0-f6d1aa272998 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:47.273120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86d0d4-fe80-421c-b8f0-f6d1aa272998 None admin] RBAC: Adding query filter params (group.id=78659048a89146daaf540a2065bb80f0, role.id=c8be6a897df546ca9b0e7b9541fb94d5, scope.project.id=c9a4c742053041558d2d857f27eedf48) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:47.273430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86d0d4-fe80-421c-b8f0-f6d1aa272998 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:47.274851 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b86d0d4-fe80-421c-b8f0-f6d1aa272998 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:47.275914 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c96a2b-bfd5-475a-af94-dbfc690d8c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:47.276728 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c96a2b-bfd5-475a-af94-dbfc690d8c30 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:47.277073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c96a2b-bfd5-475a-af94-dbfc690d8c30 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:47.277414 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c96a2b-bfd5-475a-af94-dbfc690d8c30 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:47.278232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c96a2b-bfd5-475a-af94-dbfc690d8c30 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:47.278605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c96a2b-bfd5-475a-af94-dbfc690d8c30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:47.279985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c96a2b-bfd5-475a-af94-dbfc690d8c30 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:47.280784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 214/442] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:28:47 2022] GET /identity/v3/role_assignments?group.id=78659048a89146daaf540a2065bb80f0&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=c8be6a897df546ca9b0e7b9541fb94d5 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:47.282076 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c665c7-7e04-47c0-b504-2862c5639d28 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:47.282363 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c665c7-7e04-47c0-b504-2862c5639d28 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:47.282639 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c665c7-7e04-47c0-b504-2862c5639d28 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:47.284412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 215/443] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:47.290771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922eecd0-43aa-4687-87cd-ed6465584115 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:47.291139 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922eecd0-43aa-4687-87cd-ed6465584115 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:47.291331 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922eecd0-43aa-4687-87cd-ed6465584115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:47.297791 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 229/444] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:47.299492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c96a2b-bfd5-475a-af94-dbfc690d8c30 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:47.310667 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-922eecd0-43aa-4687-87cd-ed6465584115 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:47.311900 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-922eecd0-43aa-4687-87cd-ed6465584115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:47.314394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:47.402586 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 216/445] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:47.425998 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c96a2b-bfd5-475a-af94-dbfc690d8c30 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:47.426994 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922eecd0-43aa-4687-87cd-ed6465584115 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:47.470762 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99a93e-fc57-420a-9b8e-a7e1e7506349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:47.471437 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99a93e-fc57-420a-9b8e-a7e1e7506349 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:47.471738 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99a93e-fc57-420a-9b8e-a7e1e7506349 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:47.471997 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99a93e-fc57-420a-9b8e-a7e1e7506349 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:47.495610 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:47.551074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922eecd0-43aa-4687-87cd-ed6465584115 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:47.556787 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99a93e-fc57-420a-9b8e-a7e1e7506349 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:47.558972 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99a93e-fc57-420a-9b8e-a7e1e7506349 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:47.598351 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bc337-855b-4ce9-88db-228e1656508f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:47.599571 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bc337-855b-4ce9-88db-228e1656508f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:47.599858 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bc337-855b-4ce9-88db-228e1656508f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:47.600164 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bc337-855b-4ce9-88db-228e1656508f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:47.608644 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:47.614506 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bc337-855b-4ce9-88db-228e1656508f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-e15bc337-855b-4ce9-88db-228e1656508f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:47.621999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:47.626259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:47.627291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 217/446] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:28:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:47.632640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bc337-855b-4ce9-88db-228e1656508f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:47.689107 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 230/447] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:28:47.695123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99a93e-fc57-420a-9b8e-a7e1e7506349 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:47.747019 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:47.758440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bc337-855b-4ce9-88db-228e1656508f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:47.792749 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83a7ee-89d9-4898-9ec3-38112a1f1ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:47.793436 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83a7ee-89d9-4898-9ec3-38112a1f1ee4 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:47.793716 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83a7ee-89d9-4898-9ec3-38112a1f1ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:47.793991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83a7ee-89d9-4898-9ec3-38112a1f1ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:47.794744 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83a7ee-89d9-4898-9ec3-38112a1f1ee4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:47.795054 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83a7ee-89d9-4898-9ec3-38112a1f1ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:47.796469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83a7ee-89d9-4898-9ec3-38112a1f1ee4 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:47.802847 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99a93e-fc57-420a-9b8e-a7e1e7506349 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:47.805459 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 218/448] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:47.810962 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83a7ee-89d9-4898-9ec3-38112a1f1ee4 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:47.837848 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc59b42-81c2-488d-921e-6ac1ad30ef88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:47.838752 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc59b42-81c2-488d-921e-6ac1ad30ef88 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:47.838998 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc59b42-81c2-488d-921e-6ac1ad30ef88 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:47.839362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc59b42-81c2-488d-921e-6ac1ad30ef88 None admin] PATH_INFO: `/v3/projects/c9a4c742053041558d2d857f27eedf48` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:47.840597 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc59b42-81c2-488d-921e-6ac1ad30ef88 None admin] RBAC: Authorizing `identity:get_project(project_id=c9a4c742053041558d2d857f27eedf48)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:47.842052 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc59b42-81c2-488d-921e-6ac1ad30ef88 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:47.843739 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 231/449] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:47 2022] GET /identity/v3/projects/c9a4c742053041558d2d857f27eedf48 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:47.849792 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc59b42-81c2-488d-921e-6ac1ad30ef88 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:47.873447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:47.915652 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83a7ee-89d9-4898-9ec3-38112a1f1ee4 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:47.950261 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff8d42-ffde-4e91-90d6-34f62b0cd5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:47.950917 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff8d42-ffde-4e91-90d6-34f62b0cd5ba None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:47.951200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff8d42-ffde-4e91-90d6-34f62b0cd5ba None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:47.951474 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff8d42-ffde-4e91-90d6-34f62b0cd5ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:47.952124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc59b42-81c2-488d-921e-6ac1ad30ef88 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:47.971179 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:47.999170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386a1f5-18ff-4a13-9db0-d869afed5fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:48.000069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386a1f5-18ff-4a13-9db0-d869afed5fae None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:48.001953 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386a1f5-18ff-4a13-9db0-d869afed5fae None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:48.002246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386a1f5-18ff-4a13-9db0-d869afed5fae None admin] PATH_INFO: `/v3/groups/78659048a89146daaf540a2065bb80f0` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:48.003506 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386a1f5-18ff-4a13-9db0-d869afed5fae None admin] RBAC: Authorizing `identity:get_group(group_id=78659048a89146daaf540a2065bb80f0)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:48.005641 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386a1f5-18ff-4a13-9db0-d869afed5fae None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:48.006735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 232/450] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:47 2022] GET /identity/v3/groups/78659048a89146daaf540a2065bb80f0 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:48.012723 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386a1f5-18ff-4a13-9db0-d869afed5fae None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:48.022821 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff8d42-ffde-4e91-90d6-34f62b0cd5ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:48.024390 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff8d42-ffde-4e91-90d6-34f62b0cd5ba None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:48.084123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:48.116678 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386a1f5-18ff-4a13-9db0-d869afed5fae None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:48.126861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 219/451] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:28:48.132483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff8d42-ffde-4e91-90d6-34f62b0cd5ba None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:48.151570 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961eb1e9-f74f-42a4-a4fe-c5ec96e77a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:48.152235 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961eb1e9-f74f-42a4-a4fe-c5ec96e77a5c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:48.152549 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961eb1e9-f74f-42a4-a4fe-c5ec96e77a5c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:48.152830 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961eb1e9-f74f-42a4-a4fe-c5ec96e77a5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:48.153668 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961eb1e9-f74f-42a4-a4fe-c5ec96e77a5c None admin] RBAC: Adding query filter params (group.id=78659048a89146daaf540a2065bb80f0, role.id=ff499a8782db4cce8e162a849be212ee, scope.project.id=c9a4c742053041558d2d857f27eedf48) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:48.154012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961eb1e9-f74f-42a4-a4fe-c5ec96e77a5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:48.155482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961eb1e9-f74f-42a4-a4fe-c5ec96e77a5c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:48.162554 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 233/452] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:28:48 2022] GET /identity/v3/role_assignments?group.id=78659048a89146daaf540a2065bb80f0&scope.project.id=c9a4c742053041558d2d857f27eedf48&role.id=ff499a8782db4cce8e162a849be212ee => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:48.240604 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff8d42-ffde-4e91-90d6-34f62b0cd5ba None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:48.281651 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da61556-ee4e-4f50-ad9f-4c4186956f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:48.282759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da61556-ee4e-4f50-ad9f-4c4186956f92 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:48.282885 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da61556-ee4e-4f50-ad9f-4c4186956f92 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:48.283110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da61556-ee4e-4f50-ad9f-4c4186956f92 None admin] PATH_INFO: `/v3/projects/e556dd7d9fbd4a6e9dbb4191eee92638` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:48.284384 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da61556-ee4e-4f50-ad9f-4c4186956f92 None admin] RBAC: Authorizing `identity:get_project(project_id=e556dd7d9fbd4a6e9dbb4191eee92638)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:48.286508 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da61556-ee4e-4f50-ad9f-4c4186956f92 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:48.288274 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 220/453] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:28:48 2022] GET /identity/v3/projects/e556dd7d9fbd4a6e9dbb4191eee92638 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:48.294453 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961eb1e9-f74f-42a4-a4fe-c5ec96e77a5c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:48.349296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:48.404740 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961eb1e9-f74f-42a4-a4fe-c5ec96e77a5c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:48.451011 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ab6bb-1229-4e48-accf-88d373ec4167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:48.451898 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ab6bb-1229-4e48-accf-88d373ec4167 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:48.452183 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ab6bb-1229-4e48-accf-88d373ec4167 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:48.453111 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ab6bb-1229-4e48-accf-88d373ec4167 None admin] PATH_INFO: `/v3/groups/22d8794c67914d558453bf62870e87ed` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:48.454035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ab6bb-1229-4e48-accf-88d373ec4167 None admin] RBAC: Authorizing `identity:get_group(group_id=22d8794c67914d558453bf62870e87ed)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:48.455545 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ab6bb-1229-4e48-accf-88d373ec4167 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:48.457881 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 234/454] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:28:48 2022] GET /identity/v3/groups/22d8794c67914d558453bf62870e87ed => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:48.464210 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da61556-ee4e-4f50-ad9f-4c4186956f92 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:48.496506 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:48.580740 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da61556-ee4e-4f50-ad9f-4c4186956f92 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:48.618608 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d861c0-93d6-4419-9abb-bc5fc0566c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:48.619186 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d861c0-93d6-4419-9abb-bc5fc0566c28 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:48.619475 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d861c0-93d6-4419-9abb-bc5fc0566c28 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:48.619743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d861c0-93d6-4419-9abb-bc5fc0566c28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:48.621829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d861c0-93d6-4419-9abb-bc5fc0566c28 None admin] RBAC: Adding query filter params (group.id=22d8794c67914d558453bf62870e87ed, role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=e556dd7d9fbd4a6e9dbb4191eee92638) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:48.621829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d861c0-93d6-4419-9abb-bc5fc0566c28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:48.622414 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d861c0-93d6-4419-9abb-bc5fc0566c28 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:48.629002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 221/455] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:28:48 2022] GET /identity/v3/role_assignments?group.id=22d8794c67914d558453bf62870e87ed&scope.project.id=e556dd7d9fbd4a6e9dbb4191eee92638&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:51.507270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4a550-f2f9-489e-a97d-1eda87b8cbaa None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:51.507802 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4a550-f2f9-489e-a97d-1eda87b8cbaa None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:51.507802 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4a550-f2f9-489e-a97d-1eda87b8cbaa None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:51.509745 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 235/456] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:51.515752 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d7ec8-a769-418d-929e-6577ceedbcd2 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:51.516318 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d7ec8-a769-418d-929e-6577ceedbcd2 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:51.516523 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623d7ec8-a769-418d-929e-6577ceedbcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:51.539287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-623d7ec8-a769-418d-929e-6577ceedbcd2 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:51.540521 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-623d7ec8-a769-418d-929e-6577ceedbcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:51.620496 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 222/457] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:51.669980 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4a550-f2f9-489e-a97d-1eda87b8cbaa None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:51.775742 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e419be4-3393-4706-ba24-976a50342b24 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:51.777110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e419be4-3393-4706-ba24-976a50342b24 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:51.777603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e419be4-3393-4706-ba24-976a50342b24 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:51.780250 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 223/458] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:28:51.790839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89f0b-78c4-4ca0-868e-212316139bd0 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:51.791679 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89f0b-78c4-4ca0-868e-212316139bd0 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:51.792530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89f0b-78c4-4ca0-868e-212316139bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:51.830441 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-79f89f0b-78c4-4ca0-868e-212316139bd0 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:51.833729 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-79f89f0b-78c4-4ca0-868e-212316139bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:51.849979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4a550-f2f9-489e-a97d-1eda87b8cbaa None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:51.858745 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:51.869278 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:51.889432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e1eeb-39df-4cf6-a624-4bf3a8068d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:51.890765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e1eeb-39df-4cf6-a624-4bf3a8068d11 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:51.891287 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e1eeb-39df-4cf6-a624-4bf3a8068d11 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:51.891759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e1eeb-39df-4cf6-a624-4bf3a8068d11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:51.904879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e1eeb-39df-4cf6-a624-4bf3a8068d11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:51.907017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e1eeb-39df-4cf6-a624-4bf3a8068d11 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:51.920915 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 224/459] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:51.922952 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1b913-d09e-4c09-8f02-7f86f1151c2f None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:51.923599 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1b913-d09e-4c09-8f02-7f86f1151c2f None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:51.924110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1b913-d09e-4c09-8f02-7f86f1151c2f None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:51.931161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 225/460] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:51.933773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d332b-b88a-4acc-b762-718abc18704c None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:51.934380 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d332b-b88a-4acc-b762-718abc18704c None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:51.934904 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d332b-b88a-4acc-b762-718abc18704c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-8a4e1eeb-39df-4cf6-a624-4bf3a8068d11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:51.940547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:51.942198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:51.948603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:51.948603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:51.948603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:51.948603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:51.948603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 236/461] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:51.953450 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e1eeb-39df-4cf6-a624-4bf3a8068d11 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:51.972119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-f98d332b-b88a-4acc-b762-718abc18704c None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:51.973429 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-f98d332b-b88a-4acc-b762-718abc18704c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:52.072472 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 226/462] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:52.073842 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d332b-b88a-4acc-b762-718abc18704c None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:52.107520 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e1eeb-39df-4cf6-a624-4bf3a8068d11 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:52.151561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e47db-7083-48c3-8d23-18073c4a803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:52.152564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e47db-7083-48c3-8d23-18073c4a803e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:52.152868 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e47db-7083-48c3-8d23-18073c4a803e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:52.153107 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e47db-7083-48c3-8d23-18073c4a803e None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:52.153841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e47db-7083-48c3-8d23-18073c4a803e None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:52.154147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e47db-7083-48c3-8d23-18073c4a803e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:52.155525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e47db-7083-48c3-8d23-18073c4a803e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:52.171541 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 237/463] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:52.173448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e47db-7083-48c3-8d23-18073c4a803e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:52.194310 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:52.212526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d332b-b88a-4acc-b762-718abc18704c None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:52.228726 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:52.250050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2eee7-0888-40f6-b4d1-743a81ab356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:52.250861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2eee7-0888-40f6-b4d1-743a81ab356a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:52.251150 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2eee7-0888-40f6-b4d1-743a81ab356a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:52.251465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2eee7-0888-40f6-b4d1-743a81ab356a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:52.264790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2eee7-0888-40f6-b4d1-743a81ab356a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:52.267427 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2eee7-0888-40f6-b4d1-743a81ab356a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-dfa2eee7-0888-40f6-b4d1-743a81ab356a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:52.289179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:52.290581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:52.291611 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:52.291611 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:52.291611 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:52.291611 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:52.291611 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 227/464] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:52.291611 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2667c09-cf67-4912-9b94-6b148102f01f None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:52.292258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2667c09-cf67-4912-9b94-6b148102f01f None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:52.292350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2667c09-cf67-4912-9b94-6b148102f01f None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:52.293767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 228/465] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:52.294465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2667c09-cf67-4912-9b94-6b148102f01f None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:52.343477 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e47db-7083-48c3-8d23-18073c4a803e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:52.383087 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9741ce4-7652-4d84-ac76-9a2980768528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:52.383891 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9741ce4-7652-4d84-ac76-9a2980768528 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:52.384193 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9741ce4-7652-4d84-ac76-9a2980768528 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:52.384500 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9741ce4-7652-4d84-ac76-9a2980768528 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:52.395642 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9741ce4-7652-4d84-ac76-9a2980768528 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:52.397174 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9741ce4-7652-4d84-ac76-9a2980768528 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:52.411221 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2667c09-cf67-4912-9b94-6b148102f01f None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-d9741ce4-7652-4d84-ac76-9a2980768528 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:52.416556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:52.417786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:52.423853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:52.423853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:52.423853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:52.423853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:52.423853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 238/466] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:52.423853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7400a23-8a02-47f1-905d-8149806a8e4c None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:52.423853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7400a23-8a02-47f1-905d-8149806a8e4c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:52.423853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7400a23-8a02-47f1-905d-8149806a8e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:52.439628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-d7400a23-8a02-47f1-905d-8149806a8e4c None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:52.441179 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-d7400a23-8a02-47f1-905d-8149806a8e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:52.452102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f7d50-cd78-4dac-879c-05f0435d83de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:52.452904 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f7d50-cd78-4dac-879c-05f0435d83de None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:52.453259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f7d50-cd78-4dac-879c-05f0435d83de None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:52.453597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f7d50-cd78-4dac-879c-05f0435d83de None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:52.454611 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f7d50-cd78-4dac-879c-05f0435d83de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:52.456099 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f7d50-cd78-4dac-879c-05f0435d83de None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:52.463670 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:52.517843 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 239/467] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:52.519810 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7400a23-8a02-47f1-905d-8149806a8e4c None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:52.535535 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:52.624902 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7400a23-8a02-47f1-905d-8149806a8e4c None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:52.648877 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 229/468] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:28:52 2022] POST /identity/v3/users => generated 246 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:52.649737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f7d50-cd78-4dac-879c-05f0435d83de None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:52.676658 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643b1f8-243e-478d-84c1-ba562269baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:52.677618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643b1f8-243e-478d-84c1-ba562269baf5 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:52.678763 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643b1f8-243e-478d-84c1-ba562269baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:52.679247 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643b1f8-243e-478d-84c1-ba562269baf5 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:52.680746 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643b1f8-243e-478d-84c1-ba562269baf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:52.681250 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643b1f8-243e-478d-84c1-ba562269baf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:52.683909 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643b1f8-243e-478d-84c1-ba562269baf5 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:52.702259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 240/469] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:52.703570 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643b1f8-243e-478d-84c1-ba562269baf5 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:52.804231 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f7d50-cd78-4dac-879c-05f0435d83de None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:52.820672 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:52.848689 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef8c22-4fd7-42a1-a497-bf99ce3ce75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:52.849287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef8c22-4fd7-42a1-a497-bf99ce3ce75c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:52.849576 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef8c22-4fd7-42a1-a497-bf99ce3ce75c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:52.849859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef8c22-4fd7-42a1-a497-bf99ce3ce75c None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:52.850615 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef8c22-4fd7-42a1-a497-bf99ce3ce75c None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:52.851109 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef8c22-4fd7-42a1-a497-bf99ce3ce75c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:52.852748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef8c22-4fd7-42a1-a497-bf99ce3ce75c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:52.864785 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643b1f8-243e-478d-84c1-ba562269baf5 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:52.867514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 230/470] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:52.868390 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef8c22-4fd7-42a1-a497-bf99ce3ce75c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:52.909907 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b301e-1db0-4562-89f3-6d659cec2800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:52.911213 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b301e-1db0-4562-89f3-6d659cec2800 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:52.912512 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b301e-1db0-4562-89f3-6d659cec2800 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:52.913006 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b301e-1db0-4562-89f3-6d659cec2800 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:52.926160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b301e-1db0-4562-89f3-6d659cec2800 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:52.928957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b301e-1db0-4562-89f3-6d659cec2800 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:52.932387 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c60b301e-1db0-4562-89f3-6d659cec2800 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:52.948010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:52.949491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:52.951025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:52.951025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:52.951025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:52.951025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:52.951025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 241/471] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:52.951025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b301e-1db0-4562-89f3-6d659cec2800 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:53.014960 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef8c22-4fd7-42a1-a497-bf99ce3ce75c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:53.054929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a454ab-e4df-414f-be06-e1f4e6f7f4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:53.055622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a454ab-e4df-414f-be06-e1f4e6f7f4c0 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:53.055912 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a454ab-e4df-414f-be06-e1f4e6f7f4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:53.056203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a454ab-e4df-414f-be06-e1f4e6f7f4c0 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:53.056944 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a454ab-e4df-414f-be06-e1f4e6f7f4c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:53.058389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a454ab-e4df-414f-be06-e1f4e6f7f4c0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:53.073867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b301e-1db0-4562-89f3-6d659cec2800 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:53.098240 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:53.109290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039de5c0-932a-4e37-a4e8-c0304fabee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:53.109976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de5c0-932a-4e37-a4e8-c0304fabee14 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:53.110264 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de5c0-932a-4e37-a4e8-c0304fabee14 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:53.110545 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039de5c0-932a-4e37-a4e8-c0304fabee14 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:53.111260 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039de5c0-932a-4e37-a4e8-c0304fabee14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:53.112781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039de5c0-932a-4e37-a4e8-c0304fabee14 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:53.195388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 231/472] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:28:52 2022] POST /identity/v3/users => generated 248 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:53.196289 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a454ab-e4df-414f-be06-e1f4e6f7f4c0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:53.266697 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 242/473] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:28:52 2022] POST /identity/v3/users => generated 248 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:53.268728 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b7116-30fd-42cc-9eba-3629e6a2b490 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:53.269483 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b7116-30fd-42cc-9eba-3629e6a2b490 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:53.270008 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b7116-30fd-42cc-9eba-3629e6a2b490 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:53.271934 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 243/474] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:53.278377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddde39-a293-4945-b886-3ff41a206d26 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:53.279247 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddde39-a293-4945-b886-3ff41a206d26 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:53.279993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddde39-a293-4945-b886-3ff41a206d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:53.306553 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-abddde39-a293-4945-b886-3ff41a206d26 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:53.309616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-abddde39-a293-4945-b886-3ff41a206d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:53.405429 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a454ab-e4df-414f-be06-e1f4e6f7f4c0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:53.417783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:53.449002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 244/475] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:53.468837 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3701e4e-0721-4c88-860f-c5f47f7ccbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:53.469938 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3701e4e-0721-4c88-860f-c5f47f7ccbea None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:53.470455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3701e4e-0721-4c88-860f-c5f47f7ccbea None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:53.470953 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3701e4e-0721-4c88-860f-c5f47f7ccbea None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:53.471989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3701e4e-0721-4c88-860f-c5f47f7ccbea None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:53.472553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3701e4e-0721-4c88-860f-c5f47f7ccbea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:53.479354 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3701e4e-0721-4c88-860f-c5f47f7ccbea None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:53.479555 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddde39-a293-4945-b886-3ff41a206d26 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:53.490240 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 232/476] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:53.497621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3701e4e-0721-4c88-860f-c5f47f7ccbea None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:53.610899 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddde39-a293-4945-b886-3ff41a206d26 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:53.623718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:53.634825 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3701e4e-0721-4c88-860f-c5f47f7ccbea None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:53.650865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c6291-9ab9-4c88-9e5b-f46740422872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:53.651813 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c6291-9ab9-4c88-9e5b-f46740422872 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:53.652203 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c6291-9ab9-4c88-9e5b-f46740422872 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:53.652402 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c6291-9ab9-4c88-9e5b-f46740422872 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:53.653309 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c6291-9ab9-4c88-9e5b-f46740422872 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:53.655278 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c6291-9ab9-4c88-9e5b-f46740422872 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:53.683847 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483126b-96d5-417b-980e-638ef6655015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:53.684575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483126b-96d5-417b-980e-638ef6655015 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:53.684841 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483126b-96d5-417b-980e-638ef6655015 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:53.685117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483126b-96d5-417b-980e-638ef6655015 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:53.685832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483126b-96d5-417b-980e-638ef6655015 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:53.687303 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483126b-96d5-417b-980e-638ef6655015 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:53.705343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.sql.core [None req-611c6291-9ab9-4c88-9e5b-f46740422872 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:28:53.705343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:28:53.705343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [parameters: {'id': 'a22d57cbd31a4038a446efd5e38ce3b9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:28:53.705343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=208582) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-611c6291-9ab9-4c88-9e5b-f46740422872 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application conn.query(q) May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application result.read() May 19 20:28:53.723749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return ref.to_dict() May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:28:53.724903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application next(self.gen) May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application yield resource May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application next(self.gen) May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise self.value May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application session.commit() May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:28:53.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self.session.flush() May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application compat.raise_( May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:28:53.727112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:28:53.728159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application conn.query(q) May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application result.read() May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:28:53.731030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [parameters: {'id': 'a22d57cbd31a4038a446efd5e38ce3b9', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:53.733491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:28:53.735724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:53.738947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 245/477] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:28:53 2022] POST /identity/v3/roles => generated 144 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:53.738947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c6291-9ab9-4c88-9e5b-f46740422872 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:53.779765 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 233/478] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:28:53 2022] POST /identity/v3/users => generated 249 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:53.869302 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c6291-9ab9-4c88-9e5b-f46740422872 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:53.915862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ad68d-d2b6-417c-8433-c8534777cd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:53.916810 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad68d-d2b6-417c-8433-c8534777cd87 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:53.917410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad68d-d2b6-417c-8433-c8534777cd87 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:53.917750 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad68d-d2b6-417c-8433-c8534777cd87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:53.930784 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ad68d-d2b6-417c-8433-c8534777cd87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-030ad68d-d2b6-417c-8433-c8534777cd87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:53.939457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:53.940862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:28:53.942044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:53.945720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 246/479] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:28:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:53.950206 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483126b-96d5-417b-980e-638ef6655015 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:53.979592 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:54.082952 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483126b-96d5-417b-980e-638ef6655015 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:54.128516 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a193a45-4237-4b6c-96c5-c969ba1f8294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:54.129421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193a45-4237-4b6c-96c5-c969ba1f8294 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:54.129807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193a45-4237-4b6c-96c5-c969ba1f8294 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:54.130183 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a193a45-4237-4b6c-96c5-c969ba1f8294 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:54.131063 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a193a45-4237-4b6c-96c5-c969ba1f8294 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:54.131470 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a193a45-4237-4b6c-96c5-c969ba1f8294 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:54.133028 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a193a45-4237-4b6c-96c5-c969ba1f8294 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:54.143540 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 234/480] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:55.317857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be41d50-608a-46fb-ae86-b414e1aa2e10 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:55.318426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be41d50-608a-46fb-ae86-b414e1aa2e10 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:55.318426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be41d50-608a-46fb-ae86-b414e1aa2e10 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:55.319841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 247/481] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:55.334926 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced4590-abac-45db-9038-2681aa893c4a None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:55.335479 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced4590-abac-45db-9038-2681aa893c4a None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:55.335479 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced4590-abac-45db-9038-2681aa893c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:55.361401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-eced4590-abac-45db-9038-2681aa893c4a None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:55.363221 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-eced4590-abac-45db-9038-2681aa893c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:55.463139 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 235/482] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:55.511773 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be41d50-608a-46fb-ae86-b414e1aa2e10 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:55.559860 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:55.654544 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be41d50-608a-46fb-ae86-b414e1aa2e10 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:55.706432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149caf8-ca68-4e27-b8a5-a9c2c9232e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:55.707255 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149caf8-ca68-4e27-b8a5-a9c2c9232e47 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:55.707537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149caf8-ca68-4e27-b8a5-a9c2c9232e47 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:55.707811 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149caf8-ca68-4e27-b8a5-a9c2c9232e47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:55.726413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149caf8-ca68-4e27-b8a5-a9c2c9232e47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c149caf8-ca68-4e27-b8a5-a9c2c9232e47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:55.734778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:55.735871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:55.736835 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 248/483] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:28:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:55.742724 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced4590-abac-45db-9038-2681aa893c4a None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:55.777938 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:55.899726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced4590-abac-45db-9038-2681aa893c4a None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:55.946970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b3656-ed94-4ffe-a5f7-1a9294072e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:55.948066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b3656-ed94-4ffe-a5f7-1a9294072e8b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:55.948598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b3656-ed94-4ffe-a5f7-1a9294072e8b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:55.949103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b3656-ed94-4ffe-a5f7-1a9294072e8b None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:55.950101 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b3656-ed94-4ffe-a5f7-1a9294072e8b None admin] RBAC: Adding query filter params (name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:55.950638 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b3656-ed94-4ffe-a5f7-1a9294072e8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:55.952256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b3656-ed94-4ffe-a5f7-1a9294072e8b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:55.965046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 236/484] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:28:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:55.970833 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149caf8-ca68-4e27-b8a5-a9c2c9232e47 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:56.136929 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:56.143543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149caf8-ca68-4e27-b8a5-a9c2c9232e47 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:56.193990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27977216-0d84-4661-bb34-417720b4f398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:56.194917 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977216-0d84-4661-bb34-417720b4f398 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:56.195196 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977216-0d84-4661-bb34-417720b4f398 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:56.195471 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27977216-0d84-4661-bb34-417720b4f398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:56.213540 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27977216-0d84-4661-bb34-417720b4f398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:56.215027 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27977216-0d84-4661-bb34-417720b4f398 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:56.218354 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368526d5-8c09-4997-b43e-bf7c12f32f9a None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:56.218648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368526d5-8c09-4997-b43e-bf7c12f32f9a None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:56.218928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368526d5-8c09-4997-b43e-bf7c12f32f9a None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:56.221830 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 237/485] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:56.230226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa4652-779c-4ef9-b91a-9781797bf7d0 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:56.230515 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa4652-779c-4ef9-b91a-9781797bf7d0 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:56.230858 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa4652-779c-4ef9-b91a-9781797bf7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-27977216-0d84-4661-bb34-417720b4f398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:56.234233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:56.235357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:56.236439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:56.236439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:56.236439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:56.236439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:56.236439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 249/486] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:56.245706 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27977216-0d84-4661-bb34-417720b4f398 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:56.258790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-1cfa4652-779c-4ef9-b91a-9781797bf7d0 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:56.260943 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-1cfa4652-779c-4ef9-b91a-9781797bf7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:56.350789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 238/487] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:56.352504 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690593c-316c-4d5f-814a-472fb91ad625 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:56.352793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690593c-316c-4d5f-814a-472fb91ad625 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:56.353070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690593c-316c-4d5f-814a-472fb91ad625 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:56.354767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 239/488] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:56.360283 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e70063-c316-46c0-8d13-bde1614b25ae None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:56.360610 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e70063-c316-46c0-8d13-bde1614b25ae None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:56.360891 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e70063-c316-46c0-8d13-bde1614b25ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:56.373638 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:56.386132 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-88e70063-c316-46c0-8d13-bde1614b25ae None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:56.387340 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-88e70063-c316-46c0-8d13-bde1614b25ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:56.406422 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27977216-0d84-4661-bb34-417720b4f398 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:56.447560 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c042a33-130d-4410-b9c1-47c6523e527e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:56.448731 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c042a33-130d-4410-b9c1-47c6523e527e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:56.449292 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c042a33-130d-4410-b9c1-47c6523e527e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:56.449821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c042a33-130d-4410-b9c1-47c6523e527e None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:56.450773 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c042a33-130d-4410-b9c1-47c6523e527e None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:56.451301 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c042a33-130d-4410-b9c1-47c6523e527e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:56.453246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c042a33-130d-4410-b9c1-47c6523e527e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:56.470447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 250/490] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:28:56.471167 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 240/490] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:28:56.472025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c042a33-130d-4410-b9c1-47c6523e527e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:56.476983 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e70063-c316-46c0-8d13-bde1614b25ae None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:56.484765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:56.606813 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c042a33-130d-4410-b9c1-47c6523e527e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:56.617025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e70063-c316-46c0-8d13-bde1614b25ae None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:56.644191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07044a-3ae3-4cfd-8686-7094485beb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:56.646138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07044a-3ae3-4cfd-8686-7094485beb1a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:56.646388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07044a-3ae3-4cfd-8686-7094485beb1a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:56.646623 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07044a-3ae3-4cfd-8686-7094485beb1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:56.657039 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be529d0-da44-47ce-98da-3cd26bcd81fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:56.658206 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be529d0-da44-47ce-98da-3cd26bcd81fd None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:56.658445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07044a-3ae3-4cfd-8686-7094485beb1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:56.658913 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be529d0-da44-47ce-98da-3cd26bcd81fd None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:56.659922 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be529d0-da44-47ce-98da-3cd26bcd81fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:56.663234 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-7c07044a-3ae3-4cfd-8686-7094485beb1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:56.668727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:56.669872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:56.670786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:56.671477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be529d0-da44-47ce-98da-3cd26bcd81fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:56.671477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be529d0-da44-47ce-98da-3cd26bcd81fd None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:56.678822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 251/491] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:28:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:56.680065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07044a-3ae3-4cfd-8686-7094485beb1a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-1be529d0-da44-47ce-98da-3cd26bcd81fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:28:56.690826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:56.692142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:56.693219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:56.693219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:56.693219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:28:56.693219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:28:56.693219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:28:56.693219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:28:56.693219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:28:56.693219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:28:56.693219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:28:56.693219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:56.693219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 241/492] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:28:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:28:56.693995 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be529d0-da44-47ce-98da-3cd26bcd81fd None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:56.762747 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:56.808701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07044a-3ae3-4cfd-8686-7094485beb1a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:56.816084 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be529d0-da44-47ce-98da-3cd26bcd81fd None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:56.848263 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbb44b-77e3-4553-8a73-353831a957bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:56.849550 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbb44b-77e3-4553-8a73-353831a957bb None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:56.850086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbb44b-77e3-4553-8a73-353831a957bb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:56.850558 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbb44b-77e3-4553-8a73-353831a957bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:56.855278 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e450ad7-cb0d-4cf2-be22-3e0916c71956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:56.855954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e450ad7-cb0d-4cf2-be22-3e0916c71956 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:56.856237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e450ad7-cb0d-4cf2-be22-3e0916c71956 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:56.856542 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e450ad7-cb0d-4cf2-be22-3e0916c71956 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:56.857299 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e450ad7-cb0d-4cf2-be22-3e0916c71956 None admin] RBAC: Adding query filter params (name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:56.857606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e450ad7-cb0d-4cf2-be22-3e0916c71956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:56.859318 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e450ad7-cb0d-4cf2-be22-3e0916c71956 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:56.865336 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbb44b-77e3-4553-8a73-353831a957bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:56.874669 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 242/493] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:28:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:56.874669 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e450ad7-cb0d-4cf2-be22-3e0916c71956 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-91cbb44b-77e3-4553-8a73-353831a957bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:56.877279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:56.879296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:56.881099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:56.883776 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 252/494] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:28:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:56.884715 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbb44b-77e3-4553-8a73-353831a957bb None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:56.968032 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:57.016466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbb44b-77e3-4553-8a73-353831a957bb None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:57.027933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e450ad7-cb0d-4cf2-be22-3e0916c71956 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:57.048290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:57.058558 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e54a8-4285-4c16-b718-c29873465f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:57.060486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e54a8-4285-4c16-b718-c29873465f16 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.060486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e54a8-4285-4c16-b718-c29873465f16 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.060486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e54a8-4285-4c16-b718-c29873465f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.070077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89166aa-bb39-4bfa-9e10-b7de912fa848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:57.071063 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89166aa-bb39-4bfa-9e10-b7de912fa848 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.071311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e54a8-4285-4c16-b718-c29873465f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:57.071799 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89166aa-bb39-4bfa-9e10-b7de912fa848 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.072394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89166aa-bb39-4bfa-9e10-b7de912fa848 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.072904 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e54a8-4285-4c16-b718-c29873465f16 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:57.073657 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89166aa-bb39-4bfa-9e10-b7de912fa848 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:57.074157 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89166aa-bb39-4bfa-9e10-b7de912fa848 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:57.075927 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89166aa-bb39-4bfa-9e10-b7de912fa848 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-0f9e54a8-4285-4c16-b718-c29873465f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:57.092832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:57.093894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:57.094940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:57.094940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:57.094940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:57.094940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:57.094940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 253/495] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:57.094940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e54a8-4285-4c16-b718-c29873465f16 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:57.123104 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 243/496] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:28:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:57.124877 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c4017-681c-445d-b1eb-0ffcada81851 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.125232 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c4017-681c-445d-b1eb-0ffcada81851 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.125581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c4017-681c-445d-b1eb-0ffcada81851 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.128371 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 244/497] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:57.128371 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c4017-681c-445d-b1eb-0ffcada81851 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:57.215238 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e54a8-4285-4c16-b718-c29873465f16 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:57.241056 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c4017-681c-445d-b1eb-0ffcada81851 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:57.254374 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84e228-8106-49b6-8de8-31e750f1b229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:57.257447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84e228-8106-49b6-8de8-31e750f1b229 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.257447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84e228-8106-49b6-8de8-31e750f1b229 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.257447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84e228-8106-49b6-8de8-31e750f1b229 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.257447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84e228-8106-49b6-8de8-31e750f1b229 None admin] RBAC: Adding query filter params (name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:57.257447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84e228-8106-49b6-8de8-31e750f1b229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:57.258234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84e228-8106-49b6-8de8-31e750f1b229 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:57.261333 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:57.276046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 254/498] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:28:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:57.276046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84e228-8106-49b6-8de8-31e750f1b229 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:57.283258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea68c8-8fc0-430a-82c7-a669f8bfcda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:57.283942 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea68c8-8fc0-430a-82c7-a669f8bfcda1 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.284275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea68c8-8fc0-430a-82c7-a669f8bfcda1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.285566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea68c8-8fc0-430a-82c7-a669f8bfcda1 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.286431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea68c8-8fc0-430a-82c7-a669f8bfcda1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:57.286745 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea68c8-8fc0-430a-82c7-a669f8bfcda1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:57.288142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea68c8-8fc0-430a-82c7-a669f8bfcda1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:57.301377 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 245/499] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:57.302643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d1505-e62d-4587-b39e-92a4f61ba78a None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.303044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d1505-e62d-4587-b39e-92a4f61ba78a None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.303327 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d1505-e62d-4587-b39e-92a4f61ba78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.318327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:57.321994 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-e20d1505-e62d-4587-b39e-92a4f61ba78a None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:57.323211 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-e20d1505-e62d-4587-b39e-92a4f61ba78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:57.377336 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84e228-8106-49b6-8de8-31e750f1b229 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:57.392187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 246/500] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:57.393637 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41f99-fe11-4016-9874-e59278ce8a60 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.394480 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41f99-fe11-4016-9874-e59278ce8a60 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.394761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc41f99-fe11-4016-9874-e59278ce8a60 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.396082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 247/501] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:28:57.396810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc41f99-fe11-4016-9874-e59278ce8a60 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:57.420752 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b647600-40ca-4b4f-8c68-e61866e6170b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:57.421451 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b647600-40ca-4b4f-8c68-e61866e6170b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.421682 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b647600-40ca-4b4f-8c68-e61866e6170b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.421957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b647600-40ca-4b4f-8c68-e61866e6170b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.498925 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b647600-40ca-4b4f-8c68-e61866e6170b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:57.500431 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b647600-40ca-4b4f-8c68-e61866e6170b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:57.517122 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc41f99-fe11-4016-9874-e59278ce8a60 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:57.524935 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:57.553469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f3f43-ab1f-4360-a3b3-8c36e5fd7412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:57.554275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f3f43-ab1f-4360-a3b3-8c36e5fd7412 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.554561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f3f43-ab1f-4360-a3b3-8c36e5fd7412 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.554835 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f3f43-ab1f-4360-a3b3-8c36e5fd7412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.565508 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f3f43-ab1f-4360-a3b3-8c36e5fd7412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:57.567237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f3f43-ab1f-4360-a3b3-8c36e5fd7412 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-7b3f3f43-ab1f-4360-a3b3-8c36e5fd7412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:57.585013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:57.586237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:57.587469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:57.587469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:57.587469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:57.587469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:57.587469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 248/502] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:57.587469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f3f43-ab1f-4360-a3b3-8c36e5fd7412 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:57.601548 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:57.662416 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 255/503] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:57.663742 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f1142-961c-4c97-b076-ae9cd880d8fc None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.664142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f1142-961c-4c97-b076-ae9cd880d8fc None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.664315 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f1142-961c-4c97-b076-ae9cd880d8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.683708 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-9a7f1142-961c-4c97-b076-ae9cd880d8fc None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:28:57.684948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-9a7f1142-961c-4c97-b076-ae9cd880d8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:28:57.701810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f3f43-ab1f-4360-a3b3-8c36e5fd7412 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:57.738483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdb057-4d68-4db9-83e6-61fd6eaa4780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:57.739322 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdb057-4d68-4db9-83e6-61fd6eaa4780 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.739638 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdb057-4d68-4db9-83e6-61fd6eaa4780 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.739847 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cdb057-4d68-4db9-83e6-61fd6eaa4780 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.748061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdb057-4d68-4db9-83e6-61fd6eaa4780 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:57.749558 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cdb057-4d68-4db9-83e6-61fd6eaa4780 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:57.754666 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 256/504] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:28:57.755529 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f1142-961c-4c97-b076-ae9cd880d8fc None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-d3cdb057-4d68-4db9-83e6-61fd6eaa4780 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:28:57.764674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:57.765726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 249/505] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:57.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdb057-4d68-4db9-83e6-61fd6eaa4780 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:57.791581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:57.839494 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:57.872864 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f1142-961c-4c97-b076-ae9cd880d8fc None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:57.876169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cdb057-4d68-4db9-83e6-61fd6eaa4780 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:57.915519 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860c7b5-605c-4411-b945-ec8b5c5a7767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:57.917098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860c7b5-605c-4411-b945-ec8b5c5a7767 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.917572 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860c7b5-605c-4411-b945-ec8b5c5a7767 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.918022 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860c7b5-605c-4411-b945-ec8b5c5a7767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.918312 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87921c11-1d70-44cf-8b97-b809bd098f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:57.919412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87921c11-1d70-44cf-8b97-b809bd098f0b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:57.919866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87921c11-1d70-44cf-8b97-b809bd098f0b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:57.920309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87921c11-1d70-44cf-8b97-b809bd098f0b None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:57.921590 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87921c11-1d70-44cf-8b97-b809bd098f0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:57.922107 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87921c11-1d70-44cf-8b97-b809bd098f0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:57.924290 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87921c11-1d70-44cf-8b97-b809bd098f0b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:57.934650 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860c7b5-605c-4411-b945-ec8b5c5a7767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:57.936237 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860c7b5-605c-4411-b945-ec8b5c5a7767 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:57.939966 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 250/506] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:57.941437 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87921c11-1d70-44cf-8b97-b809bd098f0b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-5860c7b5-605c-4411-b945-ec8b5c5a7767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:57.956479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:57.957585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:57.963707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:57.963707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:57.963707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:57.963707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:57.963707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 257/507] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:57.963707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860c7b5-605c-4411-b945-ec8b5c5a7767 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.025932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:58.048262 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87921c11-1d70-44cf-8b97-b809bd098f0b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.063819 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860c7b5-605c-4411-b945-ec8b5c5a7767 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.083259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f01e98-9ec0-4a7c-9b3f-315039b094ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.084048 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f01e98-9ec0-4a7c-9b3f-315039b094ea None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.084326 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f01e98-9ec0-4a7c-9b3f-315039b094ea None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.084622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f01e98-9ec0-4a7c-9b3f-315039b094ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.086829 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:58.095518 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f01e98-9ec0-4a7c-9b3f-315039b094ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.097638 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f01e98-9ec0-4a7c-9b3f-315039b094ea None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:58.098989 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e6931-4366-4833-a662-3df221669f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.099659 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6931-4366-4833-a662-3df221669f6b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.099938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6931-4366-4833-a662-3df221669f6b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.100216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e6931-4366-4833-a662-3df221669f6b None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.101092 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e6931-4366-4833-a662-3df221669f6b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:58.106165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e6931-4366-4833-a662-3df221669f6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.106165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e6931-4366-4833-a662-3df221669f6b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-e2f01e98-9ec0-4a7c-9b3f-315039b094ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:58.116080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:58.117389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:58.119562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:58.119562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:58.119562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:58.119562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:58.119562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 251/508] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:58.119562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f01e98-9ec0-4a7c-9b3f-315039b094ea None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.149783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 258/509] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:28:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:58.150692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e6931-4366-4833-a662-3df221669f6b None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.246907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f01e98-9ec0-4a7c-9b3f-315039b094ea None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.267495 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e6931-4366-4833-a662-3df221669f6b None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.277579 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:58.283918 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a8ff1-194c-44df-bfa6-d0c6120a1c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.285282 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a8ff1-194c-44df-bfa6-d0c6120a1c3b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.286833 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a8ff1-194c-44df-bfa6-d0c6120a1c3b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.286833 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a8ff1-194c-44df-bfa6-d0c6120a1c3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.298416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852a8ff1-194c-44df-bfa6-d0c6120a1c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:28:58.306457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c611fec-4705-4b75-b6a3-a43254fdb6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-852a8ff1-194c-44df-bfa6-d0c6120a1c3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:28:58.306989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:58.308082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:28:58.309036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:58.309748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c611fec-4705-4b75-b6a3-a43254fdb6d5 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.309748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c611fec-4705-4b75-b6a3-a43254fdb6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.309748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c611fec-4705-4b75-b6a3-a43254fdb6d5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.313797 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 252/510] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:58.314773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a8ff1-194c-44df-bfa6-d0c6120a1c3b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.317102 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c611fec-4705-4b75-b6a3-a43254fdb6d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.318883 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c611fec-4705-4b75-b6a3-a43254fdb6d5 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-7c611fec-4705-4b75-b6a3-a43254fdb6d5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:28:58.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:58.338528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 259/511] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:58.339950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c611fec-4705-4b75-b6a3-a43254fdb6d5 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.390768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:58.429790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a8ff1-194c-44df-bfa6-d0c6120a1c3b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.450524 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c611fec-4705-4b75-b6a3-a43254fdb6d5 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.467666 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e2941-74de-4c36-ad85-0a405a9f8841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.468355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e2941-74de-4c36-ad85-0a405a9f8841 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.468646 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e2941-74de-4c36-ad85-0a405a9f8841 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.468922 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e2941-74de-4c36-ad85-0a405a9f8841 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.469654 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e2941-74de-4c36-ad85-0a405a9f8841 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:58.469968 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e2941-74de-4c36-ad85-0a405a9f8841 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.471330 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e2941-74de-4c36-ad85-0a405a9f8841 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:58.483552 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 253/512] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:58.484280 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e2941-74de-4c36-ad85-0a405a9f8841 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.491244 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6dda7-911d-4048-bb20-268722bdabe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.492071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6dda7-911d-4048-bb20-268722bdabe0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.492376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6dda7-911d-4048-bb20-268722bdabe0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.492660 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6dda7-911d-4048-bb20-268722bdabe0 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.493393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6dda7-911d-4048-bb20-268722bdabe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:58.493701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6dda7-911d-4048-bb20-268722bdabe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.495125 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e6dda7-911d-4048-bb20-268722bdabe0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:58.507279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 260/513] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:58.510477 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6dda7-911d-4048-bb20-268722bdabe0 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.535506 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:58.586023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e2941-74de-4c36-ad85-0a405a9f8841 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.607657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:58.610408 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6dda7-911d-4048-bb20-268722bdabe0 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.621307 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c0a9a-ce35-438e-b784-d0ff420bd671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.621958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c0a9a-ce35-438e-b784-d0ff420bd671 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.622241 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c0a9a-ce35-438e-b784-d0ff420bd671 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.622514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c0a9a-ce35-438e-b784-d0ff420bd671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.646309 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eef3e-c06e-46ca-9b7e-65cbed27abc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.646890 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eef3e-c06e-46ca-9b7e-65cbed27abc5 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.647177 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eef3e-c06e-46ca-9b7e-65cbed27abc5 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.647455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2eef3e-c06e-46ca-9b7e-65cbed27abc5 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.648220 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2eef3e-c06e-46ca-9b7e-65cbed27abc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:58.648568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2eef3e-c06e-46ca-9b7e-65cbed27abc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.649962 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2eef3e-c06e-46ca-9b7e-65cbed27abc5 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:58.658468 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 261/514] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:58.659893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eef3e-c06e-46ca-9b7e-65cbed27abc5 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.698971 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c0a9a-ce35-438e-b784-d0ff420bd671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.700459 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c0a9a-ce35-438e-b784-d0ff420bd671 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:58.751851 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:58.760923 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2eef3e-c06e-46ca-9b7e-65cbed27abc5 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.796799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ade3d-89dc-4021-9b00-9c7ae41e9450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.797475 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ade3d-89dc-4021-9b00-9c7ae41e9450 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.797759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ade3d-89dc-4021-9b00-9c7ae41e9450 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.798043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ade3d-89dc-4021-9b00-9c7ae41e9450 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.798883 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ade3d-89dc-4021-9b00-9c7ae41e9450 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:58.799353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ade3d-89dc-4021-9b00-9c7ae41e9450 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.800857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ade3d-89dc-4021-9b00-9c7ae41e9450 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:58.804393 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 254/515] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:58.805225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c0a9a-ce35-438e-b784-d0ff420bd671 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.823375 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 262/516] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:58.823375 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ade3d-89dc-4021-9b00-9c7ae41e9450 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:58.843068 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:58.907087 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c0a9a-ce35-438e-b784-d0ff420bd671 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.920957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ade3d-89dc-4021-9b00-9c7ae41e9450 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:58.941126 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30089c29-5e81-408f-9f5b-3fbac6c0e935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.941806 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089c29-5e81-408f-9f5b-3fbac6c0e935 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.942092 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089c29-5e81-408f-9f5b-3fbac6c0e935 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.942372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089c29-5e81-408f-9f5b-3fbac6c0e935 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.943088 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30089c29-5e81-408f-9f5b-3fbac6c0e935 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.944547 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30089c29-5e81-408f-9f5b-3fbac6c0e935 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:58.955770 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6b3d2-30af-4afd-83be-84220eba35d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:58.957202 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6b3d2-30af-4afd-83be-84220eba35d2 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:58.957482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6b3d2-30af-4afd-83be-84220eba35d2 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:58.957755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6b3d2-30af-4afd-83be-84220eba35d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:58.964553 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6b3d2-30af-4afd-83be-84220eba35d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:58.965994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6b3d2-30af-4afd-83be-84220eba35d2 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-20d6b3d2-30af-4afd-83be-84220eba35d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:28:58.980094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:58.981293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:58.982576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:28:58.982576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:28:58.982576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:28:58.982576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:28:58.982576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:28:58.982576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:28:58.982576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:28:58.982576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:58.982576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 263/517] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:58.982576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6b3d2-30af-4afd-83be-84220eba35d2 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.036613 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 255/518] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:28:58 2022] POST /identity/v3/users => generated 247 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:28:59.037548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30089c29-5e81-408f-9f5b-3fbac6c0e935 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.048587 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.068582 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.089751 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6b3d2-30af-4afd-83be-84220eba35d2 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.129939 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ac601-3295-4287-bed4-29d0560f3f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.130758 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ac601-3295-4287-bed4-29d0560f3f5a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.131043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ac601-3295-4287-bed4-29d0560f3f5a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.131319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ac601-3295-4287-bed4-29d0560f3f5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.143629 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ac601-3295-4287-bed4-29d0560f3f5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.145165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ac601-3295-4287-bed4-29d0560f3f5a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.156682 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30089c29-5e81-408f-9f5b-3fbac6c0e935 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-929ac601-3295-4287-bed4-29d0560f3f5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:59.164467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:59.165513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:59.166487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:59.166487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:59.166487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:59.166487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:59.170186 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 264/519] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:59.170186 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ac601-3295-4287-bed4-29d0560f3f5a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.203098 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816cd72-74db-4194-a5a9-05cc25b77a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.204175 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816cd72-74db-4194-a5a9-05cc25b77a05 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.204495 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816cd72-74db-4194-a5a9-05cc25b77a05 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.204783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816cd72-74db-4194-a5a9-05cc25b77a05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.220092 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816cd72-74db-4194-a5a9-05cc25b77a05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.222622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816cd72-74db-4194-a5a9-05cc25b77a05 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-8816cd72-74db-4194-a5a9-05cc25b77a05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:59.241601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:59.242635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:59.243648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:59.243648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:59.243648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:59.243648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:59.243648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 256/520] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:59.243976 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816cd72-74db-4194-a5a9-05cc25b77a05 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.298061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ac601-3295-4287-bed4-29d0560f3f5a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.318482 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.330237 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.336446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372ad35-8d79-43df-96af-63e9d9fb2606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.345616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372ad35-8d79-43df-96af-63e9d9fb2606 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.345616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372ad35-8d79-43df-96af-63e9d9fb2606 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.345616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1372ad35-8d79-43df-96af-63e9d9fb2606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.363038 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816cd72-74db-4194-a5a9-05cc25b77a05 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.398729 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a98fb-1d53-4943-bf65-0ff3a149be1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.399408 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a98fb-1d53-4943-bf65-0ff3a149be1b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.399726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a98fb-1d53-4943-bf65-0ff3a149be1b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.400010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a98fb-1d53-4943-bf65-0ff3a149be1b None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.400796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a98fb-1d53-4943-bf65-0ff3a149be1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:59.401128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a98fb-1d53-4943-bf65-0ff3a149be1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.402689 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a98fb-1d53-4943-bf65-0ff3a149be1b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.413395 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372ad35-8d79-43df-96af-63e9d9fb2606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.415424 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1372ad35-8d79-43df-96af-63e9d9fb2606 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.426424 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 257/521] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:59.426424 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a98fb-1d53-4943-bf65-0ff3a149be1b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.526475 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 265/522] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:28:59.527345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372ad35-8d79-43df-96af-63e9d9fb2606 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.546106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a98fb-1d53-4943-bf65-0ff3a149be1b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.561871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.567906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.586196 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79e392-cca4-4fb9-b1a9-4c7f9fe888d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.589716 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79e392-cca4-4fb9-b1a9-4c7f9fe888d8 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.589716 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79e392-cca4-4fb9-b1a9-4c7f9fe888d8 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.589716 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79e392-cca4-4fb9-b1a9-4c7f9fe888d8 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.589716 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79e392-cca4-4fb9-b1a9-4c7f9fe888d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:59.589716 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79e392-cca4-4fb9-b1a9-4c7f9fe888d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.591773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79e392-cca4-4fb9-b1a9-4c7f9fe888d8 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.602597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 258/523] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:59.605423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79e392-cca4-4fb9-b1a9-4c7f9fe888d8 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.639833 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1372ad35-8d79-43df-96af-63e9d9fb2606 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.677919 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d926475-c1e9-46e5-9c07-f37471e24b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.678517 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d926475-c1e9-46e5-9c07-f37471e24b23 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.678799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d926475-c1e9-46e5-9c07-f37471e24b23 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.679193 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d926475-c1e9-46e5-9c07-f37471e24b23 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.679915 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d926475-c1e9-46e5-9c07-f37471e24b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:59.680281 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d926475-c1e9-46e5-9c07-f37471e24b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.681783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d926475-c1e9-46e5-9c07-f37471e24b23 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.699363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 266/524] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:59.700255 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d926475-c1e9-46e5-9c07-f37471e24b23 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.712218 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79e392-cca4-4fb9-b1a9-4c7f9fe888d8 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.749576 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba96144-54f0-4e23-8ac5-18736af808e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.750241 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba96144-54f0-4e23-8ac5-18736af808e4 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.750522 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba96144-54f0-4e23-8ac5-18736af808e4 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.750794 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba96144-54f0-4e23-8ac5-18736af808e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.751633 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba96144-54f0-4e23-8ac5-18736af808e4 None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=e6a982ae15a6443b9b52529df47aa837) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:28:59.752027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba96144-54f0-4e23-8ac5-18736af808e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.754032 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba96144-54f0-4e23-8ac5-18736af808e4 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.768372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 259/525] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/role_assignments?user.id=e6a982ae15a6443b9b52529df47aa837&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:28:59.768372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba96144-54f0-4e23-8ac5-18736af808e4 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.812750 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.822136 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:28:59.832275 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d926475-c1e9-46e5-9c07-f37471e24b23 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.874069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216de3-66b2-43a3-a290-6bdefb8005ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.875371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216de3-66b2-43a3-a290-6bdefb8005ef None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.875755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216de3-66b2-43a3-a290-6bdefb8005ef None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.876131 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23216de3-66b2-43a3-a290-6bdefb8005ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.893313 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23216de3-66b2-43a3-a290-6bdefb8005ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.896186 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23216de3-66b2-43a3-a290-6bdefb8005ef None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.899086 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba96144-54f0-4e23-8ac5-18736af808e4 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-23216de3-66b2-43a3-a290-6bdefb8005ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:28:59.920985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:59.922033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:59.923374 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:28:59.923374 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:28:59.923374 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:28:59.923374 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:28:59.923374 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 267/526] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:59.924010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216de3-66b2-43a3-a290-6bdefb8005ef None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:28:59.952391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46bda3-f56a-4f0f-80b5-6483a92b5fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:28:59.953546 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46bda3-f56a-4f0f-80b5-6483a92b5fd5 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:28:59.953854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46bda3-f56a-4f0f-80b5-6483a92b5fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:28:59.954132 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46bda3-f56a-4f0f-80b5-6483a92b5fd5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:28:59.966731 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46bda3-f56a-4f0f-80b5-6483a92b5fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:28:59.969667 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46bda3-f56a-4f0f-80b5-6483a92b5fd5 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-ec46bda3-f56a-4f0f-80b5-6483a92b5fd5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:28:59.987997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:28:59.989120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:28:59.990155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:28:59.990155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:28:59.990155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:28:59.990155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:28:59.990155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:28:59.990155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:28:59.990155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:28:59.990155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:28:59.990155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:28:59.990155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:28:59.992716 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 260/527] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:28:59.993418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46bda3-f56a-4f0f-80b5-6483a92b5fd5 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.062149 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23216de3-66b2-43a3-a290-6bdefb8005ef None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.099300 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7302e75-2c6d-4fc4-973e-37256db51452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.100109 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7302e75-2c6d-4fc4-973e-37256db51452 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.100434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7302e75-2c6d-4fc4-973e-37256db51452 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.100717 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7302e75-2c6d-4fc4-973e-37256db51452 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.108889 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7302e75-2c6d-4fc4-973e-37256db51452 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.110527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7302e75-2c6d-4fc4-973e-37256db51452 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-a7302e75-2c6d-4fc4-973e-37256db51452 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:00.126274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:00.128986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:00.131156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:00.131156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:00.131156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:00.131156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:00.131156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:00.131156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:00.131156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:00.131156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:00.132205 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46bda3-f56a-4f0f-80b5-6483a92b5fd5 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.133421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:00.134336 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 268/528] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:00.135564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7302e75-2c6d-4fc4-973e-37256db51452 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.141340 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:00.175962 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a319ca-b41f-4344-9da1-c8032ae59987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.176719 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a319ca-b41f-4344-9da1-c8032ae59987 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.177021 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a319ca-b41f-4344-9da1-c8032ae59987 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.177313 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a319ca-b41f-4344-9da1-c8032ae59987 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.178436 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a319ca-b41f-4344-9da1-c8032ae59987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:00.178754 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a319ca-b41f-4344-9da1-c8032ae59987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.180562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a319ca-b41f-4344-9da1-c8032ae59987 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.196429 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 261/529] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:00.197749 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a319ca-b41f-4344-9da1-c8032ae59987 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.282396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7302e75-2c6d-4fc4-973e-37256db51452 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.336132 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0934d-1c87-4fec-89ba-f850fac5064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.337247 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0934d-1c87-4fec-89ba-f850fac5064f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.339298 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a319ca-b41f-4344-9da1-c8032ae59987 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.339804 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0934d-1c87-4fec-89ba-f850fac5064f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.340228 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0934d-1c87-4fec-89ba-f850fac5064f None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.341569 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0934d-1c87-4fec-89ba-f850fac5064f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:00.341995 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0934d-1c87-4fec-89ba-f850fac5064f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.344589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0934d-1c87-4fec-89ba-f850fac5064f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.374713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 269/530] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:00.374713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0934d-1c87-4fec-89ba-f850fac5064f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.388469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d09979-b0fb-435c-a7a9-9b5c66c5eb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.389676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d09979-b0fb-435c-a7a9-9b5c66c5eb03 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.390116 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d09979-b0fb-435c-a7a9-9b5c66c5eb03 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.390562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d09979-b0fb-435c-a7a9-9b5c66c5eb03 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.391696 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d09979-b0fb-435c-a7a9-9b5c66c5eb03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:00.392375 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d09979-b0fb-435c-a7a9-9b5c66c5eb03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.394311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d09979-b0fb-435c-a7a9-9b5c66c5eb03 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.410989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 262/531] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:00.412286 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d09979-b0fb-435c-a7a9-9b5c66c5eb03 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.443273 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:00.459094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:00.508292 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0934d-1c87-4fec-89ba-f850fac5064f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.540279 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d09979-b0fb-435c-a7a9-9b5c66c5eb03 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.547739 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6553c-b925-450a-b6df-d12ec15c9e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.548581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6553c-b925-450a-b6df-d12ec15c9e06 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.548864 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6553c-b925-450a-b6df-d12ec15c9e06 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.549183 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6553c-b925-450a-b6df-d12ec15c9e06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.556926 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6553c-b925-450a-b6df-d12ec15c9e06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.558496 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6553c-b925-450a-b6df-d12ec15c9e06 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-45e6553c-b925-450a-b6df-d12ec15c9e06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:00.574243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:00.575664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:00.576729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:00.576729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:00.576729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:00.576729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:00.576729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:00.576729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:00.576729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:00.576729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:00.580493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 270/532] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:00.582124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6553c-b925-450a-b6df-d12ec15c9e06 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.582567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dacffa-2e2d-4d5b-809f-d761abf8e5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.583247 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dacffa-2e2d-4d5b-809f-d761abf8e5e9 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.583535 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dacffa-2e2d-4d5b-809f-d761abf8e5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.583811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dacffa-2e2d-4d5b-809f-d761abf8e5e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.661053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dacffa-2e2d-4d5b-809f-d761abf8e5e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.662737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dacffa-2e2d-4d5b-809f-d761abf8e5e9 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.691340 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:00.697774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6553c-b925-450a-b6df-d12ec15c9e06 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.721827 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:00.733522 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68d05d-6be6-4d5d-9085-d6b81c11a9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.734195 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68d05d-6be6-4d5d-9085-d6b81c11a9af None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.734475 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68d05d-6be6-4d5d-9085-d6b81c11a9af None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.734845 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68d05d-6be6-4d5d-9085-d6b81c11a9af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.735634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68d05d-6be6-4d5d-9085-d6b81c11a9af None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=15cb25c883074386a956a0754e12e13f) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:00.736000 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68d05d-6be6-4d5d-9085-d6b81c11a9af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.738160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68d05d-6be6-4d5d-9085-d6b81c11a9af None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.745581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 271/533] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/role_assignments?user.id=15cb25c883074386a956a0754e12e13f&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:00.752169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68d05d-6be6-4d5d-9085-d6b81c11a9af None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.779184 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 263/534] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:29:00.785966 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dacffa-2e2d-4d5b-809f-d761abf8e5e9 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.903544 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68d05d-6be6-4d5d-9085-d6b81c11a9af None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.910636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dacffa-2e2d-4d5b-809f-d761abf8e5e9 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:00.937881 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:00.952239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd51bb-b16a-4ea0-b9da-2318dd873aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.952959 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd51bb-b16a-4ea0-b9da-2318dd873aef None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.953240 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd51bb-b16a-4ea0-b9da-2318dd873aef None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.953517 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd51bb-b16a-4ea0-b9da-2318dd873aef None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.953752 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63d57e-8070-427e-ae01-09b88532fb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:00.954321 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd51bb-b16a-4ea0-b9da-2318dd873aef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:00.954629 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd51bb-b16a-4ea0-b9da-2318dd873aef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.955289 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63d57e-8070-427e-ae01-09b88532fb9d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:00.955782 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63d57e-8070-427e-ae01-09b88532fb9d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:00.956121 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd51bb-b16a-4ea0-b9da-2318dd873aef None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.956550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63d57e-8070-427e-ae01-09b88532fb9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:00.969031 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63d57e-8070-427e-ae01-09b88532fb9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:00.970884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63d57e-8070-427e-ae01-09b88532fb9d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:00.980728 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 272/535] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:00.982654 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd51bb-b16a-4ea0-b9da-2318dd873aef None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-1b63d57e-8070-427e-ae01-09b88532fb9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:00.992268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:00.993368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:00.994357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:00.994357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:00.994357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:00.994357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:00.997212 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 264/536] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:01.008761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63d57e-8070-427e-ae01-09b88532fb9d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:01.044809 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:01.103981 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd51bb-b16a-4ea0-b9da-2318dd873aef None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:01.144353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fecc38-30dc-4e80-8408-cf5d4ae4515a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:01.150189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fecc38-30dc-4e80-8408-cf5d4ae4515a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:01.150189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fecc38-30dc-4e80-8408-cf5d4ae4515a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:01.150189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fecc38-30dc-4e80-8408-cf5d4ae4515a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:01.150189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fecc38-30dc-4e80-8408-cf5d4ae4515a None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=c6ace7ef9bfe4837b87c8576862abb75) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:01.150189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fecc38-30dc-4e80-8408-cf5d4ae4515a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:01.150660 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fecc38-30dc-4e80-8408-cf5d4ae4515a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:01.156174 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 273/537] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:00 2022] GET /identity/v3/role_assignments?user.id=c6ace7ef9bfe4837b87c8576862abb75&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:01.239579 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63d57e-8070-427e-ae01-09b88532fb9d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:01.299085 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce44cfd-9078-4678-bc6e-f4765a753ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:01.300775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce44cfd-9078-4678-bc6e-f4765a753ca6 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:01.301303 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce44cfd-9078-4678-bc6e-f4765a753ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:01.301817 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce44cfd-9078-4678-bc6e-f4765a753ca6 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:01.302780 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce44cfd-9078-4678-bc6e-f4765a753ca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:01.303319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce44cfd-9078-4678-bc6e-f4765a753ca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:01.304997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce44cfd-9078-4678-bc6e-f4765a753ca6 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:01.324499 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 265/538] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:01.330122 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fecc38-30dc-4e80-8408-cf5d4ae4515a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:01.460358 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fecc38-30dc-4e80-8408-cf5d4ae4515a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:01.462437 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe49b5-6a92-4572-a609-ad83d799cb23 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:01.462739 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe49b5-6a92-4572-a609-ad83d799cb23 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:01.463129 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe49b5-6a92-4572-a609-ad83d799cb23 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:01.464717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 266/539] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:01.470497 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c3332-75b8-406c-b973-3bc0362151a5 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:01.471201 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c3332-75b8-406c-b973-3bc0362151a5 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:01.471201 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c3332-75b8-406c-b973-3bc0362151a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:01.480544 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:01.507510 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-ca7c3332-75b8-406c-b973-3bc0362151a5 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:01.510915 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-ca7c3332-75b8-406c-b973-3bc0362151a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:01.549155 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f4f1-37e0-4f7e-bccf-ace6f283930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:01.550070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f4f1-37e0-4f7e-bccf-ace6f283930f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:01.550359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f4f1-37e0-4f7e-bccf-ace6f283930f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:01.550640 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35f4f1-37e0-4f7e-bccf-ace6f283930f None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:01.557023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:01.565041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35f4f1-37e0-4f7e-bccf-ace6f283930f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:01.569924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e35f4f1-37e0-4f7e-bccf-ace6f283930f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-7e35f4f1-37e0-4f7e-bccf-ace6f283930f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:01.599482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:01.600566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:01.601575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:01.601575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:01.601575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:01.601575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:01.601575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:01.601575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:01.601575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:01.601575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:01.602048 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 274/540] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:01.607225 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f4f1-37e0-4f7e-bccf-ace6f283930f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:01.632368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 267/541] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:29:01.679057 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c3332-75b8-406c-b973-3bc0362151a5 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:01.795039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e35f4f1-37e0-4f7e-bccf-ace6f283930f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:01.839381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c3332-75b8-406c-b973-3bc0362151a5 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:01.872632 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0aad16-f532-4352-95ae-0dea387f4e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:01.873368 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aad16-f532-4352-95ae-0dea387f4e6b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:01.873649 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aad16-f532-4352-95ae-0dea387f4e6b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:01.873926 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aad16-f532-4352-95ae-0dea387f4e6b None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:01.874719 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0aad16-f532-4352-95ae-0dea387f4e6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:01.875030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0aad16-f532-4352-95ae-0dea387f4e6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:01.876513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0aad16-f532-4352-95ae-0dea387f4e6b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:01.896447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 275/542] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:01.902313 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0aad16-f532-4352-95ae-0dea387f4e6b None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:01.908618 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea5008-6d39-4635-8f2b-c7611041a227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:01.910116 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea5008-6d39-4635-8f2b-c7611041a227 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:01.910403 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea5008-6d39-4635-8f2b-c7611041a227 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:01.910676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea5008-6d39-4635-8f2b-c7611041a227 None admin] PATH_INFO: `/v3/roles/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:01.937122 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea5008-6d39-4635-8f2b-c7611041a227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-d4ea5008-6d39-4635-8f2b-c7611041a227 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:01.950395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:01.951493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:01.952443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:01.959445 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 268/543] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:01.963967 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:01.965780 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea5008-6d39-4635-8f2b-c7611041a227 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:02.028713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:02.061715 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0aad16-f532-4352-95ae-0dea387f4e6b None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:02.099468 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543571-9f06-471b-9fbe-90cdf29c13df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:02.100145 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543571-9f06-471b-9fbe-90cdf29c13df None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:02.100455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543571-9f06-471b-9fbe-90cdf29c13df None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:02.100777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6543571-9f06-471b-9fbe-90cdf29c13df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:02.101692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6543571-9f06-471b-9fbe-90cdf29c13df None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=b272b57eafcb4de9b85ce8a8a893a2d1) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:02.102036 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6543571-9f06-471b-9fbe-90cdf29c13df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:02.108367 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6543571-9f06-471b-9fbe-90cdf29c13df None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:02.112701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 276/544] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:01 2022] GET /identity/v3/role_assignments?user.id=b272b57eafcb4de9b85ce8a8a893a2d1&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:02.190436 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea5008-6d39-4635-8f2b-c7611041a227 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:02.244018 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c92d40-9c2b-4fb7-919a-4302d429822a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:02.244813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c92d40-9c2b-4fb7-919a-4302d429822a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:02.245096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c92d40-9c2b-4fb7-919a-4302d429822a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:02.245371 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c92d40-9c2b-4fb7-919a-4302d429822a None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:02.246203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c92d40-9c2b-4fb7-919a-4302d429822a None admin] RBAC: Adding query filter params (name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:02.246517 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c92d40-9c2b-4fb7-919a-4302d429822a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:02.247987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c92d40-9c2b-4fb7-919a-4302d429822a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:02.257892 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 269/545] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:02.265674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543571-9f06-471b-9fbe-90cdf29c13df None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:02.384584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:02.391800 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6543571-9f06-471b-9fbe-90cdf29c13df None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:02.415751 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d94fb-7994-42f9-9421-8e4f936c70ef None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:02.416463 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d94fb-7994-42f9-9421-8e4f936c70ef None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:02.417228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d94fb-7994-42f9-9421-8e4f936c70ef None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:02.419081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 270/546] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:02.425151 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c4d47-42c0-4187-a586-7b598d8c7202 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:02.425756 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c4d47-42c0-4187-a586-7b598d8c7202 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:02.426245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c4d47-42c0-4187-a586-7b598d8c7202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:02.455288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f22d3e-913d-4929-ba73-b445dfde43be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:02.456149 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22d3e-913d-4929-ba73-b445dfde43be None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:02.456462 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22d3e-913d-4929-ba73-b445dfde43be None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:02.456745 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f22d3e-913d-4929-ba73-b445dfde43be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:02.471914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-802c4d47-42c0-4187-a586-7b598d8c7202 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:02.473234 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-802c4d47-42c0-4187-a586-7b598d8c7202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:02.483995 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f22d3e-913d-4929-ba73-b445dfde43be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:02.485585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f22d3e-913d-4929-ba73-b445dfde43be None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-24f22d3e-913d-4929-ba73-b445dfde43be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:02.510627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:02.511673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:02.512686 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:02.512686 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:02.512686 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:02.512686 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:02.512686 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 277/547] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:02.518370 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f22d3e-913d-4929-ba73-b445dfde43be None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:02.610829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 271/548] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:29:02.647043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c4d47-42c0-4187-a586-7b598d8c7202 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:02.673786 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:02.691656 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f22d3e-913d-4929-ba73-b445dfde43be None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:02.757022 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73eabc-f49b-4e10-a6c7-e04bcbc70d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:02.757703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73eabc-f49b-4e10-a6c7-e04bcbc70d6a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:02.757987 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73eabc-f49b-4e10-a6c7-e04bcbc70d6a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:02.758262 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73eabc-f49b-4e10-a6c7-e04bcbc70d6a None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:02.759001 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73eabc-f49b-4e10-a6c7-e04bcbc70d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:02.759310 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73eabc-f49b-4e10-a6c7-e04bcbc70d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:02.763092 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73eabc-f49b-4e10-a6c7-e04bcbc70d6a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:02.785061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 278/549] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:02.796319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73eabc-f49b-4e10-a6c7-e04bcbc70d6a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:02.809859 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:02.814678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c4d47-42c0-4187-a586-7b598d8c7202 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:02.871724 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b8edd-4586-44f8-b43b-6a47dd70104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:02.872564 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b8edd-4586-44f8-b43b-6a47dd70104a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:02.872847 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b8edd-4586-44f8-b43b-6a47dd70104a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:02.873122 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b8edd-4586-44f8-b43b-6a47dd70104a None admin] PATH_INFO: `/v3/roles/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:02.898879 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b8edd-4586-44f8-b43b-6a47dd70104a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-914b8edd-4586-44f8-b43b-6a47dd70104a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:02.917320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:02.918409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:02.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 272/550] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:02.926971 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b8edd-4586-44f8-b43b-6a47dd70104a None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:02.994073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73eabc-f49b-4e10-a6c7-e04bcbc70d6a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:03.019302 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:03.054215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b8edd-4586-44f8-b43b-6a47dd70104a None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:03.068687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596beade-33ed-4d44-9c80-8a640f3aaa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:03.069508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596beade-33ed-4d44-9c80-8a640f3aaa77 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.069788 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596beade-33ed-4d44-9c80-8a640f3aaa77 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.070065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596beade-33ed-4d44-9c80-8a640f3aaa77 None admin] PATH_INFO: `/v3/users/swift` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.079008 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596beade-33ed-4d44-9c80-8a640f3aaa77 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:03.081297 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596beade-33ed-4d44-9c80-8a640f3aaa77 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-596beade-33ed-4d44-9c80-8a640f3aaa77 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:03.096241 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:03.097954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:03.099012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:03.099012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:03.099012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:03.099012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:03.099012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:03.099012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:03.099012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:03.099012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:03.099012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:03.099012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:03.099012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 279/551] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:03.103685 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596beade-33ed-4d44-9c80-8a640f3aaa77 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:03.151802 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0246a7-df4d-4bbb-8089-df5bbafd7a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:03.152496 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0246a7-df4d-4bbb-8089-df5bbafd7a5b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.152785 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0246a7-df4d-4bbb-8089-df5bbafd7a5b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.153062 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0246a7-df4d-4bbb-8089-df5bbafd7a5b None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.153820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0246a7-df4d-4bbb-8089-df5bbafd7a5b None admin] RBAC: Adding query filter params (name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:03.154132 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0246a7-df4d-4bbb-8089-df5bbafd7a5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:03.155504 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0246a7-df4d-4bbb-8089-df5bbafd7a5b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:03.165124 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 273/552] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:03.171299 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0246a7-df4d-4bbb-8089-df5bbafd7a5b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:03.181501 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:03.227432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596beade-33ed-4d44-9c80-8a640f3aaa77 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:03.281663 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0246a7-df4d-4bbb-8089-df5bbafd7a5b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:03.317020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164978d8-cf7d-48b5-b774-0f7584948ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:03.317828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164978d8-cf7d-48b5-b774-0f7584948ef5 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164978d8-cf7d-48b5-b774-0f7584948ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.318380 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164978d8-cf7d-48b5-b774-0f7584948ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.325813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:03.329538 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13283e22-effc-4729-88e6-979573028a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:03.330464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164978d8-cf7d-48b5-b774-0f7584948ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:03.330724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13283e22-effc-4729-88e6-979573028a1e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.331752 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13283e22-effc-4729-88e6-979573028a1e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.332077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13283e22-effc-4729-88e6-979573028a1e None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.332410 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164978d8-cf7d-48b5-b774-0f7584948ef5 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:03.332991 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13283e22-effc-4729-88e6-979573028a1e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:03.333312 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13283e22-effc-4729-88e6-979573028a1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:03.334775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13283e22-effc-4729-88e6-979573028a1e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-164978d8-cf7d-48b5-b774-0f7584948ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:03.351627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:03.352681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:03.354077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:03.354077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:03.354077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:03.354077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:03.354077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 274/553] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:03.358843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164978d8-cf7d-48b5-b774-0f7584948ef5 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:03.380161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 280/554] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:29:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:03.385301 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13283e22-effc-4729-88e6-979573028a1e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:03.487846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164978d8-cf7d-48b5-b774-0f7584948ef5 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:03.530431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc50d7b-0211-4c8a-9717-706a926d03e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:03.531247 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc50d7b-0211-4c8a-9717-706a926d03e1 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.531540 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc50d7b-0211-4c8a-9717-706a926d03e1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.531876 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc50d7b-0211-4c8a-9717-706a926d03e1 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.532662 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc50d7b-0211-4c8a-9717-706a926d03e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:03.532975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc50d7b-0211-4c8a-9717-706a926d03e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:03.534381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc50d7b-0211-4c8a-9717-706a926d03e1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:03.556137 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 275/555] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:03.563416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc50d7b-0211-4c8a-9717-706a926d03e1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:03.576832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13283e22-effc-4729-88e6-979573028a1e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:03.635487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:03.660406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e887dc-97ea-4d64-b7e5-d74f8d07ff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:03.661382 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e887dc-97ea-4d64-b7e5-d74f8d07ff09 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.661736 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e887dc-97ea-4d64-b7e5-d74f8d07ff09 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.662116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e887dc-97ea-4d64-b7e5-d74f8d07ff09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.676742 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:03.724909 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc50d7b-0211-4c8a-9717-706a926d03e1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:03.763447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7e40e-ff4a-444f-81ec-439bad4a6c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:03.764527 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7e40e-ff4a-444f-81ec-439bad4a6c46 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.764932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7e40e-ff4a-444f-81ec-439bad4a6c46 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.765160 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7e40e-ff4a-444f-81ec-439bad4a6c46 None admin] PATH_INFO: `/v3/users/nova` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.773511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7e40e-ff4a-444f-81ec-439bad4a6c46 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:03.775049 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7e40e-ff4a-444f-81ec-439bad4a6c46 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-bfe7e40e-ff4a-444f-81ec-439bad4a6c46 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:03.790418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:03.791562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 276/556] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:29:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:03.793622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38103676-630e-4997-ad5b-0dcd96556ef9 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.794522 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38103676-630e-4997-ad5b-0dcd96556ef9 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.794522 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38103676-630e-4997-ad5b-0dcd96556ef9 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.800075 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 277/557] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:03.800965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38103676-630e-4997-ad5b-0dcd96556ef9 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:03.806063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e887dc-97ea-4d64-b7e5-d74f8d07ff09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:03.807555 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e887dc-97ea-4d64-b7e5-d74f8d07ff09 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:03.912442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38103676-630e-4997-ad5b-0dcd96556ef9 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:03.918808 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 281/558] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:29:03.920117 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac39da-a5d9-4e79-b8c4-aae133cab1c3 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.920425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac39da-a5d9-4e79-b8c4-aae133cab1c3 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.920706 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac39da-a5d9-4e79-b8c4-aae133cab1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.929368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:03.938069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-1dac39da-a5d9-4e79-b8c4-aae133cab1c3 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:03.943774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-1dac39da-a5d9-4e79-b8c4-aae133cab1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:03.954298 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc233762-d853-4664-947a-0c4819f9e903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:03.954986 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc233762-d853-4664-947a-0c4819f9e903 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.955271 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc233762-d853-4664-947a-0c4819f9e903 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.955548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc233762-d853-4664-947a-0c4819f9e903 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.956441 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc233762-d853-4664-947a-0c4819f9e903 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:03.956758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc233762-d853-4664-947a-0c4819f9e903 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:03.958306 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc233762-d853-4664-947a-0c4819f9e903 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:03.959357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:03.980701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 278/559] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:29:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:03.980701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53344689-2164-4dda-8722-3cb32cc9dee2 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:03.980701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53344689-2164-4dda-8722-3cb32cc9dee2 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:03.980701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53344689-2164-4dda-8722-3cb32cc9dee2 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:03.980701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 279/560] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:03.980701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53344689-2164-4dda-8722-3cb32cc9dee2 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.028382 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 282/561] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:04.028382 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac39da-a5d9-4e79-b8c4-aae133cab1c3 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.092952 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53344689-2164-4dda-8722-3cb32cc9dee2 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:04.129349 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608968ab-8a12-458e-b1c7-b1cfa17c43de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:04.130162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608968ab-8a12-458e-b1c7-b1cfa17c43de None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:04.130448 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608968ab-8a12-458e-b1c7-b1cfa17c43de None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:04.130725 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608968ab-8a12-458e-b1c7-b1cfa17c43de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:04.131861 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac39da-a5d9-4e79-b8c4-aae133cab1c3 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:04.141179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608968ab-8a12-458e-b1c7-b1cfa17c43de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:04.142684 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608968ab-8a12-458e-b1c7-b1cfa17c43de None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-608968ab-8a12-458e-b1c7-b1cfa17c43de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:04.162305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:04.163370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:04.164444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:04.164444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:04.164444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:04.164444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:04.164444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 280/562] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:04.165191 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7200a6a-9550-4b84-8c6f-145b16b6e28b None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:04.165303 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7200a6a-9550-4b84-8c6f-145b16b6e28b None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:04.165592 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7200a6a-9550-4b84-8c6f-145b16b6e28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:04.176800 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0759f-3d37-4692-b267-4b67a9e9d3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:04.177610 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0759f-3d37-4692-b267-4b67a9e9d3f8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:04.177895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0759f-3d37-4692-b267-4b67a9e9d3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:04.178610 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0759f-3d37-4692-b267-4b67a9e9d3f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:04.184470 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-d7200a6a-9550-4b84-8c6f-145b16b6e28b None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:04.185867 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-d7200a6a-9550-4b84-8c6f-145b16b6e28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:04.189092 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0759f-3d37-4692-b267-4b67a9e9d3f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:04.190612 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0759f-3d37-4692-b267-4b67a9e9d3f8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:04.202112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-84f0759f-3d37-4692-b267-4b67a9e9d3f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:04.208426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:04.209434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:04.210509 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:04.210509 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:04.210509 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:04.210509 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:04.210509 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 283/563] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:04.210509 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0759f-3d37-4692-b267-4b67a9e9d3f8 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.223427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:04.263869 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 281/564] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:04.264846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7200a6a-9550-4b84-8c6f-145b16b6e28b None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.334702 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0759f-3d37-4692-b267-4b67a9e9d3f8 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:04.377693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dcffd-67c4-4a55-abe5-f8dd0b2ac1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:04.378498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dcffd-67c4-4a55-abe5-f8dd0b2ac1b3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:04.378786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dcffd-67c4-4a55-abe5-f8dd0b2ac1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:04.379063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dcffd-67c4-4a55-abe5-f8dd0b2ac1b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:04.385889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7200a6a-9550-4b84-8c6f-145b16b6e28b None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:04.390669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dcffd-67c4-4a55-abe5-f8dd0b2ac1b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-546dcffd-67c4-4a55-abe5-f8dd0b2ac1b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:04.398755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:04.399806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:04.400812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:04.404842 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 284/565] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:04.405802 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dcffd-67c4-4a55-abe5-f8dd0b2ac1b3 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.430852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e85241-4ac9-4c15-b54d-570ba6629628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:04.432003 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85241-4ac9-4c15-b54d-570ba6629628 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:04.432293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85241-4ac9-4c15-b54d-570ba6629628 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:04.432603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e85241-4ac9-4c15-b54d-570ba6629628 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:04.433351 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e85241-4ac9-4c15-b54d-570ba6629628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:04.433661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e85241-4ac9-4c15-b54d-570ba6629628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:04.435093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e85241-4ac9-4c15-b54d-570ba6629628 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:04.455004 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 282/566] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:04.455603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e85241-4ac9-4c15-b54d-570ba6629628 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.469825 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:04.486621 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:04.522919 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dcffd-67c4-4a55-abe5-f8dd0b2ac1b3 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:04.559906 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14133f-6c72-443a-9296-1c7d8842ee24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:04.560614 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14133f-6c72-443a-9296-1c7d8842ee24 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:04.560909 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14133f-6c72-443a-9296-1c7d8842ee24 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:04.561194 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14133f-6c72-443a-9296-1c7d8842ee24 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:04.561454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e85241-4ac9-4c15-b54d-570ba6629628 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:04.561929 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14133f-6c72-443a-9296-1c7d8842ee24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:04.562246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14133f-6c72-443a-9296-1c7d8842ee24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:04.563637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14133f-6c72-443a-9296-1c7d8842ee24 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:04.582189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 285/567] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:04.582189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14133f-6c72-443a-9296-1c7d8842ee24 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.603481 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f50c6-fd3f-4bb6-9c87-11536d2ae93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:04.604400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f50c6-fd3f-4bb6-9c87-11536d2ae93f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:04.604760 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f50c6-fd3f-4bb6-9c87-11536d2ae93f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:04.605127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f50c6-fd3f-4bb6-9c87-11536d2ae93f None admin] PATH_INFO: `/v3/roles/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:04.616096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f50c6-fd3f-4bb6-9c87-11536d2ae93f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-4c3f50c6-fd3f-4bb6-9c87-11536d2ae93f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:04.624224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:04.625678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:04.626598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 283/568] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:04.627371 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f50c6-fd3f-4bb6-9c87-11536d2ae93f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.697186 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14133f-6c72-443a-9296-1c7d8842ee24 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:04.706536 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:04.731586 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:04.741052 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39178f33-987a-4273-afa1-e0f7f09ad0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:04.741826 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39178f33-987a-4273-afa1-e0f7f09ad0ba None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:04.742121 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39178f33-987a-4273-afa1-e0f7f09ad0ba None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:04.742398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39178f33-987a-4273-afa1-e0f7f09ad0ba None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:04.743162 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39178f33-987a-4273-afa1-e0f7f09ad0ba None admin] RBAC: Adding query filter params (name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:04.743474 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39178f33-987a-4273-afa1-e0f7f09ad0ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:04.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f50c6-fd3f-4bb6-9c87-11536d2ae93f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:04.744891 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39178f33-987a-4273-afa1-e0f7f09ad0ba None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:04.755349 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 286/569] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:04.756194 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39178f33-987a-4273-afa1-e0f7f09ad0ba None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.786862 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead1a37-b3a1-4add-8caf-818227e643c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:04.788150 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead1a37-b3a1-4add-8caf-818227e643c4 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:04.788150 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead1a37-b3a1-4add-8caf-818227e643c4 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:04.788427 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead1a37-b3a1-4add-8caf-818227e643c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:04.796939 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead1a37-b3a1-4add-8caf-818227e643c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:04.798486 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead1a37-b3a1-4add-8caf-818227e643c4 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-7ead1a37-b3a1-4add-8caf-818227e643c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:04.813978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:04.815046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:04.816073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:04.816073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:04.816073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:04.816073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:04.816073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:04.816073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:04.816073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:04.816073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:04.816073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 284/570] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:04.817339 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead1a37-b3a1-4add-8caf-818227e643c4 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.876290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39178f33-987a-4273-afa1-e0f7f09ad0ba None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:04.910638 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d4d5b-70ea-4b68-9ff2-37bc8fba8074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:04.911435 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d4d5b-70ea-4b68-9ff2-37bc8fba8074 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:04.911726 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d4d5b-70ea-4b68-9ff2-37bc8fba8074 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:04.912003 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d4d5b-70ea-4b68-9ff2-37bc8fba8074 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:04.917961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead1a37-b3a1-4add-8caf-818227e643c4 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:04.918994 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d4d5b-70ea-4b68-9ff2-37bc8fba8074 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:04.920484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d4d5b-70ea-4b68-9ff2-37bc8fba8074 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-b04d4d5b-70ea-4b68-9ff2-37bc8fba8074 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:04.934634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:04.935841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:04.937042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:04.937042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:04.937042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:04.937042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:04.937042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:04.937042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:04.937042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:04.937042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:04.937042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 287/571] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:04.937042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d4d5b-70ea-4b68-9ff2-37bc8fba8074 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.949797 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:04.955698 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8fb93-d3d0-4e8a-8c31-3ca42e80aed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:04.957035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8fb93-d3d0-4e8a-8c31-3ca42e80aed2 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:04.957317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8fb93-d3d0-4e8a-8c31-3ca42e80aed2 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:04.965188 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b8fb93-d3d0-4e8a-8c31-3ca42e80aed2 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:04.965188 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8fb93-d3d0-4e8a-8c31-3ca42e80aed2 None admin] RBAC: Adding query filter params (name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:04.965188 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8fb93-d3d0-4e8a-8c31-3ca42e80aed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:04.965188 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b8fb93-d3d0-4e8a-8c31-3ca42e80aed2 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:04.967847 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 285/572] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:04.968674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8fb93-d3d0-4e8a-8c31-3ca42e80aed2 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:04.991687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:05.044480 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d4d5b-70ea-4b68-9ff2-37bc8fba8074 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:05.076896 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b8fb93-d3d0-4e8a-8c31-3ca42e80aed2 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:05.079808 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334b35f-900e-40c9-b4ce-425f4d773b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:05.080661 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334b35f-900e-40c9-b4ce-425f4d773b20 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.081613 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334b35f-900e-40c9-b4ce-425f4d773b20 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.081893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334b35f-900e-40c9-b4ce-425f4d773b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.092413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334b35f-900e-40c9-b4ce-425f4d773b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:05.093908 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334b35f-900e-40c9-b4ce-425f4d773b20 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-e334b35f-900e-40c9-b4ce-425f4d773b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:05.111579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:05.112707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:05.113701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:05.113701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:05.113701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:05.113701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:05.114275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7e112-b724-428a-82a4-c7bb9697f2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:05.114275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e112-b724-428a-82a4-c7bb9697f2eb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.114275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e112-b724-428a-82a4-c7bb9697f2eb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.114613 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 288/573] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:05.114705 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7e112-b724-428a-82a4-c7bb9697f2eb None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.114705 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7e112-b724-428a-82a4-c7bb9697f2eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:05.114849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea5a26-f3df-4ad2-9a8a-5292194b05ab None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.114922 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7e112-b724-428a-82a4-c7bb9697f2eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:05.115008 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea5a26-f3df-4ad2-9a8a-5292194b05ab None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.115008 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea5a26-f3df-4ad2-9a8a-5292194b05ab None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.115572 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7e112-b724-428a-82a4-c7bb9697f2eb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:05.116001 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 289/574] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:05.117003 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea5a26-f3df-4ad2-9a8a-5292194b05ab None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:05.130227 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 286/575] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:05.131040 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7e112-b724-428a-82a4-c7bb9697f2eb None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:05.184512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:05.220192 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:05.223402 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea5a26-f3df-4ad2-9a8a-5292194b05ab None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:05.231473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7e112-b724-428a-82a4-c7bb9697f2eb None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:05.259076 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a817b7-ad4b-4ef7-98c4-de23268c10cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:05.259841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a817b7-ad4b-4ef7-98c4-de23268c10cf None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.260204 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a817b7-ad4b-4ef7-98c4-de23268c10cf None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.260524 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a817b7-ad4b-4ef7-98c4-de23268c10cf None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.261326 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a817b7-ad4b-4ef7-98c4-de23268c10cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:05.261645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a817b7-ad4b-4ef7-98c4-de23268c10cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:05.263094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a817b7-ad4b-4ef7-98c4-de23268c10cf None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:05.267434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d5d4c-a8c1-47c3-be81-f0adda107408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:05.268308 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d5d4c-a8c1-47c3-be81-f0adda107408 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.268631 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d5d4c-a8c1-47c3-be81-f0adda107408 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.268911 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d5d4c-a8c1-47c3-be81-f0adda107408 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.274938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 290/576] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:05.275775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a817b7-ad4b-4ef7-98c4-de23268c10cf None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:05.284061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d5d4c-a8c1-47c3-be81-f0adda107408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:05.285551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d5d4c-a8c1-47c3-be81-f0adda107408 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-978d5d4c-a8c1-47c3-be81-f0adda107408 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:05.304293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:05.305447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:05.306438 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:05.306438 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:05.306438 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:05.306438 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:05.306438 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 287/577] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:05.306735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb238f84-efaf-4125-99b0-e3bf3612d530 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.306925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb238f84-efaf-4125-99b0-e3bf3612d530 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.307208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb238f84-efaf-4125-99b0-e3bf3612d530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.323860 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-bb238f84-efaf-4125-99b0-e3bf3612d530 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:05.325104 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-bb238f84-efaf-4125-99b0-e3bf3612d530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:05.386086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a817b7-ad4b-4ef7-98c4-de23268c10cf None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:05.393112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 288/578] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:05.393989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb238f84-efaf-4125-99b0-e3bf3612d530 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:05.422554 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66467f2-7235-4bdc-bd36-38cfaba9e70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:05.423297 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66467f2-7235-4bdc-bd36-38cfaba9e70f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.423587 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66467f2-7235-4bdc-bd36-38cfaba9e70f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.423863 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66467f2-7235-4bdc-bd36-38cfaba9e70f None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.425262 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66467f2-7235-4bdc-bd36-38cfaba9e70f None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:05.425569 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66467f2-7235-4bdc-bd36-38cfaba9e70f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:05.426981 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66467f2-7235-4bdc-bd36-38cfaba9e70f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:05.432636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:05.439079 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 291/579] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:05.440112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66467f2-7235-4bdc-bd36-38cfaba9e70f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:05.462336 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:05.500511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb238f84-efaf-4125-99b0-e3bf3612d530 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:05.534878 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ae3e1-b704-4cc7-8928-15df1e09176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:05.535540 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ae3e1-b704-4cc7-8928-15df1e09176c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.535825 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ae3e1-b704-4cc7-8928-15df1e09176c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.536101 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ae3e1-b704-4cc7-8928-15df1e09176c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.536976 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ae3e1-b704-4cc7-8928-15df1e09176c None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=14034b78ec5e4999848a3fffb2964587) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:05.537293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ae3e1-b704-4cc7-8928-15df1e09176c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:05.538718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ae3e1-b704-4cc7-8928-15df1e09176c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:05.540207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66467f2-7235-4bdc-bd36-38cfaba9e70f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:05.545310 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 289/580] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:05 2022] GET /identity/v3/role_assignments?user.id=14034b78ec5e4999848a3fffb2964587&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:05.546102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ae3e1-b704-4cc7-8928-15df1e09176c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:05.627755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f057fe-19d2-442e-a371-7baa1db2b4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:05.628613 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f057fe-19d2-442e-a371-7baa1db2b4cc None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.628894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f057fe-19d2-442e-a371-7baa1db2b4cc None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.629164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f057fe-19d2-442e-a371-7baa1db2b4cc None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/e6a982ae15a6443b9b52529df47aa837/roles/ec2a4317dc7343208d439d5f2d44d6c3` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.647455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f057fe-19d2-442e-a371-7baa1db2b4cc None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=e6a982ae15a6443b9b52529df47aa837, role_id=ec2a4317dc7343208d439d5f2d44d6c3)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:05.649100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f057fe-19d2-442e-a371-7baa1db2b4cc None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:05.693260 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:05.697302 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ae3e1-b704-4cc7-8928-15df1e09176c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:05.733501 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09404e-4195-4088-b09d-b91f3cdaaef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:05.734193 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09404e-4195-4088-b09d-b91f3cdaaef0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.734480 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09404e-4195-4088-b09d-b91f3cdaaef0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.734760 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09404e-4195-4088-b09d-b91f3cdaaef0 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.735494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09404e-4195-4088-b09d-b91f3cdaaef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:05.736556 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09404e-4195-4088-b09d-b91f3cdaaef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:05.737764 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09404e-4195-4088-b09d-b91f3cdaaef0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:05.741456 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 292/581] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:29:05 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/e6a982ae15a6443b9b52529df47aa837/roles/ec2a4317dc7343208d439d5f2d44d6c3 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:29:05.742557 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f057fe-19d2-442e-a371-7baa1db2b4cc None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:05.760209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 290/582] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:05.761745 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09404e-4195-4088-b09d-b91f3cdaaef0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:05.844064 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:05.888771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09404e-4195-4088-b09d-b91f3cdaaef0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:05.890564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f057fe-19d2-442e-a371-7baa1db2b4cc None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:05.935651 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c1b61-9db9-4ca8-b3fd-b98ea8b750b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:05.936936 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c1b61-9db9-4ca8-b3fd-b98ea8b750b2 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.937432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c1b61-9db9-4ca8-b3fd-b98ea8b750b2 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.938430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87003ca8-1b46-4703-9744-ec0b48df2e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:05.938625 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87003ca8-1b46-4703-9744-ec0b48df2e07 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:05.938909 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87003ca8-1b46-4703-9744-ec0b48df2e07 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:05.939191 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87003ca8-1b46-4703-9744-ec0b48df2e07 None admin] PATH_INFO: `/v3/users/glance` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.939494 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c1b61-9db9-4ca8-b3fd-b98ea8b750b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:05.948920 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87003ca8-1b46-4703-9744-ec0b48df2e07 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:05.950988 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87003ca8-1b46-4703-9744-ec0b48df2e07 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:05.953917 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c1b61-9db9-4ca8-b3fd-b98ea8b750b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-f59c1b61-9db9-4ca8-b3fd-b98ea8b750b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:05.962531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:05.963655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 293/583] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:05.964618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c1b61-9db9-4ca8-b3fd-b98ea8b750b2 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-87003ca8-1b46-4703-9744-ec0b48df2e07 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:05.967455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:05.968514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:05.969626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:05.969626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:05.969626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:05.969626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:05.969626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:05.969626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:05.969626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:05.969626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:05.969626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:29:05.969626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:05.969626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 291/584] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:29:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:05.970628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87003ca8-1b46-4703-9744-ec0b48df2e07 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:05.996367 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:06.079257 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87003ca8-1b46-4703-9744-ec0b48df2e07 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:06.088190 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c1b61-9db9-4ca8-b3fd-b98ea8b750b2 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:06.121691 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:06.126511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d3f69-6211-4f5f-9e57-49a4e60d28f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:06.127201 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d3f69-6211-4f5f-9e57-49a4e60d28f1 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:06.127548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d3f69-6211-4f5f-9e57-49a4e60d28f1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:06.127790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d3f69-6211-4f5f-9e57-49a4e60d28f1 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:06.128584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d3f69-6211-4f5f-9e57-49a4e60d28f1 None admin] RBAC: Adding query filter params (name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:06.128935 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d3f69-6211-4f5f-9e57-49a4e60d28f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:06.129758 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf614aa-75ca-469b-88f0-5ebe9a4ac3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:06.130278 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d3f69-6211-4f5f-9e57-49a4e60d28f1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:06.130934 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf614aa-75ca-469b-88f0-5ebe9a4ac3ae None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:06.131372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf614aa-75ca-469b-88f0-5ebe9a4ac3ae None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:06.131906 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf614aa-75ca-469b-88f0-5ebe9a4ac3ae None admin] PATH_INFO: `/v3/users/cinder` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:06.140656 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 292/585] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:06.141310 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf614aa-75ca-469b-88f0-5ebe9a4ac3ae None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:06.141428 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d3f69-6211-4f5f-9e57-49a4e60d28f1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:06.143748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf614aa-75ca-469b-88f0-5ebe9a4ac3ae None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-1cf614aa-75ca-469b-88f0-5ebe9a4ac3ae None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:06.161100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:06.162165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:06.163197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:06.163197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:06.163197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:06.163197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:06.163197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:06.163197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:06.163197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:06.163197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:06.163197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:29:06.163197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:06.170761 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 294/586] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:29:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:06.172017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf614aa-75ca-469b-88f0-5ebe9a4ac3ae None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:06.237885 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:06.259784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d3f69-6211-4f5f-9e57-49a4e60d28f1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:06.286811 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf614aa-75ca-469b-88f0-5ebe9a4ac3ae None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:06.309573 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74ea58-fecb-42f0-a31d-18e7613ffb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:06.310265 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74ea58-fecb-42f0-a31d-18e7613ffb41 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:06.310545 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74ea58-fecb-42f0-a31d-18e7613ffb41 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:06.310816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74ea58-fecb-42f0-a31d-18e7613ffb41 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:06.311668 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74ea58-fecb-42f0-a31d-18e7613ffb41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:06.311975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74ea58-fecb-42f0-a31d-18e7613ffb41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:06.313461 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74ea58-fecb-42f0-a31d-18e7613ffb41 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:06.329602 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515f37a-be7e-44cb-805e-2e59f43fd6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:06.331642 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515f37a-be7e-44cb-805e-2e59f43fd6d5 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:06.332161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515f37a-be7e-44cb-805e-2e59f43fd6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:06.332687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3515f37a-be7e-44cb-805e-2e59f43fd6d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:06.337913 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 293/587] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:29:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:06.338816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74ea58-fecb-42f0-a31d-18e7613ffb41 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:06.347853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515f37a-be7e-44cb-805e-2e59f43fd6d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:06.350252 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3515f37a-be7e-44cb-805e-2e59f43fd6d5 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-3515f37a-be7e-44cb-805e-2e59f43fd6d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:06.375028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:06.376045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:06.377168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:06.377168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:06.377168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:06.377168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:06.378598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 295/588] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:06.380788 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515f37a-be7e-44cb-805e-2e59f43fd6d5 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:06.433664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:06.459703 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74ea58-fecb-42f0-a31d-18e7613ffb41 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:06.496328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3515f37a-be7e-44cb-805e-2e59f43fd6d5 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:06.499872 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d770e5-8d00-4ac6-90eb-77df0c2f127c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:06.500897 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d770e5-8d00-4ac6-90eb-77df0c2f127c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:06.501477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d770e5-8d00-4ac6-90eb-77df0c2f127c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:06.501958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d770e5-8d00-4ac6-90eb-77df0c2f127c None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:06.503603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d770e5-8d00-4ac6-90eb-77df0c2f127c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:06.504227 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d770e5-8d00-4ac6-90eb-77df0c2f127c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:06.505844 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d770e5-8d00-4ac6-90eb-77df0c2f127c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:06.521861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:06.531987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 294/589] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:29:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:06.533094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e8333-bbde-4afb-9b0a-68242ab38658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:06.533587 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d770e5-8d00-4ac6-90eb-77df0c2f127c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:06.533968 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8333-bbde-4afb-9b0a-68242ab38658 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:06.534252 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8333-bbde-4afb-9b0a-68242ab38658 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:06.534529 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e8333-bbde-4afb-9b0a-68242ab38658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:06.546652 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e8333-bbde-4afb-9b0a-68242ab38658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:06.548157 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e8333-bbde-4afb-9b0a-68242ab38658 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-e82e8333-bbde-4afb-9b0a-68242ab38658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:06.566446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:06.567484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:06.568497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:06.568497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:06.568497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:06.568497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:06.572383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 296/590] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:06.573440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e8333-bbde-4afb-9b0a-68242ab38658 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:06.652838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d770e5-8d00-4ac6-90eb-77df0c2f127c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:06.675860 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:06.682640 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e8333-bbde-4afb-9b0a-68242ab38658 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:06.693746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1f2a5-873f-41dd-9c4b-78b0fe1595df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:06.694814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1f2a5-873f-41dd-9c4b-78b0fe1595df None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:06.695412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1f2a5-873f-41dd-9c4b-78b0fe1595df None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:06.695927 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1f2a5-873f-41dd-9c4b-78b0fe1595df None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:06.696936 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1f2a5-873f-41dd-9c4b-78b0fe1595df None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:06.697471 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1f2a5-873f-41dd-9c4b-78b0fe1595df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:06.699073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1f2a5-873f-41dd-9c4b-78b0fe1595df None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:06.715687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 295/591] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:06.717178 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1f2a5-873f-41dd-9c4b-78b0fe1595df None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:06.722104 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf51c7-5408-41e4-bf25-2f053012c98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:06.722372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf51c7-5408-41e4-bf25-2f053012c98e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:06.722637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf51c7-5408-41e4-bf25-2f053012c98e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:06.722912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf51c7-5408-41e4-bf25-2f053012c98e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:06.733843 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf51c7-5408-41e4-bf25-2f053012c98e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:06.735390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf51c7-5408-41e4-bf25-2f053012c98e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-edbf51c7-5408-41e4-bf25-2f053012c98e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:06.756743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:06.757779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:06.758799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:06.758799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:06.758799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:06.758799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:06.762561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 297/592] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:06.763444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf51c7-5408-41e4-bf25-2f053012c98e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:06.790493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:06.853305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1f2a5-873f-41dd-9c4b-78b0fe1595df None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:06.893508 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4d8c4-a94c-4003-a81f-f187c5772752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:06.894155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4d8c4-a94c-4003-a81f-f187c5772752 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:06.894458 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4d8c4-a94c-4003-a81f-f187c5772752 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:06.894754 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4d8c4-a94c-4003-a81f-f187c5772752 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:06.895485 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4d8c4-a94c-4003-a81f-f187c5772752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:06.895791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4d8c4-a94c-4003-a81f-f187c5772752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:06.896968 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf51c7-5408-41e4-bf25-2f053012c98e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:06.897531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4d8c4-a94c-4003-a81f-f187c5772752 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:06.915322 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 296/593] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:06.916207 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4d8c4-a94c-4003-a81f-f187c5772752 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:06.943836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f781-3021-4475-99d7-c56738ecd31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:06.945636 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f781-3021-4475-99d7-c56738ecd31d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:06.946446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f781-3021-4475-99d7-c56738ecd31d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:06.947017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f781-3021-4475-99d7-c56738ecd31d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:06.958350 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588f781-3021-4475-99d7-c56738ecd31d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:06.959854 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588f781-3021-4475-99d7-c56738ecd31d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-1588f781-3021-4475-99d7-c56738ecd31d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:06.976492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:06.977663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:06.978709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:06.978709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:06.978709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:06.978709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:06.978709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:06.978709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:06.978709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:06.978709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:06.978709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:29:06.978709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:06.978709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 298/594] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:06.979372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f781-3021-4475-99d7-c56738ecd31d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:06.989256 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:07.051550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4d8c4-a94c-4003-a81f-f187c5772752 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:07.071441 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:07.085058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f781-3021-4475-99d7-c56738ecd31d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:07.092180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb1554-4e9c-4695-a81f-8cdc60ab3a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:07.093244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb1554-4e9c-4695-a81f-8cdc60ab3a95 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:07.093742 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb1554-4e9c-4695-a81f-8cdc60ab3a95 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:07.094225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb1554-4e9c-4695-a81f-8cdc60ab3a95 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:07.095233 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb1554-4e9c-4695-a81f-8cdc60ab3a95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:07.095822 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb1554-4e9c-4695-a81f-8cdc60ab3a95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:07.098261 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb1554-4e9c-4695-a81f-8cdc60ab3a95 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:07.113322 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 297/595] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:07.114531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb1554-4e9c-4695-a81f-8cdc60ab3a95 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:07.127011 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1354985-21a0-46e0-a20d-a358ce0b8b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:07.127842 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1354985-21a0-46e0-a20d-a358ce0b8b1d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:07.128123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1354985-21a0-46e0-a20d-a358ce0b8b1d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:07.128426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1354985-21a0-46e0-a20d-a358ce0b8b1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:07.135928 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1354985-21a0-46e0-a20d-a358ce0b8b1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:07.137330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1354985-21a0-46e0-a20d-a358ce0b8b1d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-f1354985-21a0-46e0-a20d-a358ce0b8b1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:07.151425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:07.152458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:07.153467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:07.153467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:07.153467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:07.153467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:07.153467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:07.153467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:07.153467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:07.153467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:07.157304 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 299/596] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:07.158225 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1354985-21a0-46e0-a20d-a358ce0b8b1d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:07.228381 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:07.238212 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb1554-4e9c-4695-a81f-8cdc60ab3a95 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:07.264535 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1354985-21a0-46e0-a20d-a358ce0b8b1d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:07.276375 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb4270-0607-4930-ac0e-af77ff75285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:07.277095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb4270-0607-4930-ac0e-af77ff75285f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:07.277379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb4270-0607-4930-ac0e-af77ff75285f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:07.277656 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb4270-0607-4930-ac0e-af77ff75285f None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:07.278612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb4270-0607-4930-ac0e-af77ff75285f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:07.279029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb4270-0607-4930-ac0e-af77ff75285f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:07.280514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb4270-0607-4930-ac0e-af77ff75285f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:07.300967 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7d88d-5f18-46d8-af5a-4e21e75ce8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:07.301765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7d88d-5f18-46d8-af5a-4e21e75ce8fc None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:07.302047 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7d88d-5f18-46d8-af5a-4e21e75ce8fc None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:07.302227 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 298/597] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 20:29:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:07.302394 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7d88d-5f18-46d8-af5a-4e21e75ce8fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:07.303414 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb4270-0607-4930-ac0e-af77ff75285f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:07.310490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7d88d-5f18-46d8-af5a-4e21e75ce8fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:07.311958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7d88d-5f18-46d8-af5a-4e21e75ce8fc None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-e4c7d88d-5f18-46d8-af5a-4e21e75ce8fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:07.326931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:07.327924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:07.328973 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:07.328973 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:07.328973 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:07.328973 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:07.328973 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:07.328973 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:07.328973 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:07.328973 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:07.328973 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 300/598] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:07.333485 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7d88d-5f18-46d8-af5a-4e21e75ce8fc None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:07.341308 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:07.421952 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb4270-0607-4930-ac0e-af77ff75285f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:07.439267 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7d88d-5f18-46d8-af5a-4e21e75ce8fc None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:07.471070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:07.483643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c40ecc-f56e-4b0f-8854-71d16a332a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:07.483643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c40ecc-f56e-4b0f-8854-71d16a332a0a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:07.483643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c40ecc-f56e-4b0f-8854-71d16a332a0a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:07.483643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c40ecc-f56e-4b0f-8854-71d16a332a0a None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:07.484112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1c74a-de23-4910-a5c5-6c41b7c10cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:07.484112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1c74a-de23-4910-a5c5-6c41b7c10cd6 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:07.484292 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c40ecc-f56e-4b0f-8854-71d16a332a0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:07.484405 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1c74a-de23-4910-a5c5-6c41b7c10cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:07.484513 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c40ecc-f56e-4b0f-8854-71d16a332a0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:07.484612 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1c74a-de23-4910-a5c5-6c41b7c10cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:07.485289 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c40ecc-f56e-4b0f-8854-71d16a332a0a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:07.498851 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1c74a-de23-4910-a5c5-6c41b7c10cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:07.501675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb1c74a-de23-4910-a5c5-6c41b7c10cd6 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:07.501845 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 299/599] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:07.502059 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c40ecc-f56e-4b0f-8854-71d16a332a0a None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-8bb1c74a-de23-4910-a5c5-6c41b7c10cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:07.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:07.527167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:07.528398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:07.528398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:07.528398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:07.528398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:07.530344 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 301/600] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:07.531422 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1c74a-de23-4910-a5c5-6c41b7c10cd6 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:07.611779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:07.619660 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c40ecc-f56e-4b0f-8854-71d16a332a0a None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:07.638009 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1c74a-de23-4910-a5c5-6c41b7c10cd6 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:07.658447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c9df2-862f-4d66-a4ca-dd5fbc321250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:07.659691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c9df2-862f-4d66-a4ca-dd5fbc321250 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:07.659978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c9df2-862f-4d66-a4ca-dd5fbc321250 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:07.660254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c9df2-862f-4d66-a4ca-dd5fbc321250 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:07.661087 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c9df2-862f-4d66-a4ca-dd5fbc321250 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:07.661401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c9df2-862f-4d66-a4ca-dd5fbc321250 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:07.662952 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c9df2-862f-4d66-a4ca-dd5fbc321250 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:07.677745 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be750482-2bda-42fb-9145-3b6313a07134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:07.677978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 300/601] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:07.678941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c9df2-862f-4d66-a4ca-dd5fbc321250 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:07.680114 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be750482-2bda-42fb-9145-3b6313a07134 None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:07.684608 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be750482-2bda-42fb-9145-3b6313a07134 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:07.685136 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be750482-2bda-42fb-9145-3b6313a07134 None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/c6ace7ef9bfe4837b87c8576862abb75/roles/ec2a4317dc7343208d439d5f2d44d6c3` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:07.702490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be750482-2bda-42fb-9145-3b6313a07134 None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=c6ace7ef9bfe4837b87c8576862abb75, role_id=ec2a4317dc7343208d439d5f2d44d6c3)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:07.704249 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be750482-2bda-42fb-9145-3b6313a07134 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:07.788590 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 302/602] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:29:07 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/c6ace7ef9bfe4837b87c8576862abb75/roles/ec2a4317dc7343208d439d5f2d44d6c3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:29:07.790378 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be750482-2bda-42fb-9145-3b6313a07134 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:07.816086 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c9df2-862f-4d66-a4ca-dd5fbc321250 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:07.863845 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:07.897534 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3265f7-cc5d-47bc-9360-32a32cf450ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:07.898632 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3265f7-cc5d-47bc-9360-32a32cf450ab None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:07.899368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3265f7-cc5d-47bc-9360-32a32cf450ab None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:07.899762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3265f7-cc5d-47bc-9360-32a32cf450ab None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:07.900697 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3265f7-cc5d-47bc-9360-32a32cf450ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:07.901194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3265f7-cc5d-47bc-9360-32a32cf450ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:07.902688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3265f7-cc5d-47bc-9360-32a32cf450ab None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:07.922778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 301/603] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:07.925477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e27385-1cef-4f56-bc25-feedc8de5668 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:07.925861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e27385-1cef-4f56-bc25-feedc8de5668 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:07.926244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e27385-1cef-4f56-bc25-feedc8de5668 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:07.933499 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 302/604] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:07.934449 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e27385-1cef-4f56-bc25-feedc8de5668 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:07.958584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:07.969566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be750482-2bda-42fb-9145-3b6313a07134 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:08.027467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c710ff-e90b-4383-adc3-2411108d7c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:08.028444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c710ff-e90b-4383-adc3-2411108d7c13 None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.028978 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c710ff-e90b-4383-adc3-2411108d7c13 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.029260 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c710ff-e90b-4383-adc3-2411108d7c13 None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/15cb25c883074386a956a0754e12e13f/roles/ec2a4317dc7343208d439d5f2d44d6c3` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.052817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c710ff-e90b-4383-adc3-2411108d7c13 None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=15cb25c883074386a956a0754e12e13f, role_id=ec2a4317dc7343208d439d5f2d44d6c3)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:08.054207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c710ff-e90b-4383-adc3-2411108d7c13 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:08.073116 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e27385-1cef-4f56-bc25-feedc8de5668 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:08.110559 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b58b04-cb4f-4f68-b6a4-a440e25fa8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:08.111374 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58b04-cb4f-4f68-b6a4-a440e25fa8e4 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.111660 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58b04-cb4f-4f68-b6a4-a440e25fa8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.111967 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58b04-cb4f-4f68-b6a4-a440e25fa8e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.119332 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b58b04-cb4f-4f68-b6a4-a440e25fa8e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:08.120819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b58b04-cb4f-4f68-b6a4-a440e25fa8e4 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-87b58b04-cb4f-4f68-b6a4-a440e25fa8e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:08.135311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:08.136401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:08.137399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:08.137399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:08.137399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:08.137399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:08.137399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:08.137399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:08.137399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:08.137399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:08.137399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 303/605] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:08.137934 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbacb6e-2b81-45bf-bd20-f6aeb065ddf0 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.138016 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbacb6e-2b81-45bf-bd20-f6aeb065ddf0 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.138495 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbacb6e-2b81-45bf-bd20-f6aeb065ddf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.155504 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-9bbacb6e-2b81-45bf-bd20-f6aeb065ddf0 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:08.156779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-9bbacb6e-2b81-45bf-bd20-f6aeb065ddf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:08.223163 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 303/606] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:29:07 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/15cb25c883074386a956a0754e12e13f/roles/ec2a4317dc7343208d439d5f2d44d6c3 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:29:08.225119 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03030485-f96f-4324-b964-3d6206ac69f2 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.225553 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03030485-f96f-4324-b964-3d6206ac69f2 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.225764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03030485-f96f-4324-b964-3d6206ac69f2 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.231800 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 304/607] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:08.231953 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 304/608] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:08.235839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e6fcd-0387-4704-b51d-756832d5dcad None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.235839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e6fcd-0387-4704-b51d-756832d5dcad None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.235839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e6fcd-0387-4704-b51d-756832d5dcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.236169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03030485-f96f-4324-b964-3d6206ac69f2 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:08.250811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:08.264827 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-256e6fcd-0387-4704-b51d-756832d5dcad None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:08.266528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-256e6fcd-0387-4704-b51d-756832d5dcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:08.364670 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 305/609] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:29:08.365968 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e6fcd-0387-4704-b51d-756832d5dcad None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:08.373082 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:08.401838 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03030485-f96f-4324-b964-3d6206ac69f2 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:08.454068 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7671b2-fdfd-4144-8a6f-39fa4f2cc287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:08.454752 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7671b2-fdfd-4144-8a6f-39fa4f2cc287 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.455037 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7671b2-fdfd-4144-8a6f-39fa4f2cc287 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.455312 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7671b2-fdfd-4144-8a6f-39fa4f2cc287 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.456155 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7671b2-fdfd-4144-8a6f-39fa4f2cc287 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:08.456850 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7671b2-fdfd-4144-8a6f-39fa4f2cc287 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:08.458068 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7671b2-fdfd-4144-8a6f-39fa4f2cc287 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:08.472170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 305/610] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:08.473126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7671b2-fdfd-4144-8a6f-39fa4f2cc287 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:08.520708 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e6fcd-0387-4704-b51d-756832d5dcad None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:08.557309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c79f7e-5f1d-4543-ab21-4262d4c7363d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:08.558118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79f7e-5f1d-4543-ab21-4262d4c7363d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.558401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79f7e-5f1d-4543-ab21-4262d4c7363d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.558673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c79f7e-5f1d-4543-ab21-4262d4c7363d None admin] PATH_INFO: `/v3/roles/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.565487 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:08.572404 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c79f7e-5f1d-4543-ab21-4262d4c7363d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-65c79f7e-5f1d-4543-ab21-4262d4c7363d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:08.580381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:08.581430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:08.582350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 306/611] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:08.583152 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c79f7e-5f1d-4543-ab21-4262d4c7363d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:08.609242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7671b2-fdfd-4144-8a6f-39fa4f2cc287 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:08.662505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aba708-a315-4de6-ba99-b48acb2d1fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:08.663252 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aba708-a315-4de6-ba99-b48acb2d1fba None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.663796 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aba708-a315-4de6-ba99-b48acb2d1fba None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.664091 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aba708-a315-4de6-ba99-b48acb2d1fba None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.676690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:08.683529 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aba708-a315-4de6-ba99-b48acb2d1fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c4aba708-a315-4de6-ba99-b48acb2d1fba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:08.693781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:08.695134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:08.696231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:08.702720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 306/612] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:08.702720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aba708-a315-4de6-ba99-b48acb2d1fba None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:08.712353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c79f7e-5f1d-4543-ab21-4262d4c7363d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:08.758361 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf2538-f11d-4fe0-b9c8-81ffb567e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:08.759683 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf2538-f11d-4fe0-b9c8-81ffb567e490 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.760185 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf2538-f11d-4fe0-b9c8-81ffb567e490 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.761162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf2538-f11d-4fe0-b9c8-81ffb567e490 None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/b272b57eafcb4de9b85ce8a8a893a2d1/roles/ec2a4317dc7343208d439d5f2d44d6c3` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.782285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf2538-f11d-4fe0-b9c8-81ffb567e490 None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=b272b57eafcb4de9b85ce8a8a893a2d1, role_id=ec2a4317dc7343208d439d5f2d44d6c3)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:08.784715 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf2538-f11d-4fe0-b9c8-81ffb567e490 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:08.817077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aba708-a315-4de6-ba99-b48acb2d1fba None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:08.854645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74f4a8-2613-4e5b-b7ab-4e873fe911e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:08.855458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74f4a8-2613-4e5b-b7ab-4e873fe911e8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:08.855745 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74f4a8-2613-4e5b-b7ab-4e873fe911e8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:08.856023 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f74f4a8-2613-4e5b-b7ab-4e873fe911e8 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:08.857126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74f4a8-2613-4e5b-b7ab-4e873fe911e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:08.857539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74f4a8-2613-4e5b-b7ab-4e873fe911e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:08.859422 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f74f4a8-2613-4e5b-b7ab-4e873fe911e8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:08.872415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 307/613] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:08.872415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74f4a8-2613-4e5b-b7ab-4e873fe911e8 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:08.887763 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 307/614] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:29:08 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/b272b57eafcb4de9b85ce8a8a893a2d1/roles/ec2a4317dc7343208d439d5f2d44d6c3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:29:08.889320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf2538-f11d-4fe0-b9c8-81ffb567e490 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:08.957011 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:09.001920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:09.026076 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf2538-f11d-4fe0-b9c8-81ffb567e490 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:09.070824 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd42dc-2352-4778-bb03-698b804d362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:09.072074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd42dc-2352-4778-bb03-698b804d362d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:09.072569 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd42dc-2352-4778-bb03-698b804d362d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:09.073003 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd42dc-2352-4778-bb03-698b804d362d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:09.094837 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd42dc-2352-4778-bb03-698b804d362d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:09.096686 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd42dc-2352-4778-bb03-698b804d362d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:09.123958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f74f4a8-2613-4e5b-b7ab-4e873fe911e8 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-d6dd42dc-2352-4778-bb03-698b804d362d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:09.138043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:09.139244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:09.140428 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:09.140428 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:09.140428 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:09.140428 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:09.149713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 308/615] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:09.155337 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd42dc-2352-4778-bb03-698b804d362d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:09.182453 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398ec40-233f-4684-a0f0-fa557ba94c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:09.183141 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398ec40-233f-4684-a0f0-fa557ba94c75 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:09.183462 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398ec40-233f-4684-a0f0-fa557ba94c75 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:09.183777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398ec40-233f-4684-a0f0-fa557ba94c75 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:09.185749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398ec40-233f-4684-a0f0-fa557ba94c75 None admin] RBAC: Adding query filter params (name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:09.186071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398ec40-233f-4684-a0f0-fa557ba94c75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:09.189024 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398ec40-233f-4684-a0f0-fa557ba94c75 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:09.199224 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 308/616] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:09.209463 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398ec40-233f-4684-a0f0-fa557ba94c75 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:09.329194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:09.337232 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd42dc-2352-4778-bb03-698b804d362d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:09.379587 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b468c0b-c6c4-4c14-850e-897e1d07c162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:09.380752 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b468c0b-c6c4-4c14-850e-897e1d07c162 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:09.381314 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b468c0b-c6c4-4c14-850e-897e1d07c162 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:09.381810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b468c0b-c6c4-4c14-850e-897e1d07c162 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:09.382771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b468c0b-c6c4-4c14-850e-897e1d07c162 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:09.384765 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b468c0b-c6c4-4c14-850e-897e1d07c162 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:09.386404 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b468c0b-c6c4-4c14-850e-897e1d07c162 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:09.412553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 309/617] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:09.418547 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b468c0b-c6c4-4c14-850e-897e1d07c162 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:09.444182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398ec40-233f-4684-a0f0-fa557ba94c75 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:09.500786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:09.549479 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b468c0b-c6c4-4c14-850e-897e1d07c162 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:09.558021 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f75684-105b-4089-940a-205a0f8ab772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:09.558847 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75684-105b-4089-940a-205a0f8ab772 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:09.559136 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75684-105b-4089-940a-205a0f8ab772 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:09.559403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f75684-105b-4089-940a-205a0f8ab772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:09.588531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80cf80-7872-411d-8cd4-e390dc948e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:09.589356 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80cf80-7872-411d-8cd4-e390dc948e1b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:09.589641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80cf80-7872-411d-8cd4-e390dc948e1b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:09.590067 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80cf80-7872-411d-8cd4-e390dc948e1b None admin] PATH_INFO: `/v3/users/swift` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:09.590757 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f75684-105b-4089-940a-205a0f8ab772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:09.592250 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f75684-105b-4089-940a-205a0f8ab772 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:09.600236 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80cf80-7872-411d-8cd4-e390dc948e1b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:09.601708 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80cf80-7872-411d-8cd4-e390dc948e1b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-95f75684-105b-4089-940a-205a0f8ab772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:09.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:09.626261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:09.627270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:09.627270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:09.627270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:09.627270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-ab80cf80-7872-411d-8cd4-e390dc948e1b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:09.635761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:09.636856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:09.637914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:09.637914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:09.637914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:09.637914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:09.637914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:09.637914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:09.637914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:09.637914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:09.637914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:09.637914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:09.638591 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 309/618] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:09.638591 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f75684-105b-4089-940a-205a0f8ab772 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:09.638752 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 310/619] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:09.645052 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80cf80-7872-411d-8cd4-e390dc948e1b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:09.684643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:09.822248 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80cf80-7872-411d-8cd4-e390dc948e1b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:09.860365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688037df-a1ce-4cd2-972a-ac5b10e55113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:09.861089 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688037df-a1ce-4cd2-972a-ac5b10e55113 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:09.861378 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688037df-a1ce-4cd2-972a-ac5b10e55113 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:09.862387 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688037df-a1ce-4cd2-972a-ac5b10e55113 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:09.863637 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688037df-a1ce-4cd2-972a-ac5b10e55113 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:09.864270 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688037df-a1ce-4cd2-972a-ac5b10e55113 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:09.865802 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688037df-a1ce-4cd2-972a-ac5b10e55113 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:09.886118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 311/620] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:29:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:09.891316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688037df-a1ce-4cd2-972a-ac5b10e55113 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:09.898290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f75684-105b-4089-940a-205a0f8ab772 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:09.978022 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:10.001962 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688037df-a1ce-4cd2-972a-ac5b10e55113 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:10.017647 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46df3a-961c-4fd9-961e-ee9b3d7fa81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:10.018443 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46df3a-961c-4fd9-961e-ee9b3d7fa81c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.018650 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46df3a-961c-4fd9-961e-ee9b3d7fa81c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.018933 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46df3a-961c-4fd9-961e-ee9b3d7fa81c None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:10.019674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46df3a-961c-4fd9-961e-ee9b3d7fa81c None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:10.019990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46df3a-961c-4fd9-961e-ee9b3d7fa81c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:10.021409 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46df3a-961c-4fd9-961e-ee9b3d7fa81c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:10.040909 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc2ebf-d363-4f63-863f-fcee9c66c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:10.043061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc2ebf-d363-4f63-863f-fcee9c66c211 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.043468 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc2ebf-d363-4f63-863f-fcee9c66c211 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.043856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc2ebf-d363-4f63-863f-fcee9c66c211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:10.057659 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc2ebf-d363-4f63-863f-fcee9c66c211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:10.058851 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 310/621] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:10.059539 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc2ebf-d363-4f63-863f-fcee9c66c211 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:10.069476 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46df3a-961c-4fd9-961e-ee9b3d7fa81c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-5acc2ebf-d363-4f63-863f-fcee9c66c211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:10.082931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:10.084027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:10.085077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:10.085077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:10.085077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:10.085077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:10.085077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 312/622] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:10.091486 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc2ebf-d363-4f63-863f-fcee9c66c211 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:10.192952 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:10.207425 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc2ebf-d363-4f63-863f-fcee9c66c211 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:10.240702 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:10.249830 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac9820-3f60-4548-bb8d-a32bc61d1316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:10.250534 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9820-3f60-4548-bb8d-a32bc61d1316 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.250967 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9820-3f60-4548-bb8d-a32bc61d1316 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.251256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac9820-3f60-4548-bb8d-a32bc61d1316 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:10.252004 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac9820-3f60-4548-bb8d-a32bc61d1316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:10.252405 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac9820-3f60-4548-bb8d-a32bc61d1316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:10.253891 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac9820-3f60-4548-bb8d-a32bc61d1316 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:10.268868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 313/623] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:10.271101 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ee516-6aa1-414a-a11f-64a4ca5ea3dc None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.271381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ee516-6aa1-414a-a11f-64a4ca5ea3dc None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.271666 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ee516-6aa1-414a-a11f-64a4ca5ea3dc None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:10.277698 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 314/624] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:10.277698 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ee516-6aa1-414a-a11f-64a4ca5ea3dc None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:10.326831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46df3a-961c-4fd9-961e-ee9b3d7fa81c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:10.376449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0c02b-9442-418c-8c56-a96bcb3ed3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:10.377222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0c02b-9442-418c-8c56-a96bcb3ed3c3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.377593 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0c02b-9442-418c-8c56-a96bcb3ed3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.377876 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0c02b-9442-418c-8c56-a96bcb3ed3c3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:10.385668 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0c02b-9442-418c-8c56-a96bcb3ed3c3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:10.387121 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0c02b-9442-418c-8c56-a96bcb3ed3c3 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:10.401475 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ee516-6aa1-414a-a11f-64a4ca5ea3dc None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c3b0c02b-9442-418c-8c56-a96bcb3ed3c3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:10.403352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:10.404410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:10.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:10.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:10.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:10.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:10.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:10.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:10.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:10.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:10.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:29:10.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:10.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 311/625] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:29:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:10.406591 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a7ec-9e65-4fdb-a990-0a4cdf0b2ffb None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.406765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a7ec-9e65-4fdb-a990-0a4cdf0b2ffb None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.408424 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58a7ec-9e65-4fdb-a990-0a4cdf0b2ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:10.429596 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-9c58a7ec-9e65-4fdb-a990-0a4cdf0b2ffb None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:10.430960 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-9c58a7ec-9e65-4fdb-a990-0a4cdf0b2ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:10.439526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434933c-c524-43a6-8042-8babd22ae2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:10.440541 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434933c-c524-43a6-8042-8babd22ae2ce None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.440830 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434933c-c524-43a6-8042-8babd22ae2ce None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.441112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434933c-c524-43a6-8042-8babd22ae2ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:10.459858 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434933c-c524-43a6-8042-8babd22ae2ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:10.461451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434933c-c524-43a6-8042-8babd22ae2ce None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-9434933c-c524-43a6-8042-8babd22ae2ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:10.477629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:10.479937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:10.482432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:10.482432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:10.482432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:10.482432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:10.482432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:10.482432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:10.482432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:10.482432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:10.482432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 315/626] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:10.482432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434933c-c524-43a6-8042-8babd22ae2ce None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:10.507230 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 312/627] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:10.508240 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58a7ec-9e65-4fdb-a990-0a4cdf0b2ffb None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:10.531339 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:10.538537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:10.593526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434933c-c524-43a6-8042-8babd22ae2ce None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:10.626805 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58a7ec-9e65-4fdb-a990-0a4cdf0b2ffb None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:10.632062 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2791f8-a51f-4790-b067-e8e609d43602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:10.633000 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2791f8-a51f-4790-b067-e8e609d43602 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.633451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2791f8-a51f-4790-b067-e8e609d43602 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.633887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2791f8-a51f-4790-b067-e8e609d43602 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:10.634909 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2791f8-a51f-4790-b067-e8e609d43602 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:10.635493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2791f8-a51f-4790-b067-e8e609d43602 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:10.637184 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2791f8-a51f-4790-b067-e8e609d43602 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:10.657699 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 316/628] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:29:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:10.659574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2791f8-a51f-4790-b067-e8e609d43602 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:10.666162 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872359a5-3416-429f-8ef5-4b56d6fa2e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:10.667147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872359a5-3416-429f-8ef5-4b56d6fa2e94 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.667425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872359a5-3416-429f-8ef5-4b56d6fa2e94 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.667702 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872359a5-3416-429f-8ef5-4b56d6fa2e94 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:10.668552 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872359a5-3416-429f-8ef5-4b56d6fa2e94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:10.668870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872359a5-3416-429f-8ef5-4b56d6fa2e94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:10.670327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872359a5-3416-429f-8ef5-4b56d6fa2e94 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:10.683878 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 313/629] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:10.684822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872359a5-3416-429f-8ef5-4b56d6fa2e94 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:10.774014 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2791f8-a51f-4790-b067-e8e609d43602 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:10.777864 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:10.779014 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:10.796308 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872359a5-3416-429f-8ef5-4b56d6fa2e94 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:10.812247 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06902ca2-f5f3-41e2-962e-68a5e781951a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:10.813086 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06902ca2-f5f3-41e2-962e-68a5e781951a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.813380 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06902ca2-f5f3-41e2-962e-68a5e781951a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.813924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06902ca2-f5f3-41e2-962e-68a5e781951a None admin] PATH_INFO: `/v3/roles/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:10.824687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06902ca2-f5f3-41e2-962e-68a5e781951a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:10.831551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489972f7-7d31-4ae1-aaf2-f0dc4fd180b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:10.832300 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489972f7-7d31-4ae1-aaf2-f0dc4fd180b9 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.832529 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489972f7-7d31-4ae1-aaf2-f0dc4fd180b9 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.832809 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489972f7-7d31-4ae1-aaf2-f0dc4fd180b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-06902ca2-f5f3-41e2-962e-68a5e781951a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:10.833029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:10.834007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:10.834928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 317/630] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:10.835672 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06902ca2-f5f3-41e2-962e-68a5e781951a None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:10.916505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489972f7-7d31-4ae1-aaf2-f0dc4fd180b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:10.918068 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489972f7-7d31-4ae1-aaf2-f0dc4fd180b9 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:10.956110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06902ca2-f5f3-41e2-962e-68a5e781951a None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:10.992635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124333f-4f3c-4e71-bd37-6a0c1f338156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:10.993531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124333f-4f3c-4e71-bd37-6a0c1f338156 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:10.993840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124333f-4f3c-4e71-bd37-6a0c1f338156 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:10.994812 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124333f-4f3c-4e71-bd37-6a0c1f338156 None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/14034b78ec5e4999848a3fffb2964587/roles/ec2a4317dc7343208d439d5f2d44d6c3` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.010168 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124333f-4f3c-4e71-bd37-6a0c1f338156 None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=14034b78ec5e4999848a3fffb2964587, role_id=ec2a4317dc7343208d439d5f2d44d6c3)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:11.010516 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:11.011605 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124333f-4f3c-4e71-bd37-6a0c1f338156 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:11.027797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 314/631] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:29:11.029254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7b662-ab75-4f51-94ea-802445dd0513 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.030093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7b662-ab75-4f51-94ea-802445dd0513 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.030383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7b662-ab75-4f51-94ea-802445dd0513 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.031831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 315/632] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:11.032658 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7b662-ab75-4f51-94ea-802445dd0513 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:11.079575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 318/633] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:29:10 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/14034b78ec5e4999848a3fffb2964587/roles/ec2a4317dc7343208d439d5f2d44d6c3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:29:11.081462 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124333f-4f3c-4e71-bd37-6a0c1f338156 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:11.099315 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:11.175811 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7b662-ab75-4f51-94ea-802445dd0513 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:11.205450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124333f-4f3c-4e71-bd37-6a0c1f338156 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:11.219744 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d3890-42e5-487f-9e3a-b2a2b13e772a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:11.220867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d3890-42e5-487f-9e3a-b2a2b13e772a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.221449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d3890-42e5-487f-9e3a-b2a2b13e772a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.221910 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d3890-42e5-487f-9e3a-b2a2b13e772a None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.222864 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d3890-42e5-487f-9e3a-b2a2b13e772a None admin] RBAC: Adding query filter params (name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:11.223385 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d3890-42e5-487f-9e3a-b2a2b13e772a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:11.225210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26d3890-42e5-487f-9e3a-b2a2b13e772a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:11.235747 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 316/634] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:11.238030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475cc99-c144-4cb0-8f3d-92daf701c6e3 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.238521 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475cc99-c144-4cb0-8f3d-92daf701c6e3 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.238967 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475cc99-c144-4cb0-8f3d-92daf701c6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.255552 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944ce2f-a792-4feb-9bac-b26ead5189ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:11.256648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944ce2f-a792-4feb-9bac-b26ead5189ff None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.257009 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944ce2f-a792-4feb-9bac-b26ead5189ff None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.257356 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944ce2f-a792-4feb-9bac-b26ead5189ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.259288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-8475cc99-c144-4cb0-8f3d-92daf701c6e3 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:11.260966 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-8475cc99-c144-4cb0-8f3d-92daf701c6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:11.270432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944ce2f-a792-4feb-9bac-b26ead5189ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:11.271624 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944ce2f-a792-4feb-9bac-b26ead5189ff None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-8944ce2f-a792-4feb-9bac-b26ead5189ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:11.291846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:11.292958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:11.294390 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:11.294390 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:11.294390 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:11.294390 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:11.294390 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 319/635] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:11.294390 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944ce2f-a792-4feb-9bac-b26ead5189ff None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:11.323148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:11.363409 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 317/636] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:11.364841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475cc99-c144-4cb0-8f3d-92daf701c6e3 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:11.393869 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:11.431431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944ce2f-a792-4feb-9bac-b26ead5189ff None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:11.474160 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0535f12-614e-4f41-aba3-6346771fcc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:11.475246 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0535f12-614e-4f41-aba3-6346771fcc12 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.475687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0535f12-614e-4f41-aba3-6346771fcc12 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.476129 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0535f12-614e-4f41-aba3-6346771fcc12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.489162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0535f12-614e-4f41-aba3-6346771fcc12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:11.490854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0535f12-614e-4f41-aba3-6346771fcc12 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:11.492004 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8475cc99-c144-4cb0-8f3d-92daf701c6e3 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-e0535f12-614e-4f41-aba3-6346771fcc12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:11.511097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:11.512159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:11.513221 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:11.513221 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:11.513221 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:11.513221 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:11.514096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 320/637] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:11.515394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0535f12-614e-4f41-aba3-6346771fcc12 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:11.537582 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e8508-c267-40c5-b367-689fb6788641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:11.538386 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e8508-c267-40c5-b367-689fb6788641 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.538670 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e8508-c267-40c5-b367-689fb6788641 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.539148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e8508-c267-40c5-b367-689fb6788641 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.540007 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e8508-c267-40c5-b367-689fb6788641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:11.540318 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e8508-c267-40c5-b367-689fb6788641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:11.542035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e8508-c267-40c5-b367-689fb6788641 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:11.554216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 318/638] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:11.555489 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7e199-7d82-48ae-af37-5eb64cd1af5c None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.555777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7e199-7d82-48ae-af37-5eb64cd1af5c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.556058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7e199-7d82-48ae-af37-5eb64cd1af5c None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.560389 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 319/639] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:11.560389 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7e199-7d82-48ae-af37-5eb64cd1af5c None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:11.596668 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:11.640236 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0535f12-614e-4f41-aba3-6346771fcc12 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:11.660959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:11.665738 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7e199-7d82-48ae-af37-5eb64cd1af5c None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:11.682320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ac8c1-5364-4195-95da-4abe74f4432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:11.683039 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ac8c1-5364-4195-95da-4abe74f4432f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.683318 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ac8c1-5364-4195-95da-4abe74f4432f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.683590 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ac8c1-5364-4195-95da-4abe74f4432f None admin] PATH_INFO: `/v3/roles/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.693971 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ac8c1-5364-4195-95da-4abe74f4432f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-f46ac8c1-5364-4195-95da-4abe74f4432f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:11.702128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:11.703342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:11.704285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:11.706678 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f5a83-0a2f-4fda-8cd0-6ce87063bcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:11.711642 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 321/640] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:11.711779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5a83-0a2f-4fda-8cd0-6ce87063bcb1 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.711779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5a83-0a2f-4fda-8cd0-6ce87063bcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.711779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5a83-0a2f-4fda-8cd0-6ce87063bcb1 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.712067 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ac8c1-5364-4195-95da-4abe74f4432f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:11.712160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f5a83-0a2f-4fda-8cd0-6ce87063bcb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:11.712160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f5a83-0a2f-4fda-8cd0-6ce87063bcb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:11.712160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f5a83-0a2f-4fda-8cd0-6ce87063bcb1 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:11.725247 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 320/641] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:11.726383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318049f1-65f4-4518-a133-d0ebd44188a3 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.726674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318049f1-65f4-4518-a133-d0ebd44188a3 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.726945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318049f1-65f4-4518-a133-d0ebd44188a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.743903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-318049f1-65f4-4518-a133-d0ebd44188a3 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:11.745112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-318049f1-65f4-4518-a133-d0ebd44188a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:11.826785 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 321/642] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:11.827946 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318049f1-65f4-4518-a133-d0ebd44188a3 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:11.836543 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ac8c1-5364-4195-95da-4abe74f4432f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:11.868412 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:11.872913 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0028a0-9b60-4ece-851c-76a2c203cc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:11.873981 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0028a0-9b60-4ece-851c-76a2c203cc4b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.874462 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0028a0-9b60-4ece-851c-76a2c203cc4b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.874968 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0028a0-9b60-4ece-851c-76a2c203cc4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.883011 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0028a0-9b60-4ece-851c-76a2c203cc4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:11.884783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0028a0-9b60-4ece-851c-76a2c203cc4b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-4d0028a0-9b60-4ece-851c-76a2c203cc4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:11.901096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:11.902140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:11.903093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:11.903093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:11.903093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:11.903093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:11.903093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:11.903093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:11.903093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:11.903093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:11.903093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 322/643] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:11.903642 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0028a0-9b60-4ece-851c-76a2c203cc4b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:11.947411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:11.954542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318049f1-65f4-4518-a133-d0ebd44188a3 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:11.993034 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65532d-e39e-47b2-913a-dbef833cdaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:11.994371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65532d-e39e-47b2-913a-dbef833cdaf8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:11.995021 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65532d-e39e-47b2-913a-dbef833cdaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:11.995509 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65532d-e39e-47b2-913a-dbef833cdaf8 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:11.996767 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65532d-e39e-47b2-913a-dbef833cdaf8 None admin] RBAC: Adding query filter params (name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:11.997272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65532d-e39e-47b2-913a-dbef833cdaf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:11.999287 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65532d-e39e-47b2-913a-dbef833cdaf8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:12.011009 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0028a0-9b60-4ece-851c-76a2c203cc4b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.015329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 322/644] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:12.016456 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65532d-e39e-47b2-913a-dbef833cdaf8 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:12.048130 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e0a5c-631b-444c-8194-bc8e400fa16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.048949 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e0a5c-631b-444c-8194-bc8e400fa16d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.049782 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e0a5c-631b-444c-8194-bc8e400fa16d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.050039 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e0a5c-631b-444c-8194-bc8e400fa16d None admin] PATH_INFO: `/v3/users/glance` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.057410 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e0a5c-631b-444c-8194-bc8e400fa16d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:12.058844 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e0a5c-631b-444c-8194-bc8e400fa16d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-066e0a5c-631b-444c-8194-bc8e400fa16d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:12.075277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:12.076357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:12.077528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:12.077528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:12.077528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:12.077528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:12.077528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:12.077528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:12.077528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:12.077528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:12.077528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:29:12.077528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:12.079511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 323/645] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:29:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:12.080528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e0a5c-631b-444c-8194-bc8e400fa16d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:12.128429 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:12.141339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65532d-e39e-47b2-913a-dbef833cdaf8 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.178810 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228d92b-380b-4a0c-a90f-8b2377b18f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.179916 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228d92b-380b-4a0c-a90f-8b2377b18f76 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.180255 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228d92b-380b-4a0c-a90f-8b2377b18f76 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.180508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228d92b-380b-4a0c-a90f-8b2377b18f76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.183562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:12.186972 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e0a5c-631b-444c-8194-bc8e400fa16d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.193061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228d92b-380b-4a0c-a90f-8b2377b18f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-0228d92b-380b-4a0c-a90f-8b2377b18f76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:12.201221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:12.202207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:12.203142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 323/646] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:12.203910 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228d92b-380b-4a0c-a90f-8b2377b18f76 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:12.228310 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81c07a-2fb8-4742-bb63-8ddcc7df968f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.229018 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81c07a-2fb8-4742-bb63-8ddcc7df968f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.229297 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81c07a-2fb8-4742-bb63-8ddcc7df968f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.229599 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81c07a-2fb8-4742-bb63-8ddcc7df968f None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.230348 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81c07a-2fb8-4742-bb63-8ddcc7df968f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:12.230649 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81c07a-2fb8-4742-bb63-8ddcc7df968f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:12.232065 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81c07a-2fb8-4742-bb63-8ddcc7df968f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:12.250050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 324/647] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:12.250050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81c07a-2fb8-4742-bb63-8ddcc7df968f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:12.322070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228d92b-380b-4a0c-a90f-8b2377b18f76 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.368925 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2037a3b-4c02-411e-9b3a-97b006fefdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.369444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81c07a-2fb8-4742-bb63-8ddcc7df968f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.369740 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2037a3b-4c02-411e-9b3a-97b006fefdf7 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.370025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2037a3b-4c02-411e-9b3a-97b006fefdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.370298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2037a3b-4c02-411e-9b3a-97b006fefdf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.382377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2037a3b-4c02-411e-9b3a-97b006fefdf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:12.383832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2037a3b-4c02-411e-9b3a-97b006fefdf7 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:12.392147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-d2037a3b-4c02-411e-9b3a-97b006fefdf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:12.402511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:12.403488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:12.404492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:12.404492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:12.404492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:12.404492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:12.404492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 324/648] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:12.405434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2037a3b-4c02-411e-9b3a-97b006fefdf7 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:12.413062 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373f8ca-1400-4d55-8239-bc6b65132871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.413750 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373f8ca-1400-4d55-8239-bc6b65132871 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.414270 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373f8ca-1400-4d55-8239-bc6b65132871 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.414383 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373f8ca-1400-4d55-8239-bc6b65132871 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.426735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373f8ca-1400-4d55-8239-bc6b65132871 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:12.426735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373f8ca-1400-4d55-8239-bc6b65132871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:12.426735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373f8ca-1400-4d55-8239-bc6b65132871 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:12.440749 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 325/649] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:29:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:12.440749 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373f8ca-1400-4d55-8239-bc6b65132871 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:12.460939 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:12.531618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2037a3b-4c02-411e-9b3a-97b006fefdf7 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.551506 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373f8ca-1400-4d55-8239-bc6b65132871 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.571239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd502d95-6ecc-4963-85b1-843914616d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.571920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd502d95-6ecc-4963-85b1-843914616d30 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.572344 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd502d95-6ecc-4963-85b1-843914616d30 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.572653 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd502d95-6ecc-4963-85b1-843914616d30 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.573418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd502d95-6ecc-4963-85b1-843914616d30 None admin] RBAC: Adding query filter params (name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:12.573731 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd502d95-6ecc-4963-85b1-843914616d30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:12.575136 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd502d95-6ecc-4963-85b1-843914616d30 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:12.591713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 325/650] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:12.591713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd502d95-6ecc-4963-85b1-843914616d30 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:12.595482 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb1d36-54c8-40aa-adae-9eff0ec466b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.596398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb1d36-54c8-40aa-adae-9eff0ec466b2 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.596712 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb1d36-54c8-40aa-adae-9eff0ec466b2 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.596957 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb1d36-54c8-40aa-adae-9eff0ec466b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.598674 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb1d36-54c8-40aa-adae-9eff0ec466b2 None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=e6a982ae15a6443b9b52529df47aa837) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:12.598993 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb1d36-54c8-40aa-adae-9eff0ec466b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:12.600735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb1d36-54c8-40aa-adae-9eff0ec466b2 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:12.616725 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 326/651] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:12 2022] GET /identity/v3/role_assignments?user.id=e6a982ae15a6443b9b52529df47aa837&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:12.616725 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb1d36-54c8-40aa-adae-9eff0ec466b2 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:12.680785 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:12.716727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd502d95-6ecc-4963-85b1-843914616d30 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.742783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:12.758328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50578e-57cc-44dc-9c1d-4fcc3f9012b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.759476 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50578e-57cc-44dc-9c1d-4fcc3f9012b1 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.759967 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50578e-57cc-44dc-9c1d-4fcc3f9012b1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.760486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50578e-57cc-44dc-9c1d-4fcc3f9012b1 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.761457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50578e-57cc-44dc-9c1d-4fcc3f9012b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:12.762004 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50578e-57cc-44dc-9c1d-4fcc3f9012b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:12.763750 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50578e-57cc-44dc-9c1d-4fcc3f9012b1 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:12.769657 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb1d36-54c8-40aa-adae-9eff0ec466b2 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.779572 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 326/652] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:12.780922 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50578e-57cc-44dc-9c1d-4fcc3f9012b1 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:12.812558 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce842bb-e175-4a43-a6b3-1169d3483145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.813235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce842bb-e175-4a43-a6b3-1169d3483145 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.813527 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce842bb-e175-4a43-a6b3-1169d3483145 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.813812 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce842bb-e175-4a43-a6b3-1169d3483145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.907176 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50578e-57cc-44dc-9c1d-4fcc3f9012b1 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:12.941147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:12.952976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bdacd-db56-4490-a4b5-91c802c2a785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:12.953787 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bdacd-db56-4490-a4b5-91c802c2a785 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:12.956309 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bdacd-db56-4490-a4b5-91c802c2a785 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:12.956309 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bdacd-db56-4490-a4b5-91c802c2a785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:12.967844 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600bdacd-db56-4490-a4b5-91c802c2a785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:12.969777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600bdacd-db56-4490-a4b5-91c802c2a785 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-600bdacd-db56-4490-a4b5-91c802c2a785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:12.991234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:12.992272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:12.993388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:12.993388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:12.993388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:12.993388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:12.993388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 327/653] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:12.993722 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bdacd-db56-4490-a4b5-91c802c2a785 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:13.028219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce842bb-e175-4a43-a6b3-1169d3483145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:13.031102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce842bb-e175-4a43-a6b3-1169d3483145 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:13.110876 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bdacd-db56-4490-a4b5-91c802c2a785 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:13.136140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:13.155578 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccecfe-63e2-47eb-b7b7-c45449ffc503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:13.157131 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccecfe-63e2-47eb-b7b7-c45449ffc503 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:13.157639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccecfe-63e2-47eb-b7b7-c45449ffc503 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:13.158144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccecfe-63e2-47eb-b7b7-c45449ffc503 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:13.160586 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 327/654] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:29:13.161597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce842bb-e175-4a43-a6b3-1169d3483145 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:13.168830 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccecfe-63e2-47eb-b7b7-c45449ffc503 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:13.170667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccecfe-63e2-47eb-b7b7-c45449ffc503 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-ecccecfe-63e2-47eb-b7b7-c45449ffc503 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:13.188849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:13.190002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:13.191042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:13.191042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:13.191042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:13.191042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:13.191042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:13.191042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:13.191042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:13.191042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:13.191042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:29:13.191042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:13.193815 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 328/655] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:29:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:13.194975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccecfe-63e2-47eb-b7b7-c45449ffc503 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:13.234848 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:13.285866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce842bb-e175-4a43-a6b3-1169d3483145 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:13.307682 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccecfe-63e2-47eb-b7b7-c45449ffc503 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:13.324720 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9496b26-269e-4b9c-8597-fdd7a1940087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:13.325399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9496b26-269e-4b9c-8597-fdd7a1940087 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:13.325736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9496b26-269e-4b9c-8597-fdd7a1940087 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:13.326008 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9496b26-269e-4b9c-8597-fdd7a1940087 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:13.326790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9496b26-269e-4b9c-8597-fdd7a1940087 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:13.327251 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9496b26-269e-4b9c-8597-fdd7a1940087 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:13.328764 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9496b26-269e-4b9c-8597-fdd7a1940087 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:13.346564 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 328/656] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:13.347255 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7087ce-d057-4d65-9074-d70085d62695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:13.347489 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9496b26-269e-4b9c-8597-fdd7a1940087 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:13.348558 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7087ce-d057-4d65-9074-d70085d62695 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:13.349080 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7087ce-d057-4d65-9074-d70085d62695 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:13.349581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7087ce-d057-4d65-9074-d70085d62695 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:13.361547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7087ce-d057-4d65-9074-d70085d62695 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:13.363960 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7087ce-d057-4d65-9074-d70085d62695 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-ca7087ce-d057-4d65-9074-d70085d62695 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:13.389025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:13.390074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:13.391075 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:13.391075 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:13.391075 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:13.391075 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:13.393644 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 329/657] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:13.394684 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7087ce-d057-4d65-9074-d70085d62695 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:13.396988 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:13.465252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9496b26-269e-4b9c-8597-fdd7a1940087 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:13.511256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b13499-5b30-4690-925f-2873b4aec1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:13.511256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13499-5b30-4690-925f-2873b4aec1c7 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:13.511256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13499-5b30-4690-925f-2873b4aec1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:13.511256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b13499-5b30-4690-925f-2873b4aec1c7 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:13.511710 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b13499-5b30-4690-925f-2873b4aec1c7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:13.511710 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b13499-5b30-4690-925f-2873b4aec1c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:13.511710 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b13499-5b30-4690-925f-2873b4aec1c7 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:13.535177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 329/658] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:29:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:13.535177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced3e36-b1c3-45f2-b968-5dba50c15e3b None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:13.535177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced3e36-b1c3-45f2-b968-5dba50c15e3b None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:13.535177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced3e36-b1c3-45f2-b968-5dba50c15e3b None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:13.538517 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 330/659] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:13.539439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced3e36-b1c3-45f2-b968-5dba50c15e3b None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:13.610246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:13.625687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7087ce-d057-4d65-9074-d70085d62695 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:13.653159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:13.659061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced3e36-b1c3-45f2-b968-5dba50c15e3b None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:13.664035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29a623-95ff-42c2-9df3-d13b4abc3db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:13.664926 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29a623-95ff-42c2-9df3-d13b4abc3db5 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:13.665687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29a623-95ff-42c2-9df3-d13b4abc3db5 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:13.665687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29a623-95ff-42c2-9df3-d13b4abc3db5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:13.673985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29a623-95ff-42c2-9df3-d13b4abc3db5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:13.675422 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29a623-95ff-42c2-9df3-d13b4abc3db5 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-9d29a623-95ff-42c2-9df3-d13b4abc3db5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:13.689687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:13.690950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:13.692010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:13.692010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:13.692010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:13.692010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:13.692010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:13.692010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:13.692010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:13.692010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:13.692010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:29:13.692010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:13.692010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 330/660] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:13.692702 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29a623-95ff-42c2-9df3-d13b4abc3db5 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:13.699973 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184083e-d3a0-40e3-96b7-b88a0668daac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:13.700689 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184083e-d3a0-40e3-96b7-b88a0668daac None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:13.701000 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184083e-d3a0-40e3-96b7-b88a0668daac None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:13.701256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184083e-d3a0-40e3-96b7-b88a0668daac None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:13.701987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184083e-d3a0-40e3-96b7-b88a0668daac None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:13.702296 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184083e-d3a0-40e3-96b7-b88a0668daac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:13.703705 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184083e-d3a0-40e3-96b7-b88a0668daac None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:13.717205 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 331/661] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:13.718517 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2707-4376-4d0c-84ab-1ef2211aeabd None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:13.718808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2707-4376-4d0c-84ab-1ef2211aeabd None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:13.719085 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca2707-4376-4d0c-84ab-1ef2211aeabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:13.735618 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-09ca2707-4376-4d0c-84ab-1ef2211aeabd None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:13.736862 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-09ca2707-4376-4d0c-84ab-1ef2211aeabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:13.808225 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29a623-95ff-42c2-9df3-d13b4abc3db5 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:13.811161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 332/662] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:13.817696 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca2707-4376-4d0c-84ab-1ef2211aeabd None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:13.845740 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988ac2d-f092-4646-8aa3-ec8fa36f7237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:13.847185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988ac2d-f092-4646-8aa3-ec8fa36f7237 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:13.847544 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988ac2d-f092-4646-8aa3-ec8fa36f7237 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:13.847746 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988ac2d-f092-4646-8aa3-ec8fa36f7237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:13.866111 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:13.920268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:13.925131 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988ac2d-f092-4646-8aa3-ec8fa36f7237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:13.926650 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988ac2d-f092-4646-8aa3-ec8fa36f7237 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:13.936938 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca2707-4376-4d0c-84ab-1ef2211aeabd None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:13.981444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9510c8a-d8c1-4eb7-b716-94efb86eaa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:13.982129 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9510c8a-d8c1-4eb7-b716-94efb86eaa8b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:13.982437 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9510c8a-d8c1-4eb7-b716-94efb86eaa8b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:13.982767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9510c8a-d8c1-4eb7-b716-94efb86eaa8b None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:13.983614 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9510c8a-d8c1-4eb7-b716-94efb86eaa8b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:13.983924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9510c8a-d8c1-4eb7-b716-94efb86eaa8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:13.985394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9510c8a-d8c1-4eb7-b716-94efb86eaa8b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:14.004394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 333/663] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 20:29:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:14.005334 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9510c8a-d8c1-4eb7-b716-94efb86eaa8b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:14.041453 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 331/664] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:29:14.045248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988ac2d-f092-4646-8aa3-ec8fa36f7237 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:14.099128 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:14.116810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9510c8a-d8c1-4eb7-b716-94efb86eaa8b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:14.152836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6ba5b-baff-4dfd-a43a-9e97ff8cacac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:14.153635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6ba5b-baff-4dfd-a43a-9e97ff8cacac None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:14.153919 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6ba5b-baff-4dfd-a43a-9e97ff8cacac None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:14.154192 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6ba5b-baff-4dfd-a43a-9e97ff8cacac None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:14.155113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988ac2d-f092-4646-8aa3-ec8fa36f7237 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:14.161603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6ba5b-baff-4dfd-a43a-9e97ff8cacac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:14.163034 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6ba5b-baff-4dfd-a43a-9e97ff8cacac None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-a0f6ba5b-baff-4dfd-a43a-9e97ff8cacac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:14.177929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:14.178905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:14.179880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:14.179880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:14.179880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:14.179880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:14.179880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:14.179880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:14.179880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:14.179880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:14.179880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 334/665] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:14.179880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6ba5b-baff-4dfd-a43a-9e97ff8cacac None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:14.192829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:14.193186 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73a37-ed69-4ad4-aea4-03fd4ed70c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:14.194456 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73a37-ed69-4ad4-aea4-03fd4ed70c38 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:14.194741 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73a37-ed69-4ad4-aea4-03fd4ed70c38 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:14.195016 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73a37-ed69-4ad4-aea4-03fd4ed70c38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:14.208689 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb73a37-ed69-4ad4-aea4-03fd4ed70c38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-5eb73a37-ed69-4ad4-aea4-03fd4ed70c38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:14.218156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:14.219288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:14.220242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:14.222010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 332/666] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:14.222377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73a37-ed69-4ad4-aea4-03fd4ed70c38 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:14.296367 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6ba5b-baff-4dfd-a43a-9e97ff8cacac None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:14.332487 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf441e5-c6f7-468e-b9c6-747c6decc7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:14.333153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf441e5-c6f7-468e-b9c6-747c6decc7fb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:14.333440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf441e5-c6f7-468e-b9c6-747c6decc7fb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:14.333737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf441e5-c6f7-468e-b9c6-747c6decc7fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:14.336818 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73a37-ed69-4ad4-aea4-03fd4ed70c38 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:14.345049 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:14.374324 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff0474-ea32-4588-94d4-67181c34f9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:14.375041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff0474-ea32-4588-94d4-67181c34f9b9 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:14.375324 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff0474-ea32-4588-94d4-67181c34f9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:14.375670 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff0474-ea32-4588-94d4-67181c34f9b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:14.386990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff0474-ea32-4588-94d4-67181c34f9b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:14.389560 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff0474-ea32-4588-94d4-67181c34f9b9 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-faff0474-ea32-4588-94d4-67181c34f9b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:14.410035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:14.411077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:14.412085 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:14.412085 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:14.412085 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:14.412085 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:14.412085 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 333/667] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:14.412085 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff0474-ea32-4588-94d4-67181c34f9b9 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:14.419676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf441e5-c6f7-468e-b9c6-747c6decc7fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:14.421153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf441e5-c6f7-468e-b9c6-747c6decc7fb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:14.430668 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:14.528376 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 335/668] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:29:14.528860 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf441e5-c6f7-468e-b9c6-747c6decc7fb None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:14.537870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff0474-ea32-4588-94d4-67181c34f9b9 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:14.572928 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240c6e4-1783-4a43-a879-9c5980f0c5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:14.573597 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240c6e4-1783-4a43-a879-9c5980f0c5ef None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:14.573884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240c6e4-1783-4a43-a879-9c5980f0c5ef None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:14.574162 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240c6e4-1783-4a43-a879-9c5980f0c5ef None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:14.574952 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240c6e4-1783-4a43-a879-9c5980f0c5ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:14.575361 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240c6e4-1783-4a43-a879-9c5980f0c5ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:14.576857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240c6e4-1783-4a43-a879-9c5980f0c5ef None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:14.590708 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 334/669] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:14.591321 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240c6e4-1783-4a43-a879-9c5980f0c5ef None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:14.614787 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:14.645449 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf441e5-c6f7-468e-b9c6-747c6decc7fb None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:14.653827 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:14.681761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0ece2-07fe-4484-8983-8789d89ce7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:14.682520 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ece2-07fe-4484-8983-8789d89ce7bf None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:14.682866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ece2-07fe-4484-8983-8789d89ce7bf None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:14.683211 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ece2-07fe-4484-8983-8789d89ce7bf None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:14.684039 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0ece2-07fe-4484-8983-8789d89ce7bf None admin] RBAC: Adding query filter params (name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:14.684447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0ece2-07fe-4484-8983-8789d89ce7bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:14.686175 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0ece2-07fe-4484-8983-8789d89ce7bf None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:14.700189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 336/670] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:14.700189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0ece2-07fe-4484-8983-8789d89ce7bf None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:14.709233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240c6e4-1783-4a43-a879-9c5980f0c5ef None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:14.746471 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcaae9-2873-45bb-a57d-14b7b627dabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:14.747235 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcaae9-2873-45bb-a57d-14b7b627dabb None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:14.747588 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcaae9-2873-45bb-a57d-14b7b627dabb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:14.747934 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcaae9-2873-45bb-a57d-14b7b627dabb None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:14.748774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfcaae9-2873-45bb-a57d-14b7b627dabb None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:14.749155 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfcaae9-2873-45bb-a57d-14b7b627dabb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:14.750626 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfcaae9-2873-45bb-a57d-14b7b627dabb None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:14.768411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 335/671] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:14.768411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcaae9-2873-45bb-a57d-14b7b627dabb None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:14.808687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0ece2-07fe-4484-8983-8789d89ce7bf None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:14.845932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510cb46-3c72-4f76-920d-a5ca0369447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:14.846826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510cb46-3c72-4f76-920d-a5ca0369447d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:14.847200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510cb46-3c72-4f76-920d-a5ca0369447d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:14.847543 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510cb46-3c72-4f76-920d-a5ca0369447d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:14.859087 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:14.862761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510cb46-3c72-4f76-920d-a5ca0369447d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:14.864372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a510cb46-3c72-4f76-920d-a5ca0369447d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-a510cb46-3c72-4f76-920d-a5ca0369447d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:14.883997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:14.885388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:14.886666 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:14.886666 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:14.886666 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:14.886666 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:14.886985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfcaae9-2873-45bb-a57d-14b7b627dabb None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:14.888424 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 337/672] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:14.889505 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510cb46-3c72-4f76-920d-a5ca0369447d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:14.913870 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:14.928872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9af242-2381-4dc8-93b2-b1c6a21758d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:14.929543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9af242-2381-4dc8-93b2-b1c6a21758d4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:14.929832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9af242-2381-4dc8-93b2-b1c6a21758d4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:14.930114 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9af242-2381-4dc8-93b2-b1c6a21758d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:14.930961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9af242-2381-4dc8-93b2-b1c6a21758d4 None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=c6ace7ef9bfe4837b87c8576862abb75) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:14.931278 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9af242-2381-4dc8-93b2-b1c6a21758d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:14.932812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9af242-2381-4dc8-93b2-b1c6a21758d4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:14.948670 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 336/673] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:14 2022] GET /identity/v3/role_assignments?user.id=c6ace7ef9bfe4837b87c8576862abb75&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:14.948670 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9af242-2381-4dc8-93b2-b1c6a21758d4 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:15.026343 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a510cb46-3c72-4f76-920d-a5ca0369447d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:15.070226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a1088-b260-4451-9358-be9aac344426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:15.071048 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a1088-b260-4451-9358-be9aac344426 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:15.071333 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a1088-b260-4451-9358-be9aac344426 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:15.071613 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a1088-b260-4451-9358-be9aac344426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:15.080366 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9af242-2381-4dc8-93b2-b1c6a21758d4 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:15.083171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a1088-b260-4451-9358-be9aac344426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:15.084681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a1088-b260-4451-9358-be9aac344426 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-455a1088-b260-4451-9358-be9aac344426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:15.104106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:15.105214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:15.106189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:15.106189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:15.106189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:15.106189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:15.106189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 338/674] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:15.107192 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a1088-b260-4451-9358-be9aac344426 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:15.123603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6e4a4-8637-47bd-b676-3577518b93e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:15.124460 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6e4a4-8637-47bd-b676-3577518b93e4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:15.124743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6e4a4-8637-47bd-b676-3577518b93e4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:15.125017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6e4a4-8637-47bd-b676-3577518b93e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:15.133214 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6e4a4-8637-47bd-b676-3577518b93e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:15.134743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6e4a4-8637-47bd-b676-3577518b93e4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-9ff6e4a4-8637-47bd-b676-3577518b93e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:15.155768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:15.156882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:15.157880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:15.157880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:15.157880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:15.157880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:15.157880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:15.157880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:15.157880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:15.157880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:15.157880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 337/675] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:15.158510 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26ab1c-a525-4e8d-8a89-5fa0edb05a22 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:15.158621 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26ab1c-a525-4e8d-8a89-5fa0edb05a22 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:15.158930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26ab1c-a525-4e8d-8a89-5fa0edb05a22 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:15.160392 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 338/676] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:15.161333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26ab1c-a525-4e8d-8a89-5fa0edb05a22 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:15.165846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:15.198423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:15.244376 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a1088-b260-4451-9358-be9aac344426 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:15.294505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26ab1c-a525-4e8d-8a89-5fa0edb05a22 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:15.303173 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866a41d-265d-47a7-8ad7-92a2dbd8bd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:15.304267 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866a41d-265d-47a7-8ad7-92a2dbd8bd03 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:15.304787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866a41d-265d-47a7-8ad7-92a2dbd8bd03 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:15.305279 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866a41d-265d-47a7-8ad7-92a2dbd8bd03 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:15.306572 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866a41d-265d-47a7-8ad7-92a2dbd8bd03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:15.307108 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866a41d-265d-47a7-8ad7-92a2dbd8bd03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:15.309612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f866a41d-265d-47a7-8ad7-92a2dbd8bd03 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:15.329209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 339/677] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:15.330943 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866a41d-265d-47a7-8ad7-92a2dbd8bd03 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:15.335295 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce6883-eb00-48e9-887c-e87ee6e6552f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:15.336060 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce6883-eb00-48e9-887c-e87ee6e6552f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:15.336370 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce6883-eb00-48e9-887c-e87ee6e6552f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:15.336652 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce6883-eb00-48e9-887c-e87ee6e6552f None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:15.337392 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce6883-eb00-48e9-887c-e87ee6e6552f None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:15.337704 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce6883-eb00-48e9-887c-e87ee6e6552f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:15.339090 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce6883-eb00-48e9-887c-e87ee6e6552f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:15.352381 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 339/678] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:15.353579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90ce54-9204-41d4-bea4-2194fab8e07b None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:15.353867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90ce54-9204-41d4-bea4-2194fab8e07b None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:15.354144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db90ce54-9204-41d4-bea4-2194fab8e07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:15.371262 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-db90ce54-9204-41d4-bea4-2194fab8e07b None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:15.372831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-db90ce54-9204-41d4-bea4-2194fab8e07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:15.437254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:15.460360 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 340/679] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:15.460360 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90ce54-9204-41d4-bea4-2194fab8e07b None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:15.464361 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866a41d-265d-47a7-8ad7-92a2dbd8bd03 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:15.495684 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:15.500300 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba144f5-c979-4007-8fa0-d4b268c748e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:15.501013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba144f5-c979-4007-8fa0-d4b268c748e2 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:15.501294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba144f5-c979-4007-8fa0-d4b268c748e2 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:15.501574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba144f5-c979-4007-8fa0-d4b268c748e2 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:15.503107 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba144f5-c979-4007-8fa0-d4b268c748e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:15.510862 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba144f5-c979-4007-8fa0-d4b268c748e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:15.510862 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba144f5-c979-4007-8fa0-d4b268c748e2 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:15.518375 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 340/680] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:15.519205 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba144f5-c979-4007-8fa0-d4b268c748e2 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:15.576073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db90ce54-9204-41d4-bea4-2194fab8e07b None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:15.614283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a05c2-d5d7-4211-ba96-1616c483231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:15.615813 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a05c2-d5d7-4211-ba96-1616c483231c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:15.616675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a05c2-d5d7-4211-ba96-1616c483231c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:15.617452 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83a05c2-d5d7-4211-ba96-1616c483231c None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:15.625793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba144f5-c979-4007-8fa0-d4b268c748e2 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:15.628319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a05c2-d5d7-4211-ba96-1616c483231c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:15.630828 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83a05c2-d5d7-4211-ba96-1616c483231c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-a83a05c2-d5d7-4211-ba96-1616c483231c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:15.646161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:15.647276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:15.648288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:15.648288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:15.648288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:15.648288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:15.648288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:15.648288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:15.648288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:15.648288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:15.648288 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 341/681] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:15.648872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a05c2-d5d7-4211-ba96-1616c483231c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:15.667932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f4fe4-62cf-4c47-8b1c-1349cffa708b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:15.668835 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f4fe4-62cf-4c47-8b1c-1349cffa708b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:15.669133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f4fe4-62cf-4c47-8b1c-1349cffa708b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:15.669414 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f4fe4-62cf-4c47-8b1c-1349cffa708b None admin] PATH_INFO: `/v3/users/swift` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:15.676974 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f4fe4-62cf-4c47-8b1c-1349cffa708b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:15.678414 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f4fe4-62cf-4c47-8b1c-1349cffa708b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-389f4fe4-62cf-4c47-8b1c-1349cffa708b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:15.693718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:15.694762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:15.695836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:15.695836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:15.695836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:15.695836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:15.695836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:15.695836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:15.695836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:15.695836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:15.695836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:15.695836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:15.695836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 341/682] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:15.696538 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f4fe4-62cf-4c47-8b1c-1349cffa708b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:15.715722 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:15.756911 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:15.770098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83a05c2-d5d7-4211-ba96-1616c483231c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:15.807618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49585f-e571-454a-ae19-2d53d7ecf71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:15.808182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f4fe4-62cf-4c47-8b1c-1349cffa708b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:15.808460 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49585f-e571-454a-ae19-2d53d7ecf71e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:15.808749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49585f-e571-454a-ae19-2d53d7ecf71e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:15.809028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49585f-e571-454a-ae19-2d53d7ecf71e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:15.822441 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f49585f-e571-454a-ae19-2d53d7ecf71e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-7f49585f-e571-454a-ae19-2d53d7ecf71e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:15.830226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:15.831270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 342/683] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:15.832206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49585f-e571-454a-ae19-2d53d7ecf71e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:15.853699 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b673af-5684-408d-9b4a-baebfed473f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:15.854366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b673af-5684-408d-9b4a-baebfed473f9 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:15.854651 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b673af-5684-408d-9b4a-baebfed473f9 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:15.854928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b673af-5684-408d-9b4a-baebfed473f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:15.855782 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b673af-5684-408d-9b4a-baebfed473f9 None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=15cb25c883074386a956a0754e12e13f) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:15.856100 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b673af-5684-408d-9b4a-baebfed473f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:15.857558 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b673af-5684-408d-9b4a-baebfed473f9 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:15.864409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 342/684] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:15 2022] GET /identity/v3/role_assignments?user.id=15cb25c883074386a956a0754e12e13f&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:15.865209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b673af-5684-408d-9b4a-baebfed473f9 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:15.989039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f49585f-e571-454a-ae19-2d53d7ecf71e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:16.008273 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b673af-5684-408d-9b4a-baebfed473f9 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:16.028193 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522400dc-b765-4e0e-bb89-caaffc247a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:16.028918 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522400dc-b765-4e0e-bb89-caaffc247a7c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:16.029224 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522400dc-b765-4e0e-bb89-caaffc247a7c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:16.029490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522400dc-b765-4e0e-bb89-caaffc247a7c None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:16.030553 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522400dc-b765-4e0e-bb89-caaffc247a7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:16.030697 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522400dc-b765-4e0e-bb89-caaffc247a7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:16.032291 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522400dc-b765-4e0e-bb89-caaffc247a7c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:16.034225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:16.035365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:16.052166 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 343/685] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:16.053540 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522400dc-b765-4e0e-bb89-caaffc247a7c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:16.058633 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96a872-eb86-4787-9ba3-6f37d40dcaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:16.059420 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96a872-eb86-4787-9ba3-6f37d40dcaa4 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:16.059795 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96a872-eb86-4787-9ba3-6f37d40dcaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:16.060147 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96a872-eb86-4787-9ba3-6f37d40dcaa4 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:16.061190 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96a872-eb86-4787-9ba3-6f37d40dcaa4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:16.061581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96a872-eb86-4787-9ba3-6f37d40dcaa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:16.063114 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96a872-eb86-4787-9ba3-6f37d40dcaa4 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:16.086006 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 343/686] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:29:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:16.089863 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96a872-eb86-4787-9ba3-6f37d40dcaa4 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:16.186757 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522400dc-b765-4e0e-bb89-caaffc247a7c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:16.225065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8d5ed-f03f-4643-9b0c-2a296863f068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:16.225801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8d5ed-f03f-4643-9b0c-2a296863f068 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:16.226033 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8d5ed-f03f-4643-9b0c-2a296863f068 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:16.226314 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8d5ed-f03f-4643-9b0c-2a296863f068 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:16.227082 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8d5ed-f03f-4643-9b0c-2a296863f068 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:16.227394 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8d5ed-f03f-4643-9b0c-2a296863f068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:16.228508 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96a872-eb86-4787-9ba3-6f37d40dcaa4 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:16.228977 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8d5ed-f03f-4643-9b0c-2a296863f068 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:16.238689 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 344/687] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:29:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:16.239667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8d5ed-f03f-4643-9b0c-2a296863f068 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:16.273000 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a6195-3b5b-48e7-9c81-ba5b4e25efb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:16.273685 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a6195-3b5b-48e7-9c81-ba5b4e25efb3 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:16.273975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a6195-3b5b-48e7-9c81-ba5b4e25efb3 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:16.274304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a6195-3b5b-48e7-9c81-ba5b4e25efb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:16.275171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a6195-3b5b-48e7-9c81-ba5b4e25efb3 None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=b272b57eafcb4de9b85ce8a8a893a2d1) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:16.275526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a6195-3b5b-48e7-9c81-ba5b4e25efb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:16.276959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a6195-3b5b-48e7-9c81-ba5b4e25efb3 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:16.288455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 344/688] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:16 2022] GET /identity/v3/role_assignments?user.id=b272b57eafcb4de9b85ce8a8a893a2d1&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:16.288455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a6195-3b5b-48e7-9c81-ba5b4e25efb3 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:16.317259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:16.339037 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:16.440179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a6195-3b5b-48e7-9c81-ba5b4e25efb3 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:16.489569 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e99b-62bd-4433-8ffc-89bd0f832afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:16.491355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e99b-62bd-4433-8ffc-89bd0f832afb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:16.491847 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e99b-62bd-4433-8ffc-89bd0f832afb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:16.492289 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96e99b-62bd-4433-8ffc-89bd0f832afb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:16.510104 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8d5ed-f03f-4643-9b0c-2a296863f068 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:16.515493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96e99b-62bd-4433-8ffc-89bd0f832afb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:16.523699 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96e99b-62bd-4433-8ffc-89bd0f832afb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-4f96e99b-62bd-4433-8ffc-89bd0f832afb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:16.541058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:16.542566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:16.543813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:16.543813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:16.543813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:16.543813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:16.543813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 345/689] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:16.552608 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e99b-62bd-4433-8ffc-89bd0f832afb None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:16.567447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e775-3a9f-4d08-ba9e-159aa4fb3e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:16.568390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e775-3a9f-4d08-ba9e-159aa4fb3e84 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:16.568882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e775-3a9f-4d08-ba9e-159aa4fb3e84 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:16.569383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8613e775-3a9f-4d08-ba9e-159aa4fb3e84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:16.645901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613e775-3a9f-4d08-ba9e-159aa4fb3e84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:16.647621 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8613e775-3a9f-4d08-ba9e-159aa4fb3e84 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:16.664774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:16.668414 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96e99b-62bd-4433-8ffc-89bd0f832afb None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:16.688303 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:16.714029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c36708-a91d-4971-8fa5-a68bff728717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:16.714719 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c36708-a91d-4971-8fa5-a68bff728717 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:16.715003 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c36708-a91d-4971-8fa5-a68bff728717 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:16.715279 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c36708-a91d-4971-8fa5-a68bff728717 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:16.716409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c36708-a91d-4971-8fa5-a68bff728717 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:16.716930 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c36708-a91d-4971-8fa5-a68bff728717 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:16.718865 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c36708-a91d-4971-8fa5-a68bff728717 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:16.738230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 346/690] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:16.744820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c36708-a91d-4971-8fa5-a68bff728717 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:16.775294 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 345/691] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:29:16.779740 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e775-3a9f-4d08-ba9e-159aa4fb3e84 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:16.870254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c36708-a91d-4971-8fa5-a68bff728717 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:16.910194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4881b06-6a30-4c91-9529-b0146e5e2b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:16.911005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4881b06-6a30-4c91-9529-b0146e5e2b2e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:16.911539 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4881b06-6a30-4c91-9529-b0146e5e2b2e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:16.911815 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4881b06-6a30-4c91-9529-b0146e5e2b2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:16.923463 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4881b06-6a30-4c91-9529-b0146e5e2b2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:16.925621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4881b06-6a30-4c91-9529-b0146e5e2b2e None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-e4881b06-6a30-4c91-9529-b0146e5e2b2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:16.946072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:16.947454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:16.948758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:16.948758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:16.948758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:16.948758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:16.948758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:16.948758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:16.948758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:16.948758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:16.948758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:29:16.948758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:16.948758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 347/692] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:29:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:16.954961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4881b06-6a30-4c91-9529-b0146e5e2b2e None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:16.977645 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:16.989773 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8613e775-3a9f-4d08-ba9e-159aa4fb3e84 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:17.055294 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d33fad-2d46-41fd-bd13-ffd0655035b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:17.056117 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d33fad-2d46-41fd-bd13-ffd0655035b0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:17.056436 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d33fad-2d46-41fd-bd13-ffd0655035b0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:17.056723 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d33fad-2d46-41fd-bd13-ffd0655035b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:17.060111 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:17.075537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d33fad-2d46-41fd-bd13-ffd0655035b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:17.077039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d33fad-2d46-41fd-bd13-ffd0655035b0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-00d33fad-2d46-41fd-bd13-ffd0655035b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:17.110081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:17.111113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:17.112717 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:17.112717 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:17.112717 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:17.112717 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:17.113435 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 346/693] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:17.118082 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d33fad-2d46-41fd-bd13-ffd0655035b0 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:17.150249 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4881b06-6a30-4c91-9529-b0146e5e2b2e None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:17.207386 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851987-9c8e-421f-bb06-a0449d414966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:17.208082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851987-9c8e-421f-bb06-a0449d414966 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:17.208400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851987-9c8e-421f-bb06-a0449d414966 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:17.208684 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25851987-9c8e-421f-bb06-a0449d414966 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:17.209535 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25851987-9c8e-421f-bb06-a0449d414966 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:17.209848 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25851987-9c8e-421f-bb06-a0449d414966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:17.211279 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25851987-9c8e-421f-bb06-a0449d414966 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:17.231962 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 348/694] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:29:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:17.242219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851987-9c8e-421f-bb06-a0449d414966 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:17.293316 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d33fad-2d46-41fd-bd13-ffd0655035b0 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:17.349588 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30de89-5ad9-4c02-ba20-f36567407f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:17.349588 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30de89-5ad9-4c02-ba20-f36567407f41 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:17.349588 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30de89-5ad9-4c02-ba20-f36567407f41 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:17.349588 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30de89-5ad9-4c02-ba20-f36567407f41 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:17.350516 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30de89-5ad9-4c02-ba20-f36567407f41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:17.350516 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30de89-5ad9-4c02-ba20-f36567407f41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:17.350516 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30de89-5ad9-4c02-ba20-f36567407f41 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:17.360120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 347/695] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:17.368873 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30de89-5ad9-4c02-ba20-f36567407f41 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:17.397701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:17.433092 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25851987-9c8e-421f-bb06-a0449d414966 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:17.453167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:17.472402 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12701a-6185-40dd-835c-c90acf6a1762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:17.473023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12701a-6185-40dd-835c-c90acf6a1762 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:17.473309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12701a-6185-40dd-835c-c90acf6a1762 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:17.473589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12701a-6185-40dd-835c-c90acf6a1762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:17.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30de89-5ad9-4c02-ba20-f36567407f41 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:17.564299 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12701a-6185-40dd-835c-c90acf6a1762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:17.566549 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12701a-6185-40dd-835c-c90acf6a1762 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:17.592174 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4008c5-0b16-4f8d-b154-1e06e9d13a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:17.593171 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4008c5-0b16-4f8d-b154-1e06e9d13a04 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:17.593543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4008c5-0b16-4f8d-b154-1e06e9d13a04 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:17.593912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4008c5-0b16-4f8d-b154-1e06e9d13a04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:17.601989 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4008c5-0b16-4f8d-b154-1e06e9d13a04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:17.603458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4008c5-0b16-4f8d-b154-1e06e9d13a04 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-ce4008c5-0b16-4f8d-b154-1e06e9d13a04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:17.618831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:17.619865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:17.620996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:17.620996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:17.620996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:17.620996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:17.620996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:17.620996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:17.620996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:17.620996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:17.620996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 348/696] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:17.622084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d44911-6bd7-4221-bb4d-5c87bf577f81 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:17.628426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d44911-6bd7-4221-bb4d-5c87bf577f81 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:17.628426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d44911-6bd7-4221-bb4d-5c87bf577f81 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:17.629088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 349/697] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:17.629942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d44911-6bd7-4221-bb4d-5c87bf577f81 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:17.657830 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:17.696879 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 349/698] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:29:17.698697 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf102969-a56a-4d56-a98f-bf6bea732700 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:17.699181 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf102969-a56a-4d56-a98f-bf6bea732700 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:17.699787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf102969-a56a-4d56-a98f-bf6bea732700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:17.719464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-cf102969-a56a-4d56-a98f-bf6bea732700 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:17.720956 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-cf102969-a56a-4d56-a98f-bf6bea732700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:17.779324 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d44911-6bd7-4221-bb4d-5c87bf577f81 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:17.789026 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:17.800681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 350/699] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:17.808357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf102969-a56a-4d56-a98f-bf6bea732700 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:17.824413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56293-c019-413b-83b4-7685944a1230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:17.824413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56293-c019-413b-83b4-7685944a1230 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:17.824413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56293-c019-413b-83b4-7685944a1230 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:17.824413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f56293-c019-413b-83b4-7685944a1230 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:17.824855 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56293-c019-413b-83b4-7685944a1230 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:17.824855 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56293-c019-413b-83b4-7685944a1230 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:17.824855 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f56293-c019-413b-83b4-7685944a1230 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:17.840855 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 350/700] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:17.847568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56293-c019-413b-83b4-7685944a1230 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:17.935343 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:17.950543 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf102969-a56a-4d56-a98f-bf6bea732700 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:17.985140 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f56293-c019-413b-83b4-7685944a1230 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:18.003648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f66bf9-8e67-4362-aaa1-73ec1991a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:18.004763 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f66bf9-8e67-4362-aaa1-73ec1991a3ad None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:18.005050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f66bf9-8e67-4362-aaa1-73ec1991a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:18.005326 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f66bf9-8e67-4362-aaa1-73ec1991a3ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:18.019348 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f66bf9-8e67-4362-aaa1-73ec1991a3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:18.021527 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f66bf9-8e67-4362-aaa1-73ec1991a3ad None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:18.026453 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3f183-8786-4412-94a9-66942c5e07c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:18.027381 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f183-8786-4412-94a9-66942c5e07c4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:18.027873 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f183-8786-4412-94a9-66942c5e07c4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:18.028310 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3f183-8786-4412-94a9-66942c5e07c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:18.029507 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3f183-8786-4412-94a9-66942c5e07c4 None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=14034b78ec5e4999848a3fffb2964587) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:18.030048 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3f183-8786-4412-94a9-66942c5e07c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:18.031833 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3f183-8786-4412-94a9-66942c5e07c4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:18.039081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 351/701] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:17 2022] GET /identity/v3/role_assignments?user.id=14034b78ec5e4999848a3fffb2964587&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:18.040171 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3f183-8786-4412-94a9-66942c5e07c4 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-45f66bf9-8e67-4362-aaa1-73ec1991a3ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:18.045980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:18.047041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:18.048053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:18.048053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:18.048053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:18.048053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:18.053543 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 351/702] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:18.060133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f66bf9-8e67-4362-aaa1-73ec1991a3ad None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:18.113569 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:18.226168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3f183-8786-4412-94a9-66942c5e07c4 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:18.237762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f66bf9-8e67-4362-aaa1-73ec1991a3ad None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:18.265184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc02fe-7736-4a9b-b888-65336094bfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:18.266372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc02fe-7736-4a9b-b888-65336094bfdb None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:18.266898 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc02fe-7736-4a9b-b888-65336094bfdb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:18.267403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc02fe-7736-4a9b-b888-65336094bfdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:18.279533 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d2633-be13-413a-93d5-057f95f5db3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:18.280309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2633-be13-413a-93d5-057f95f5db3c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:18.280698 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2633-be13-413a-93d5-057f95f5db3c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:18.280940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc02fe-7736-4a9b-b888-65336094bfdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:18.281066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d2633-be13-413a-93d5-057f95f5db3c None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:18.281954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d2633-be13-413a-93d5-057f95f5db3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:18.282276 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d2633-be13-413a-93d5-057f95f5db3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:18.283179 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc02fe-7736-4a9b-b888-65336094bfdb None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:18.283814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d2633-be13-413a-93d5-057f95f5db3c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:18.299667 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 352/703] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-0dfc02fe-7736-4a9b-b888-65336094bfdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:18.309356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:18.310418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:18.311398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:18.311398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:18.311398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:18.311398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:18.314058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 352/704] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:18.316688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d2633-be13-413a-93d5-057f95f5db3c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:18.320877 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc02fe-7736-4a9b-b888-65336094bfdb None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:18.348182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:18.439092 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:18.467669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc02fe-7736-4a9b-b888-65336094bfdb None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:18.480916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d2633-be13-413a-93d5-057f95f5db3c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:18.510488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852638f-4261-4c3b-adf5-ac54ab564964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:18.511199 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852638f-4261-4c3b-adf5-ac54ab564964 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:18.511439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852638f-4261-4c3b-adf5-ac54ab564964 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:18.511711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852638f-4261-4c3b-adf5-ac54ab564964 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:18.513645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852638f-4261-4c3b-adf5-ac54ab564964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:18.513988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852638f-4261-4c3b-adf5-ac54ab564964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:18.515371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4852638f-4261-4c3b-adf5-ac54ab564964 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:18.524238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cb9ee-ce2b-4bd6-beda-228fa6b44442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:18.525798 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb9ee-ce2b-4bd6-beda-228fa6b44442 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:18.526100 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb9ee-ce2b-4bd6-beda-228fa6b44442 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:18.526380 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cb9ee-ce2b-4bd6-beda-228fa6b44442 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:18.532475 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 353/705] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:18.534278 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cb9ee-ce2b-4bd6-beda-228fa6b44442 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:18.534717 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86422862-9e18-4a97-bbfc-fc5dbaa88d19 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:18.535093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86422862-9e18-4a97-bbfc-fc5dbaa88d19 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:18.535419 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86422862-9e18-4a97-bbfc-fc5dbaa88d19 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:18.535983 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cb9ee-ce2b-4bd6-beda-228fa6b44442 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:18.537205 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 354/706] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:18.543806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fd728-ce22-48b8-b446-972703bd78db None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:18.544089 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fd728-ce22-48b8-b446-972703bd78db None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:18.544419 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fd728-ce22-48b8-b446-972703bd78db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-a07cb9ee-ce2b-4bd6-beda-228fa6b44442 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:18.553630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:18.554691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:18.555681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:18.555681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:18.555681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:18.555681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:18.555681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:18.555681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:18.555681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:18.555681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:18.555681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 353/707] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:18.556231 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cb9ee-ce2b-4bd6-beda-228fa6b44442 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:18.570705 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-788fd728-ce22-48b8-b446-972703bd78db None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:18.572369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-788fd728-ce22-48b8-b446-972703bd78db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:18.649348 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:18.661937 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 355/708] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:18.666403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fd728-ce22-48b8-b446-972703bd78db None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:18.700363 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cb9ee-ce2b-4bd6-beda-228fa6b44442 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:18.737175 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e9fed-829c-4436-a429-d542a86908b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:18.737948 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e9fed-829c-4436-a429-d542a86908b1 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:18.738198 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e9fed-829c-4436-a429-d542a86908b1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:18.738544 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e9fed-829c-4436-a429-d542a86908b1 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:18.739531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e9fed-829c-4436-a429-d542a86908b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:18.740868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195e9fed-829c-4436-a429-d542a86908b1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:18.763163 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:18.795975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fd728-ce22-48b8-b446-972703bd78db None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:18.832700 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14804bf-3832-4271-bc1b-7c3dfee6a0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:18.833379 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14804bf-3832-4271-bc1b-7c3dfee6a0f9 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:18.833667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14804bf-3832-4271-bc1b-7c3dfee6a0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:18.833947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14804bf-3832-4271-bc1b-7c3dfee6a0f9 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:18.834748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14804bf-3832-4271-bc1b-7c3dfee6a0f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:18.835068 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14804bf-3832-4271-bc1b-7c3dfee6a0f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:18.836602 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14804bf-3832-4271-bc1b-7c3dfee6a0f9 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:18.852373 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 356/709] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:18.852373 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14804bf-3832-4271-bc1b-7c3dfee6a0f9 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:18.871099 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 354/710] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:29:18 2022] POST /identity/v3/users => generated 254 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:18.872460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e9fed-829c-4436-a429-d542a86908b1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:19.058066 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14804bf-3832-4271-bc1b-7c3dfee6a0f9 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:19.100411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:19.106536 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d103793-9987-4ee3-8393-bde584477e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:19.114415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d103793-9987-4ee3-8393-bde584477e8a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:19.114415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d103793-9987-4ee3-8393-bde584477e8a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:19.114415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d103793-9987-4ee3-8393-bde584477e8a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:19.114415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d103793-9987-4ee3-8393-bde584477e8a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:19.114415 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d103793-9987-4ee3-8393-bde584477e8a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-6d103793-9987-4ee3-8393-bde584477e8a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:19.125675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:19.127363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:29:19.128319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:19.129225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e9fed-829c-4436-a429-d542a86908b1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:19.132436 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 357/711] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 20:29:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:19.143431 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d103793-9987-4ee3-8393-bde584477e8a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:19.181896 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:19.191562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfa2b8d-42b8-45c5-a924-d5e77dd8cce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:19.196711 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa2b8d-42b8-45c5-a924-d5e77dd8cce8 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:19.196711 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa2b8d-42b8-45c5-a924-d5e77dd8cce8 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:19.196711 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfa2b8d-42b8-45c5-a924-d5e77dd8cce8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:19.196711 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa2b8d-42b8-45c5-a924-d5e77dd8cce8 None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=e6a982ae15a6443b9b52529df47aa837) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:19.196711 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa2b8d-42b8-45c5-a924-d5e77dd8cce8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:19.197105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfa2b8d-42b8-45c5-a924-d5e77dd8cce8 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:19.201832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 355/712] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:18 2022] GET /identity/v3/role_assignments?user.id=e6a982ae15a6443b9b52529df47aa837&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:19.202926 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f89c7-a97e-43e7-b9d9-59089c321797 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:19.203217 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f89c7-a97e-43e7-b9d9-59089c321797 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:19.203495 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f89c7-a97e-43e7-b9d9-59089c321797 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:19.204985 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 356/713] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:19.211637 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b407b-226f-4b44-bfe5-aef07c81fc93 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:19.211637 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b407b-226f-4b44-bfe5-aef07c81fc93 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:19.211946 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b407b-226f-4b44-bfe5-aef07c81fc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:19.233745 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-086b407b-226f-4b44-bfe5-aef07c81fc93 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:19.234965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-086b407b-226f-4b44-bfe5-aef07c81fc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:19.292812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d103793-9987-4ee3-8393-bde584477e8a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:19.342472 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 357/714] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:29:19.350831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6dbe9-5612-40ba-ab23-5d8050a38e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:19.351903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6dbe9-5612-40ba-ab23-5d8050a38e9d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:19.352505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6dbe9-5612-40ba-ab23-5d8050a38e9d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:19.352984 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6dbe9-5612-40ba-ab23-5d8050a38e9d None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:19.354051 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6dbe9-5612-40ba-ab23-5d8050a38e9d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:19.354569 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6dbe9-5612-40ba-ab23-5d8050a38e9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:19.356230 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6dbe9-5612-40ba-ab23-5d8050a38e9d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:19.362562 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 358/715] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:29:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:19.368470 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b407b-226f-4b44-bfe5-aef07c81fc93 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:19.369819 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6dbe9-5612-40ba-ab23-5d8050a38e9d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:19.456634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:19.498051 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b407b-226f-4b44-bfe5-aef07c81fc93 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:19.516644 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6dbe9-5612-40ba-ab23-5d8050a38e9d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:19.521329 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:19.536768 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0bdfa-ba92-42f4-abcf-1eca50d33fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:19.543866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0bdfa-ba92-42f4-abcf-1eca50d33fef None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:19.543866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0bdfa-ba92-42f4-abcf-1eca50d33fef None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:19.543866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0bdfa-ba92-42f4-abcf-1eca50d33fef None admin] PATH_INFO: `/v3/services/network` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:19.543866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0bdfa-ba92-42f4-abcf-1eca50d33fef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:19.543866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0bdfa-ba92-42f4-abcf-1eca50d33fef None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-fbd0bdfa-ba92-42f4-abcf-1eca50d33fef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:19.556148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:19.557434 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:19.558726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 358/716] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:19 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:19.561249 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480ffcc-1ce3-4750-8144-b4045fb5980d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:19.562191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ffcc-1ce3-4750-8144-b4045fb5980d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:19.562856 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ffcc-1ce3-4750-8144-b4045fb5980d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:19.563454 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480ffcc-1ce3-4750-8144-b4045fb5980d None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:19.563664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0bdfa-ba92-42f4-abcf-1eca50d33fef None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:19.565563 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480ffcc-1ce3-4750-8144-b4045fb5980d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:19.565879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480ffcc-1ce3-4750-8144-b4045fb5980d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:19.568767 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480ffcc-1ce3-4750-8144-b4045fb5980d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:19.574009 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 359/717] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:29:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:19.744671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0bdfa-ba92-42f4-abcf-1eca50d33fef None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:19.803041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700cf7-a2b7-4b8c-a405-67b4adf1e24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:19.803801 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700cf7-a2b7-4b8c-a405-67b4adf1e24b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:19.804090 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700cf7-a2b7-4b8c-a405-67b4adf1e24b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:19.804389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700cf7-a2b7-4b8c-a405-67b4adf1e24b None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:19.805124 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700cf7-a2b7-4b8c-a405-67b4adf1e24b None admin] RBAC: Adding query filter params (name=network) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:19.805439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700cf7-a2b7-4b8c-a405-67b4adf1e24b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:19.806824 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700cf7-a2b7-4b8c-a405-67b4adf1e24b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:19.812207 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 359/718] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:19.818190 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480ffcc-1ce3-4750-8144-b4045fb5980d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:19.946670 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480ffcc-1ce3-4750-8144-b4045fb5980d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:19.982371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:19.998041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf575dc6-6c9b-4cda-8507-14239f77603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:19.998664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf575dc6-6c9b-4cda-8507-14239f77603f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:19.998952 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf575dc6-6c9b-4cda-8507-14239f77603f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:19.999232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf575dc6-6c9b-4cda-8507-14239f77603f None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:20.000015 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf575dc6-6c9b-4cda-8507-14239f77603f None admin] RBAC: Adding query filter params (type=network) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:20.000351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf575dc6-6c9b-4cda-8507-14239f77603f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:20.001904 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf575dc6-6c9b-4cda-8507-14239f77603f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:20.012353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 360/719] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:20.850215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61837d5-0eb7-4085-9b48-fb522f3a7219 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:20.850765 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61837d5-0eb7-4085-9b48-fb522f3a7219 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:20.850765 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61837d5-0eb7-4085-9b48-fb522f3a7219 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:20.852411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 360/720] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:20.866880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc8246-ab9b-409b-ab2b-74273a151546 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:20.867380 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc8246-ab9b-409b-ab2b-74273a151546 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:20.867508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc8246-ab9b-409b-ab2b-74273a151546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:20.894279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-94fc8246-ab9b-409b-ab2b-74273a151546 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:20.895478 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-94fc8246-ab9b-409b-ab2b-74273a151546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:20.985859 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 361/721] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:21.017953 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61837d5-0eb7-4085-9b48-fb522f3a7219 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:21.089830 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:21.148093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61837d5-0eb7-4085-9b48-fb522f3a7219 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:21.184942 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d3f14-0d1f-45e3-9d40-6a27f38990fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:21.186143 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3f14-0d1f-45e3-9d40-6a27f38990fa None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:21.186643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3f14-0d1f-45e3-9d40-6a27f38990fa None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:21.187155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d3f14-0d1f-45e3-9d40-6a27f38990fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:21.199121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d3f14-0d1f-45e3-9d40-6a27f38990fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-507d3f14-0d1f-45e3-9d40-6a27f38990fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:21.207252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:21.208316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:21.209281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:21.212278 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 361/722] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:21.218614 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc8246-ab9b-409b-ab2b-74273a151546 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:21.345080 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc8246-ab9b-409b-ab2b-74273a151546 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:21.373303 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:21.397326 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee891a-13fe-4746-8172-d5de707106d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:21.398054 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee891a-13fe-4746-8172-d5de707106d5 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:21.398394 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee891a-13fe-4746-8172-d5de707106d5 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:21.398622 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee891a-13fe-4746-8172-d5de707106d5 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:21.399387 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee891a-13fe-4746-8172-d5de707106d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:21.399699 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee891a-13fe-4746-8172-d5de707106d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:21.401168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee891a-13fe-4746-8172-d5de707106d5 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:21.416046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 362/723] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:29:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:21.421940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d3f14-0d1f-45e3-9d40-6a27f38990fa None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:21.556744 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d3f14-0d1f-45e3-9d40-6a27f38990fa None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:21.566436 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:21.594117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbcafb-bf5d-41d3-ad10-55eae3e3fc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:21.594975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbcafb-bf5d-41d3-ad10-55eae3e3fc32 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:21.595267 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbcafb-bf5d-41d3-ad10-55eae3e3fc32 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:21.596079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddbcafb-bf5d-41d3-ad10-55eae3e3fc32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:21.607439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbcafb-bf5d-41d3-ad10-55eae3e3fc32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:21.608931 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddbcafb-bf5d-41d3-ad10-55eae3e3fc32 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-eddbcafb-bf5d-41d3-ad10-55eae3e3fc32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:21.628640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:21.629673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:21.631079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:21.631079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:21.631079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:21.631079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:21.631079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 362/724] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:21.635926 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee891a-13fe-4746-8172-d5de707106d5 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:21.773802 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee891a-13fe-4746-8172-d5de707106d5 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:21.827000 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c7c46-9dbf-4454-a66a-38671ec1b002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:21.828091 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c7c46-9dbf-4454-a66a-38671ec1b002 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:21.828616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c7c46-9dbf-4454-a66a-38671ec1b002 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:21.829093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1c7c46-9dbf-4454-a66a-38671ec1b002 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:21.830592 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c7c46-9dbf-4454-a66a-38671ec1b002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:21.831105 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c7c46-9dbf-4454-a66a-38671ec1b002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:21.832731 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1c7c46-9dbf-4454-a66a-38671ec1b002 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:21.853057 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 363/725] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:21.858694 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbcafb-bf5d-41d3-ad10-55eae3e3fc32 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:22.019591 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddbcafb-bf5d-41d3-ad10-55eae3e3fc32 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:22.054853 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afe6d0-9263-454a-8bfb-2a87450cd370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:22.055664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe6d0-9263-454a-8bfb-2a87450cd370 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.055947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe6d0-9263-454a-8bfb-2a87450cd370 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.056224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe6d0-9263-454a-8bfb-2a87450cd370 None admin] PATH_INFO: `/v3/users/swift` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.064197 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afe6d0-9263-454a-8bfb-2a87450cd370 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:22.065824 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afe6d0-9263-454a-8bfb-2a87450cd370 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-93afe6d0-9263-454a-8bfb-2a87450cd370 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:22.088365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:22.089502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:22.091119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:22.091119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:22.091119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:22.091119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:22.091119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:22.091119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:22.091119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:22.091119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:22.091119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:22.091119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:22.091119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 363/726] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:22.095133 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c7c46-9dbf-4454-a66a-38671ec1b002 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:22.180863 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd46fc3-df52-4e21-ac8e-1deed961447c None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.181205 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd46fc3-df52-4e21-ac8e-1deed961447c None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.181429 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd46fc3-df52-4e21-ac8e-1deed961447c None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.181750 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:22.183071 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 364/727] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:22.188131 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5ca4e-a38e-41cf-af79-6ae159c02449 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.188461 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5ca4e-a38e-41cf-af79-6ae159c02449 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.188745 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5ca4e-a38e-41cf-af79-6ae159c02449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.214170 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-87a5ca4e-a38e-41cf-af79-6ae159c02449 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:22.215800 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-87a5ca4e-a38e-41cf-af79-6ae159c02449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:22.233262 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:22.252867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1c7c46-9dbf-4454-a66a-38671ec1b002 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:22.293074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0145c0f-e307-4093-81b6-b6aa686199cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:22.293673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145c0f-e307-4093-81b6-b6aa686199cc None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.293962 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145c0f-e307-4093-81b6-b6aa686199cc None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.294245 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0145c0f-e307-4093-81b6-b6aa686199cc None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.295121 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0145c0f-e307-4093-81b6-b6aa686199cc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:22.295441 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0145c0f-e307-4093-81b6-b6aa686199cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:22.297118 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0145c0f-e307-4093-81b6-b6aa686199cc None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:22.299850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 365/728] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:29:22.301717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2aff-5ebd-41e5-8ae7-291a4ea6196a None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.302214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2aff-5ebd-41e5-8ae7-291a4ea6196a None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.302690 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb2aff-5ebd-41e5-8ae7-291a4ea6196a None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.304501 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 366/729] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:22.313852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacbb37-2fde-4fb4-8869-1906d7e38113 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.315070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacbb37-2fde-4fb4-8869-1906d7e38113 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.315698 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacbb37-2fde-4fb4-8869-1906d7e38113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.327177 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 364/730] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:29:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:22.333073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0145c0f-e307-4093-81b6-b6aa686199cc None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:22.339916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-9eacbb37-2fde-4fb4-8869-1906d7e38113 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:22.341183 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-9eacbb37-2fde-4fb4-8869-1906d7e38113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:22.428431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 367/731] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:22.429394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacbb37-2fde-4fb4-8869-1906d7e38113 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:22.472727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:22.479319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0145c0f-e307-4093-81b6-b6aa686199cc None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:22.527101 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c2a7b-7db9-4965-83ea-a9554e3d609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:22.528527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c2a7b-7db9-4965-83ea-a9554e3d609b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.528997 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c2a7b-7db9-4965-83ea-a9554e3d609b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.529429 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c2a7b-7db9-4965-83ea-a9554e3d609b None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.541135 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:22.541966 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6c2a7b-7db9-4965-83ea-a9554e3d609b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-fd6c2a7b-7db9-4965-83ea-a9554e3d609b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:22.551903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:22.552988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 365/732] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:22.555005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c2a7b-7db9-4965-83ea-a9554e3d609b None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:22.566648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacbb37-2fde-4fb4-8869-1906d7e38113 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:22.600986 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b2edf-933e-46de-8129-028299b65fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:22.601648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b2edf-933e-46de-8129-028299b65fdc None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.601928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b2edf-933e-46de-8129-028299b65fdc None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.602201 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b2edf-933e-46de-8129-028299b65fdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.670961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c2a7b-7db9-4965-83ea-a9554e3d609b None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:22.677064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b2edf-933e-46de-8129-028299b65fdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:22.678512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b2edf-933e-46de-8129-028299b65fdc None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:22.707160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5398f-682d-4ee2-8d9d-5c614c5b0ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:22.708077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5398f-682d-4ee2-8d9d-5c614c5b0ecf None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.708503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5398f-682d-4ee2-8d9d-5c614c5b0ecf None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.708782 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5398f-682d-4ee2-8d9d-5c614c5b0ecf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.719609 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5398f-682d-4ee2-8d9d-5c614c5b0ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-87f5398f-682d-4ee2-8d9d-5c614c5b0ecf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:22.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:22.729333 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:22.730875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 366/733] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:22.731641 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fca2c0-15de-47f3-a73b-50c85add1cd2 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.731641 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fca2c0-15de-47f3-a73b-50c85add1cd2 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.731641 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fca2c0-15de-47f3-a73b-50c85add1cd2 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.736524 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 367/734] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:22.736524 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fca2c0-15de-47f3-a73b-50c85add1cd2 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:22.759409 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:22.773088 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:22.802503 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 368/735] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:29:22.802503 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b2edf-933e-46de-8129-028299b65fdc None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:22.864137 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fca2c0-15de-47f3-a73b-50c85add1cd2 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:22.902562 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc5ec1-c0e1-430e-8fb1-f67bead8acfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:22.903239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5ec1-c0e1-430e-8fb1-f67bead8acfb None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.903525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5ec1-c0e1-430e-8fb1-f67bead8acfb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.903802 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc5ec1-c0e1-430e-8fb1-f67bead8acfb None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.904583 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc5ec1-c0e1-430e-8fb1-f67bead8acfb None admin] RBAC: Adding query filter params (name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:22.905124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc5ec1-c0e1-430e-8fb1-f67bead8acfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:22.906272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc5ec1-c0e1-430e-8fb1-f67bead8acfb None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:22.913947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b2edf-933e-46de-8129-028299b65fdc None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:22.919575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 368/736] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:22.919575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317c519-47bf-457d-9daa-237721bc78d4 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.919575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317c519-47bf-457d-9daa-237721bc78d4 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.919575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317c519-47bf-457d-9daa-237721bc78d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.935156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-1317c519-47bf-457d-9daa-237721bc78d4 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:22.936402 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-1317c519-47bf-457d-9daa-237721bc78d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:22.955804 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8248fc0-69a4-453f-a7d9-c881b3166020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:22.956524 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8248fc0-69a4-453f-a7d9-c881b3166020 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:22.956810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8248fc0-69a4-453f-a7d9-c881b3166020 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:22.957086 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8248fc0-69a4-453f-a7d9-c881b3166020 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:22.957966 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8248fc0-69a4-453f-a7d9-c881b3166020 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:22.958273 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8248fc0-69a4-453f-a7d9-c881b3166020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:22.959629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8248fc0-69a4-453f-a7d9-c881b3166020 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:22.968099 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 369/737] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:29:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:22.968718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8248fc0-69a4-453f-a7d9-c881b3166020 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:23.007349 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 369/738] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:23.008202 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317c519-47bf-457d-9daa-237721bc78d4 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:23.017288 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:23.022422 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:23.076959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8248fc0-69a4-453f-a7d9-c881b3166020 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:23.109846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317c519-47bf-457d-9daa-237721bc78d4 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:23.111612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc8c93-2e29-4755-a851-659928218baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:23.112499 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc8c93-2e29-4755-a851-659928218baa None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:23.112787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc8c93-2e29-4755-a851-659928218baa None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:23.113062 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc8c93-2e29-4755-a851-659928218baa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:23.124187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc8c93-2e29-4755-a851-659928218baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:23.125636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc8c93-2e29-4755-a851-659928218baa None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:23.144116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97800584-f016-44ea-a829-a001a4830ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-dccc8c93-2e29-4755-a851-659928218baa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:23.144304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:23.145319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:23.146305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:23.146305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:23.146305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:23.146305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:23.146556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97800584-f016-44ea-a829-a001a4830ca8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:23.146556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97800584-f016-44ea-a829-a001a4830ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:23.146704 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 370/739] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:23.146865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97800584-f016-44ea-a829-a001a4830ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:23.146943 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928e8b7-75ad-4df7-9d7a-392c52bd32b9 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:23.147016 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928e8b7-75ad-4df7-9d7a-392c52bd32b9 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:23.147628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928e8b7-75ad-4df7-9d7a-392c52bd32b9 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:23.148683 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 371/740] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:23.149415 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928e8b7-75ad-4df7-9d7a-392c52bd32b9 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:23.161115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97800584-f016-44ea-a829-a001a4830ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:23.163058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97800584-f016-44ea-a829-a001a4830ca8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-97800584-f016-44ea-a829-a001a4830ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:23.181543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:23.182573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:23.183564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:23.183564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:23.183564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:23.183564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:23.183564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 370/741] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:23.183564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97800584-f016-44ea-a829-a001a4830ca8 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:23.250039 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:23.250742 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:23.258025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928e8b7-75ad-4df7-9d7a-392c52bd32b9 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:23.289379 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97800584-f016-44ea-a829-a001a4830ca8 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:23.292188 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89938c1-be18-47d4-a2fe-634317949670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:23.293031 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89938c1-be18-47d4-a2fe-634317949670 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:23.293315 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89938c1-be18-47d4-a2fe-634317949670 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:23.293587 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89938c1-be18-47d4-a2fe-634317949670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:23.304394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89938c1-be18-47d4-a2fe-634317949670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:23.305828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89938c1-be18-47d4-a2fe-634317949670 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-c89938c1-be18-47d4-a2fe-634317949670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:23.324738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:23.325774 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:23.326748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:23.326748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:23.326748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:23.326748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:23.326748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 372/742] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:23.326748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89938c1-be18-47d4-a2fe-634317949670 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:23.329127 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fb8d6-b2de-43bd-92a8-482be8845486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:23.329859 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fb8d6-b2de-43bd-92a8-482be8845486 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:23.330146 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fb8d6-b2de-43bd-92a8-482be8845486 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:23.330423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fb8d6-b2de-43bd-92a8-482be8845486 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:23.331155 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fb8d6-b2de-43bd-92a8-482be8845486 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:23.332586 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fb8d6-b2de-43bd-92a8-482be8845486 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:23.416700 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 371/743] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:29:23 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:23.418068 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b567e-529f-4c73-9ec0-cae37021b4b5 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:23.418368 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b567e-529f-4c73-9ec0-cae37021b4b5 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:23.418669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b567e-529f-4c73-9ec0-cae37021b4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:23.431286 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89938c1-be18-47d4-a2fe-634317949670 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:23.446997 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-7e2b567e-529f-4c73-9ec0-cae37021b4b5 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:23.449602 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-7e2b567e-529f-4c73-9ec0-cae37021b4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:23.491897 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f637675-f373-4861-acf6-12cbd19162f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:23.492970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f637675-f373-4861-acf6-12cbd19162f0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:23.493387 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f637675-f373-4861-acf6-12cbd19162f0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:23.493585 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f637675-f373-4861-acf6-12cbd19162f0 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:23.494366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f637675-f373-4861-acf6-12cbd19162f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:23.494802 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f637675-f373-4861-acf6-12cbd19162f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:23.496276 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f637675-f373-4861-acf6-12cbd19162f0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:23.514717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 373/744] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:23.514717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f637675-f373-4861-acf6-12cbd19162f0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:23.515559 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:23.543418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 372/745] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:23 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:29:23.546125 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b567e-529f-4c73-9ec0-cae37021b4b5 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:23.607872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:23.644638 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f637675-f373-4861-acf6-12cbd19162f0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:23.664279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b567e-529f-4c73-9ec0-cae37021b4b5 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:23.683134 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fdfd2-a86f-4ea1-9983-a76b5b1638d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:23.683732 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fdfd2-a86f-4ea1-9983-a76b5b1638d6 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:23.684017 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fdfd2-a86f-4ea1-9983-a76b5b1638d6 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:23.684290 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fdfd2-a86f-4ea1-9983-a76b5b1638d6 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:23.685294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fdfd2-a86f-4ea1-9983-a76b5b1638d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:23.685528 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fdfd2-a86f-4ea1-9983-a76b5b1638d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:23.686787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fdfd2-a86f-4ea1-9983-a76b5b1638d6 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:23.706948 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 374/746] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:23.706948 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fdfd2-a86f-4ea1-9983-a76b5b1638d6 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:23.712223 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d876b5-6327-4581-9b4d-6bd74ed137d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:23.713078 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d876b5-6327-4581-9b4d-6bd74ed137d3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:23.713371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d876b5-6327-4581-9b4d-6bd74ed137d3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:23.713653 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d876b5-6327-4581-9b4d-6bd74ed137d3 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:23.714472 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d876b5-6327-4581-9b4d-6bd74ed137d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:23.714817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d876b5-6327-4581-9b4d-6bd74ed137d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:23.716252 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d876b5-6327-4581-9b4d-6bd74ed137d3 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:23.728585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 373/747] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:23.733541 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d876b5-6327-4581-9b4d-6bd74ed137d3 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:23.775918 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:23.818046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fdfd2-a86f-4ea1-9983-a76b5b1638d6 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:23.847760 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d876b5-6327-4581-9b4d-6bd74ed137d3 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:23.853397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d8a84-764f-4203-a477-476bbaa081cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:23.854209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8a84-764f-4203-a477-476bbaa081cc None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:23.854499 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8a84-764f-4203-a477-476bbaa081cc None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:23.854782 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d8a84-764f-4203-a477-476bbaa081cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:23.862129 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d8a84-764f-4203-a477-476bbaa081cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:23.863571 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d8a84-764f-4203-a477-476bbaa081cc None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:23.864864 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-b35d8a84-764f-4203-a477-476bbaa081cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:23.878692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:23.879818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:23.880937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:23.880937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:23.880937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:23.880937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:23.880937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:23.880937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:23.880937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:23.880937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:23.880937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 375/748] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:23.881527 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d8a84-764f-4203-a477-476bbaa081cc None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:23.894500 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b0adb-ef5a-4778-9a90-63f7de09dbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:23.895198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b0adb-ef5a-4778-9a90-63f7de09dbef None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:23.895487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b0adb-ef5a-4778-9a90-63f7de09dbef None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:23.895766 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76b0adb-ef5a-4778-9a90-63f7de09dbef None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:23.896901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b0adb-ef5a-4778-9a90-63f7de09dbef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:23.898635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76b0adb-ef5a-4778-9a90-63f7de09dbef None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:23.992511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 374/749] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:29:23 2022] POST /identity/v3/services => generated 240 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:23.992511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b0adb-ef5a-4778-9a90-63f7de09dbef None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:24.002324 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d8a84-764f-4203-a477-476bbaa081cc None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:24.031770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:24.052111 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8909b1a-705a-4b0a-91fc-69f21621cb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:24.052979 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8909b1a-705a-4b0a-91fc-69f21621cb9c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:24.053260 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8909b1a-705a-4b0a-91fc-69f21621cb9c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:24.053531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8909b1a-705a-4b0a-91fc-69f21621cb9c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:24.063576 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8909b1a-705a-4b0a-91fc-69f21621cb9c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:24.066306 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8909b1a-705a-4b0a-91fc-69f21621cb9c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-c8909b1a-705a-4b0a-91fc-69f21621cb9c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:24.084873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:24.086025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:24.087176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:24.087176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:24.087176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:24.087176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:24.087176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:24.087176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:24.087176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:24.087176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:24.087176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:24.087176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:24.087176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 376/750] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:29:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:24.087862 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8909b1a-705a-4b0a-91fc-69f21621cb9c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:24.136390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76b0adb-ef5a-4778-9a90-63f7de09dbef None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:24.191317 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2a046-df28-4a06-8f7e-cc1a2f0388da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:24.192205 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2a046-df28-4a06-8f7e-cc1a2f0388da None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:24.192527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2a046-df28-4a06-8f7e-cc1a2f0388da None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:24.194037 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2a046-df28-4a06-8f7e-cc1a2f0388da None admin] PATH_INFO: `/v3/users/nova` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:24.204600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2a046-df28-4a06-8f7e-cc1a2f0388da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:24.206740 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2a046-df28-4a06-8f7e-cc1a2f0388da None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:24.211347 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8909b1a-705a-4b0a-91fc-69f21621cb9c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-21b2a046-df28-4a06-8f7e-cc1a2f0388da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:24.225112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:24.227469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 375/751] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:29:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:24.229963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2a046-df28-4a06-8f7e-cc1a2f0388da None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:24.254635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a73b06-9fcd-47f9-8eed-865ed5d8b05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:24.255511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a73b06-9fcd-47f9-8eed-865ed5d8b05c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:24.255864 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a73b06-9fcd-47f9-8eed-865ed5d8b05c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:24.256224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a73b06-9fcd-47f9-8eed-865ed5d8b05c None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:24.257195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a73b06-9fcd-47f9-8eed-865ed5d8b05c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:24.257622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a73b06-9fcd-47f9-8eed-865ed5d8b05c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:24.259266 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a73b06-9fcd-47f9-8eed-865ed5d8b05c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:24.271152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:24.278852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 377/752] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:24.279840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a73b06-9fcd-47f9-8eed-865ed5d8b05c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:24.336599 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:24.350451 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2a046-df28-4a06-8f7e-cc1a2f0388da None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:24.388365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ed678-c117-4cd6-bff1-f5ba8a2a05c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:24.389219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ed678-c117-4cd6-bff1-f5ba8a2a05c3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:24.389617 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ed678-c117-4cd6-bff1-f5ba8a2a05c3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:24.389992 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ed678-c117-4cd6-bff1-f5ba8a2a05c3 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:24.390910 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ed678-c117-4cd6-bff1-f5ba8a2a05c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:24.391280 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ed678-c117-4cd6-bff1-f5ba8a2a05c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:24.392861 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ed678-c117-4cd6-bff1-f5ba8a2a05c3 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:24.413159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 376/753] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:29:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:24.414069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ed678-c117-4cd6-bff1-f5ba8a2a05c3 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:24.425750 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a73b06-9fcd-47f9-8eed-865ed5d8b05c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:24.478179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f6540-77a6-4d0f-9c69-fda8f5dcf744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:24.479639 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f6540-77a6-4d0f-9c69-fda8f5dcf744 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:24.480011 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f6540-77a6-4d0f-9c69-fda8f5dcf744 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:24.480426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f6540-77a6-4d0f-9c69-fda8f5dcf744 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:24.481669 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f6540-77a6-4d0f-9c69-fda8f5dcf744 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:24.482111 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f6540-77a6-4d0f-9c69-fda8f5dcf744 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:24.484933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f6540-77a6-4d0f-9c69-fda8f5dcf744 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:24.522369 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 378/754] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:29:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:24.523691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f6540-77a6-4d0f-9c69-fda8f5dcf744 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:24.525638 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:24.537618 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ed678-c117-4cd6-bff1-f5ba8a2a05c3 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:24.577796 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f993b-b7f5-46bf-a19c-1cdc25a0751a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:24.578754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f993b-b7f5-46bf-a19c-1cdc25a0751a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:24.579211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f993b-b7f5-46bf-a19c-1cdc25a0751a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:24.579669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f993b-b7f5-46bf-a19c-1cdc25a0751a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:24.580736 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f993b-b7f5-46bf-a19c-1cdc25a0751a None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=14034b78ec5e4999848a3fffb2964587) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:24.581233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f993b-b7f5-46bf-a19c-1cdc25a0751a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:24.582907 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f993b-b7f5-46bf-a19c-1cdc25a0751a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:24.599048 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 377/755] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:24 2022] GET /identity/v3/role_assignments?user.id=14034b78ec5e4999848a3fffb2964587&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:24.600770 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f993b-b7f5-46bf-a19c-1cdc25a0751a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:24.650720 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:24.659196 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f6540-77a6-4d0f-9c69-fda8f5dcf744 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:24.701385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd646ba9-9def-4dcd-a844-10ff81563213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:24.703524 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd646ba9-9def-4dcd-a844-10ff81563213 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:24.704058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd646ba9-9def-4dcd-a844-10ff81563213 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:24.704558 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd646ba9-9def-4dcd-a844-10ff81563213 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:24.789997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd646ba9-9def-4dcd-a844-10ff81563213 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:24.792071 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd646ba9-9def-4dcd-a844-10ff81563213 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:24.819586 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f993b-b7f5-46bf-a19c-1cdc25a0751a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:24.872363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497b478-77c2-4681-8d73-9856ebc8da6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:24.873188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497b478-77c2-4681-8d73-9856ebc8da6d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:24.873475 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497b478-77c2-4681-8d73-9856ebc8da6d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:24.873753 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497b478-77c2-4681-8d73-9856ebc8da6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:24.893772 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497b478-77c2-4681-8d73-9856ebc8da6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:24.895261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497b478-77c2-4681-8d73-9856ebc8da6d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c497b478-77c2-4681-8d73-9856ebc8da6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:24.921901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:24.922944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:24.923985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:24.923985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:24.923985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:24.923985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:24.924258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:24.925980 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 378/756] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:24.931881 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497b478-77c2-4681-8d73-9856ebc8da6d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:24.939451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 379/757] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:29:24.945344 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:24.946104 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd646ba9-9def-4dcd-a844-10ff81563213 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:25.110824 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd646ba9-9def-4dcd-a844-10ff81563213 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:25.127061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497b478-77c2-4681-8d73-9856ebc8da6d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:25.148013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed12c96-96ca-42c1-9eaf-24baa23cb15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:25.148870 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed12c96-96ca-42c1-9eaf-24baa23cb15c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:25.149154 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed12c96-96ca-42c1-9eaf-24baa23cb15c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:25.149431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed12c96-96ca-42c1-9eaf-24baa23cb15c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:25.164365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed12c96-96ca-42c1-9eaf-24baa23cb15c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:25.164365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed12c96-96ca-42c1-9eaf-24baa23cb15c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-9ed12c96-96ca-42c1-9eaf-24baa23cb15c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:25.181179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:25.182244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:25.183226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:25.183226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:25.183226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:25.183226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:25.183226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 380/758] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:25.188070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed12c96-96ca-42c1-9eaf-24baa23cb15c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:25.217071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415514e-e686-40e1-96c8-01dd731bbcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:25.217781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415514e-e686-40e1-96c8-01dd731bbcae None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:25.218064 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415514e-e686-40e1-96c8-01dd731bbcae None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:25.218347 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415514e-e686-40e1-96c8-01dd731bbcae None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:25.219349 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415514e-e686-40e1-96c8-01dd731bbcae None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:25.219482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415514e-e686-40e1-96c8-01dd731bbcae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:25.220854 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415514e-e686-40e1-96c8-01dd731bbcae None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:25.245812 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:25.252408 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 379/759] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:25.261395 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415514e-e686-40e1-96c8-01dd731bbcae None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:25.296695 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed12c96-96ca-42c1-9eaf-24baa23cb15c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:25.335377 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec25275-619e-4e1c-9573-53cb252b31a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:25.336064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec25275-619e-4e1c-9573-53cb252b31a0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:25.336384 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec25275-619e-4e1c-9573-53cb252b31a0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:25.336706 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec25275-619e-4e1c-9573-53cb252b31a0 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:25.337455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec25275-619e-4e1c-9573-53cb252b31a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:25.337770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec25275-619e-4e1c-9573-53cb252b31a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:25.339153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec25275-619e-4e1c-9573-53cb252b31a0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:25.351712 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 381/760] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:25.357278 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec25275-619e-4e1c-9573-53cb252b31a0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:25.479621 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec25275-619e-4e1c-9573-53cb252b31a0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:25.490251 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:25.521845 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88eb836-8a97-456e-83ef-f093a89cb442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:25.523225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88eb836-8a97-456e-83ef-f093a89cb442 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:25.525162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88eb836-8a97-456e-83ef-f093a89cb442 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:25.525664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88eb836-8a97-456e-83ef-f093a89cb442 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:25.527093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:25.545632 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88eb836-8a97-456e-83ef-f093a89cb442 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:25.547590 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88eb836-8a97-456e-83ef-f093a89cb442 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-a88eb836-8a97-456e-83ef-f093a89cb442 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:25.579776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:25.581264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:25.582464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:25.582464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:25.582464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:25.582464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:25.582464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:25.582464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:25.582464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:25.582464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:25.588174 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415514e-e686-40e1-96c8-01dd731bbcae None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:25.596785 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 382/761] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:25.597739 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88eb836-8a97-456e-83ef-f093a89cb442 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:25.651484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452d90c-2664-44c8-8d9f-ebc139bb3d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:25.651484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452d90c-2664-44c8-8d9f-ebc139bb3d71 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:25.651484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452d90c-2664-44c8-8d9f-ebc139bb3d71 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:25.651484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452d90c-2664-44c8-8d9f-ebc139bb3d71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:25.670081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452d90c-2664-44c8-8d9f-ebc139bb3d71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:25.671896 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452d90c-2664-44c8-8d9f-ebc139bb3d71 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-4452d90c-2664-44c8-8d9f-ebc139bb3d71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:25.708993 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:25.710115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:25.711100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:25.711100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:25.711100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:25.711100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:25.711100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:25.711100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:25.711100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:25.711100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:25.716630 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 380/762] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:25.719606 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452d90c-2664-44c8-8d9f-ebc139bb3d71 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:25.882347 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88eb836-8a97-456e-83ef-f093a89cb442 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:25.918451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72874b2f-5e0c-4639-9590-9c53a714372c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:25.919128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72874b2f-5e0c-4639-9590-9c53a714372c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:25.919419 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72874b2f-5e0c-4639-9590-9c53a714372c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:25.919699 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72874b2f-5e0c-4639-9590-9c53a714372c None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:25.920550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72874b2f-5e0c-4639-9590-9c53a714372c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:25.920866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72874b2f-5e0c-4639-9590-9c53a714372c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:25.922402 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72874b2f-5e0c-4639-9590-9c53a714372c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:25.939972 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 383/763] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:25.941291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72874b2f-5e0c-4639-9590-9c53a714372c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:25.963221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452d90c-2664-44c8-8d9f-ebc139bb3d71 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:25.970928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:26.026953 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:26.045513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2026b814-2d58-4428-8260-2ff1ba6de4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:26.046678 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b814-2d58-4428-8260-2ff1ba6de4d0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:26.047200 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b814-2d58-4428-8260-2ff1ba6de4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:26.047636 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b814-2d58-4428-8260-2ff1ba6de4d0 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:26.050410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026b814-2d58-4428-8260-2ff1ba6de4d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:26.050945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026b814-2d58-4428-8260-2ff1ba6de4d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:26.052924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2026b814-2d58-4428-8260-2ff1ba6de4d0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:26.053335 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72874b2f-5e0c-4639-9590-9c53a714372c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:26.082526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 381/764] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:26.085259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52f1af-73ac-4c96-ac0e-85ae6d41f78f None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:26.085563 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52f1af-73ac-4c96-ac0e-85ae6d41f78f None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:26.085850 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52f1af-73ac-4c96-ac0e-85ae6d41f78f None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:26.091854 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 382/765] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:26.091854 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52f1af-73ac-4c96-ac0e-85ae6d41f78f None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:26.113107 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa2419-bc98-43d8-856c-51adb2db7128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:26.113804 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa2419-bc98-43d8-856c-51adb2db7128 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:26.114072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa2419-bc98-43d8-856c-51adb2db7128 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:26.114348 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa2419-bc98-43d8-856c-51adb2db7128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:26.115202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa2419-bc98-43d8-856c-51adb2db7128 None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=5f5896afd9434adebe8c1d8db70b28bf) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:26.115516 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa2419-bc98-43d8-856c-51adb2db7128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:26.116979 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa2419-bc98-43d8-856c-51adb2db7128 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:26.136702 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 384/766] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:25 2022] GET /identity/v3/role_assignments?user.id=5f5896afd9434adebe8c1d8db70b28bf&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:26.138447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69855b89-aeb6-4594-a26d-1012385fac80 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:26.138581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69855b89-aeb6-4594-a26d-1012385fac80 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:26.138813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69855b89-aeb6-4594-a26d-1012385fac80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:26.179013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-69855b89-aeb6-4594-a26d-1012385fac80 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:26.182956 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-69855b89-aeb6-4594-a26d-1012385fac80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:26.323821 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 385/767] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:26.324598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52f1af-73ac-4c96-ac0e-85ae6d41f78f None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:26.359425 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69855b89-aeb6-4594-a26d-1012385fac80 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:26.366298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956e69d-c516-4fd7-b718-ee5f9af9c092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:26.367225 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956e69d-c516-4fd7-b718-ee5f9af9c092 None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:26.367511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956e69d-c516-4fd7-b718-ee5f9af9c092 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:26.367791 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956e69d-c516-4fd7-b718-ee5f9af9c092 None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/e6a982ae15a6443b9b52529df47aa837/roles/8eaff9cd613a49b29548e90d7e3e9215` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:26.369905 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956e69d-c516-4fd7-b718-ee5f9af9c092 None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=e6a982ae15a6443b9b52529df47aa837, role_id=8eaff9cd613a49b29548e90d7e3e9215)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:26.371380 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956e69d-c516-4fd7-b718-ee5f9af9c092 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:26.397328 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:26.465332 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 383/768] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:29:26 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/e6a982ae15a6443b9b52529df47aa837/roles/8eaff9cd613a49b29548e90d7e3e9215 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:29:26.498186 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69855b89-aeb6-4594-a26d-1012385fac80 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:26.550130 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c416c5b-38fd-4c86-9202-759fb9f77b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:26.551158 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c416c5b-38fd-4c86-9202-759fb9f77b5c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:26.551496 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c416c5b-38fd-4c86-9202-759fb9f77b5c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:26.551778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c416c5b-38fd-4c86-9202-759fb9f77b5c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:26.552532 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c416c5b-38fd-4c86-9202-759fb9f77b5c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:26.554006 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c416c5b-38fd-4c86-9202-759fb9f77b5c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-1c416c5b-38fd-4c86-9202-759fb9f77b5c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:26.567346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:26.568733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:26.569889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 386/769] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 20:29:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:26.575106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956e69d-c516-4fd7-b718-ee5f9af9c092 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:26.668878 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:26.735714 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956e69d-c516-4fd7-b718-ee5f9af9c092 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:26.744211 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87192ba4-cb76-4c58-a7d0-6eff16420da9 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:26.744557 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87192ba4-cb76-4c58-a7d0-6eff16420da9 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:26.744828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87192ba4-cb76-4c58-a7d0-6eff16420da9 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:26.746459 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 387/770] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:26.754544 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437f8c5-7f28-4964-bab4-0f0683ff8f25 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:26.754823 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437f8c5-7f28-4964-bab4-0f0683ff8f25 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:26.755110 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437f8c5-7f28-4964-bab4-0f0683ff8f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:26.777439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-8437f8c5-7f28-4964-bab4-0f0683ff8f25 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:26.778655 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-8437f8c5-7f28-4964-bab4-0f0683ff8f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:26.791913 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c276b-8499-4365-b128-946959f2524f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:26.793136 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c276b-8499-4365-b128-946959f2524f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:26.793136 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c276b-8499-4365-b128-946959f2524f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:26.793354 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c276b-8499-4365-b128-946959f2524f None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:26.793991 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c276b-8499-4365-b128-946959f2524f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:26.794303 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c276b-8499-4365-b128-946959f2524f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:26.795697 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c276b-8499-4365-b128-946959f2524f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:26.802232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 384/771] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:29:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:26.808855 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c276b-8499-4365-b128-946959f2524f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:26.919061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 388/772] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:26.934523 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c276b-8499-4365-b128-946959f2524f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:26.955968 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:26.977441 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437f8c5-7f28-4964-bab4-0f0683ff8f25 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:26.982089 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85ebed-debe-40d2-a27e-32e44f515509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:26.983028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85ebed-debe-40d2-a27e-32e44f515509 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:26.983506 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85ebed-debe-40d2-a27e-32e44f515509 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:26.984029 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85ebed-debe-40d2-a27e-32e44f515509 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:26.984976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85ebed-debe-40d2-a27e-32e44f515509 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:26.985492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85ebed-debe-40d2-a27e-32e44f515509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:26.987687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85ebed-debe-40d2-a27e-32e44f515509 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:26.993745 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 385/773] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:29:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:27.005625 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85ebed-debe-40d2-a27e-32e44f515509 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:27.013114 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:27.105644 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8437f8c5-7f28-4964-bab4-0f0683ff8f25 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:27.145303 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be3255-fc70-42b5-a442-e5d2101bc02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:27.146961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3255-fc70-42b5-a442-e5d2101bc02d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:27.147445 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3255-fc70-42b5-a442-e5d2101bc02d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:27.147913 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3255-fc70-42b5-a442-e5d2101bc02d None admin] PATH_INFO: `/v3/services/network` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:27.148795 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be3255-fc70-42b5-a442-e5d2101bc02d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:27.150889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be3255-fc70-42b5-a442-e5d2101bc02d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-a6be3255-fc70-42b5-a442-e5d2101bc02d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:27.164560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:27.166992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:29:27.168839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:27.175192 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 389/774] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:26 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:27.181697 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be3255-fc70-42b5-a442-e5d2101bc02d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:27.309623 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be3255-fc70-42b5-a442-e5d2101bc02d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:27.315478 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:27.350884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbc2d3-6f20-4766-8065-84ebc58330d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:27.351819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbc2d3-6f20-4766-8065-84ebc58330d5 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:27.352288 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbc2d3-6f20-4766-8065-84ebc58330d5 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:27.352754 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbc2d3-6f20-4766-8065-84ebc58330d5 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:27.353966 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbc2d3-6f20-4766-8065-84ebc58330d5 None admin] RBAC: Adding query filter params (name=network) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:27.354476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbc2d3-6f20-4766-8065-84ebc58330d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:27.355983 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbc2d3-6f20-4766-8065-84ebc58330d5 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:27.361790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 390/775] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:27.364291 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85ebed-debe-40d2-a27e-32e44f515509 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:27.368797 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbc2d3-6f20-4766-8065-84ebc58330d5 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:27.421455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71c5d9-a7b0-4f1e-855e-72dc0d3221b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:27.422108 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71c5d9-a7b0-4f1e-855e-72dc0d3221b8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:27.422390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71c5d9-a7b0-4f1e-855e-72dc0d3221b8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:27.422667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71c5d9-a7b0-4f1e-855e-72dc0d3221b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:27.423521 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71c5d9-a7b0-4f1e-855e-72dc0d3221b8 None admin] RBAC: Adding query filter params (interface=public, service_id=6650e82ea6174801bce2c7e33b305e02, region_id=RegionOne) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:27.424074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71c5d9-a7b0-4f1e-855e-72dc0d3221b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:27.425348 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71c5d9-a7b0-4f1e-855e-72dc0d3221b8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:27.442487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 386/776] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:29:27 2022] GET /identity/v3/endpoints?service_id=6650e82ea6174801bce2c7e33b305e02&interface=public®ion_id=RegionOne => generated 194 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:27.449867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71c5d9-a7b0-4f1e-855e-72dc0d3221b8 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:27.495425 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbc2d3-6f20-4766-8065-84ebc58330d5 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:27.535886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:27.542374 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d148b1-f772-40b7-876d-01080e97888e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:27.543523 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d148b1-f772-40b7-876d-01080e97888e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:27.543874 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d148b1-f772-40b7-876d-01080e97888e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:27.544234 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d148b1-f772-40b7-876d-01080e97888e None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:27.545225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d148b1-f772-40b7-876d-01080e97888e None admin] RBAC: Adding query filter params (type=network) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:27.551927 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d148b1-f772-40b7-876d-01080e97888e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:27.551927 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d148b1-f772-40b7-876d-01080e97888e None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:27.560381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 391/777] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:27.560381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876d24a-4851-468f-b94a-57a0094bd3a0 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:27.560381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876d24a-4851-468f-b94a-57a0094bd3a0 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:27.560381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876d24a-4851-468f-b94a-57a0094bd3a0 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:27.560381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 392/778] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:27.564660 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876d24a-4851-468f-b94a-57a0094bd3a0 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:27.588052 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:27.626266 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71c5d9-a7b0-4f1e-855e-72dc0d3221b8 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:27.667429 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6a5a4-f9b6-4cd1-b2b5-fa40d605729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:27.668125 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a5a4-f9b6-4cd1-b2b5-fa40d605729b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:27.668438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a5a4-f9b6-4cd1-b2b5-fa40d605729b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:27.668717 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6a5a4-f9b6-4cd1-b2b5-fa40d605729b None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:27.669408 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6a5a4-f9b6-4cd1-b2b5-fa40d605729b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:27.671359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6a5a4-f9b6-4cd1-b2b5-fa40d605729b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:27.684443 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 387/779] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:29:27 2022] GET /identity/v3/services => generated 772 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:27.684443 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f86f2f-fa70-422a-ba9e-328982bdb3cc None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:27.684443 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f86f2f-fa70-422a-ba9e-328982bdb3cc None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:27.684443 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f86f2f-fa70-422a-ba9e-328982bdb3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:27.721099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-f8f86f2f-fa70-422a-ba9e-328982bdb3cc None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:27.722359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-f8f86f2f-fa70-422a-ba9e-328982bdb3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:27.729516 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876d24a-4851-468f-b94a-57a0094bd3a0 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:27.802274 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468aa52-911d-469c-94cf-f5c280b8bb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:27.802968 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468aa52-911d-469c-94cf-f5c280b8bb84 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:27.803257 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468aa52-911d-469c-94cf-f5c280b8bb84 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:27.803537 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468aa52-911d-469c-94cf-f5c280b8bb84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:27.804450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468aa52-911d-469c-94cf-f5c280b8bb84 None admin] RBAC: Adding query filter params (interface=public, service_id=51694a71dd694695a3bc5c0ed17367ee, region_id=RegionOne) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:27.804861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468aa52-911d-469c-94cf-f5c280b8bb84 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:27.806632 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468aa52-911d-469c-94cf-f5c280b8bb84 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:27.815608 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 393/780] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:29:27 2022] GET /identity/v3/endpoints?service_id=51694a71dd694695a3bc5c0ed17367ee&interface=public®ion_id=RegionOne => generated 194 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:27.821304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468aa52-911d-469c-94cf-f5c280b8bb84 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:27.871337 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 388/781] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:27.912121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:27.922074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f86f2f-fa70-422a-ba9e-328982bdb3cc None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:27.946824 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468aa52-911d-469c-94cf-f5c280b8bb84 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:27.973841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:27.986339 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d3165-ca2e-4f01-8a66-76881da1ce87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:27.987071 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d3165-ca2e-4f01-8a66-76881da1ce87 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:27.987356 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d3165-ca2e-4f01-8a66-76881da1ce87 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:27.987629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d3165-ca2e-4f01-8a66-76881da1ce87 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:27.988377 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d3165-ca2e-4f01-8a66-76881da1ce87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:27.989753 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d3165-ca2e-4f01-8a66-76881da1ce87 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:27.998041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 394/782] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:29:27 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:28.053441 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f86f2f-fa70-422a-ba9e-328982bdb3cc None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:28.093294 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fe6d0-d337-48cf-87d7-6c09779c8786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:28.094187 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fe6d0-d337-48cf-87d7-6c09779c8786 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:28.094477 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fe6d0-d337-48cf-87d7-6c09779c8786 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:28.094748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1fe6d0-d337-48cf-87d7-6c09779c8786 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:28.108005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1fe6d0-d337-48cf-87d7-6c09779c8786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-0d1fe6d0-d337-48cf-87d7-6c09779c8786 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:28.116693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:28.118010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:28.119182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 389/783] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:28.126195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d3165-ca2e-4f01-8a66-76881da1ce87 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:28.261348 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d3165-ca2e-4f01-8a66-76881da1ce87 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:28.299057 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afcbbe-1950-4a0d-b4a8-f6d78dcfbad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:28.299783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afcbbe-1950-4a0d-b4a8-f6d78dcfbad0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:28.300071 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afcbbe-1950-4a0d-b4a8-f6d78dcfbad0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:28.300366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afcbbe-1950-4a0d-b4a8-f6d78dcfbad0 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:28.301139 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afcbbe-1950-4a0d-b4a8-f6d78dcfbad0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:28.301453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afcbbe-1950-4a0d-b4a8-f6d78dcfbad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:28.302834 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27afcbbe-1950-4a0d-b4a8-f6d78dcfbad0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:28.305275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:28.313147 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 395/784] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:29:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:28.319276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fe6d0-d337-48cf-87d7-6c09779c8786 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:28.427608 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:28.441596 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1fe6d0-d337-48cf-87d7-6c09779c8786 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:28.494287 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e6a05-55f4-4a20-8070-4c09c55ae69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:28.495487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e6a05-55f4-4a20-8070-4c09c55ae69a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:28.495996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e6a05-55f4-4a20-8070-4c09c55ae69a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:28.496517 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e6a05-55f4-4a20-8070-4c09c55ae69a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:28.507779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e6a05-55f4-4a20-8070-4c09c55ae69a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:28.517684 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e6a05-55f4-4a20-8070-4c09c55ae69a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-607e6a05-55f4-4a20-8070-4c09c55ae69a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:28.534116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:28.535144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:28.536177 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:28.536177 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:28.536177 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:28.536177 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:28.538532 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 390/785] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:28.544142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afcbbe-1950-4a0d-b4a8-f6d78dcfbad0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:28.707162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afcbbe-1950-4a0d-b4a8-f6d78dcfbad0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:28.747553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd4296-0022-41e5-9495-eaa6a1769b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:28.748234 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd4296-0022-41e5-9495-eaa6a1769b74 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:28.748538 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd4296-0022-41e5-9495-eaa6a1769b74 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:28.748812 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd4296-0022-41e5-9495-eaa6a1769b74 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:28.749544 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd4296-0022-41e5-9495-eaa6a1769b74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:28.749852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd4296-0022-41e5-9495-eaa6a1769b74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:28.751299 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd4296-0022-41e5-9495-eaa6a1769b74 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:28.768897 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 396/786] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:28.774405 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e6a05-55f4-4a20-8070-4c09c55ae69a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:28.825685 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d407c2-8079-49b9-b948-cf1cdf47ffa9 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:28.826038 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d407c2-8079-49b9-b948-cf1cdf47ffa9 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:28.826246 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d407c2-8079-49b9-b948-cf1cdf47ffa9 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:28.827821 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 397/787] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:29:28.834342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad3510-aa8d-4b48-a1ea-6f11060ff686 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:28.834633 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad3510-aa8d-4b48-a1ea-6f11060ff686 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:28.834914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad3510-aa8d-4b48-a1ea-6f11060ff686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:28.873438 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-0fad3510-aa8d-4b48-a1ea-6f11060ff686 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:28.874646 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-0fad3510-aa8d-4b48-a1ea-6f11060ff686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:28.917321 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e6a05-55f4-4a20-8070-4c09c55ae69a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:28.957171 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda603de-1cb0-4004-b588-2209be356607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:28.958245 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda603de-1cb0-4004-b588-2209be356607 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:28.958536 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda603de-1cb0-4004-b588-2209be356607 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:28.958815 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda603de-1cb0-4004-b588-2209be356607 None admin] PATH_INFO: `/v3/users/swift` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:28.968037 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda603de-1cb0-4004-b588-2209be356607 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:28.969676 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda603de-1cb0-4004-b588-2209be356607 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-fda603de-1cb0-4004-b588-2209be356607 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:28.987958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:28.989138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:28.990173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:28.990173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:28.990173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:28.990173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:28.990173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:28.990173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:28.990173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:28.990173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:28.990173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:28.990173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:28.990173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 391/788] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:28.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:28.997757 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda603de-1cb0-4004-b588-2209be356607 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:29.008481 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:29.042314 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 398/789] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) May 19 20:29:29.070096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad3510-aa8d-4b48-a1ea-6f11060ff686 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:29.169165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda603de-1cb0-4004-b588-2209be356607 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:29.208095 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca5654-bf5c-4267-abb1-3f827568fa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:29.209087 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca5654-bf5c-4267-abb1-3f827568fa7d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:29.209376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca5654-bf5c-4267-abb1-3f827568fa7d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:29.209658 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca5654-bf5c-4267-abb1-3f827568fa7d None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:29.210511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca5654-bf5c-4267-abb1-3f827568fa7d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:29.210825 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca5654-bf5c-4267-abb1-3f827568fa7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:29.210986 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad3510-aa8d-4b48-a1ea-6f11060ff686 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:29.212279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca5654-bf5c-4267-abb1-3f827568fa7d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:29.236157 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 392/790] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:29:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:29.237656 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c489a2a-7266-45ac-a897-6c81df5c3933 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:29.238429 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c489a2a-7266-45ac-a897-6c81df5c3933 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:29.238707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c489a2a-7266-45ac-a897-6c81df5c3933 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:29.240600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 393/791] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:29.242310 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c489a2a-7266-45ac-a897-6c81df5c3933 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:29.262325 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da84ccc-4206-45cf-a74a-d1a2219d3986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:29.263491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da84ccc-4206-45cf-a74a-d1a2219d3986 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:29.263925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da84ccc-4206-45cf-a74a-d1a2219d3986 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:29.264265 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da84ccc-4206-45cf-a74a-d1a2219d3986 None admin] PATH_INFO: `/v3/roles/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:29.279156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da84ccc-4206-45cf-a74a-d1a2219d3986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-9da84ccc-4206-45cf-a74a-d1a2219d3986 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:29.289671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:29.290959 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:29.292397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:29.297750 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 399/792] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:29.297750 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc30a2-1fff-49cf-b936-5d5efcc9e3d9 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:29.297750 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc30a2-1fff-49cf-b936-5d5efcc9e3d9 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:29.297750 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedc30a2-1fff-49cf-b936-5d5efcc9e3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:29.316972 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-cedc30a2-1fff-49cf-b936-5d5efcc9e3d9 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:29.318847 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-cedc30a2-1fff-49cf-b936-5d5efcc9e3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:29.332075 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:29.367569 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:29.376628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c489a2a-7266-45ac-a897-6c81df5c3933 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:29.393517 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 400/793] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:29.393517 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc30a2-1fff-49cf-b936-5d5efcc9e3d9 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:29.437601 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd47a6a-007d-49a5-8750-f38d2420fb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:29.439225 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd47a6a-007d-49a5-8750-f38d2420fb92 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:29.439661 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd47a6a-007d-49a5-8750-f38d2420fb92 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:29.440101 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd47a6a-007d-49a5-8750-f38d2420fb92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:29.456707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd47a6a-007d-49a5-8750-f38d2420fb92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:29.458653 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd47a6a-007d-49a5-8750-f38d2420fb92 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-edd47a6a-007d-49a5-8750-f38d2420fb92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:29.478498 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:29.479801 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:29.480979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:29.480979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:29.480979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:29.480979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:29.483137 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 394/794] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:29.484272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd47a6a-007d-49a5-8750-f38d2420fb92 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:29.516718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedc30a2-1fff-49cf-b936-5d5efcc9e3d9 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:29.557453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70b5ff-aa15-4224-9da2-c72c54694c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:29.558050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70b5ff-aa15-4224-9da2-c72c54694c02 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:29.558423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70b5ff-aa15-4224-9da2-c72c54694c02 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:29.558701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70b5ff-aa15-4224-9da2-c72c54694c02 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:29.559518 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70b5ff-aa15-4224-9da2-c72c54694c02 None admin] RBAC: Adding query filter params (name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:29.559893 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70b5ff-aa15-4224-9da2-c72c54694c02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:29.561549 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70b5ff-aa15-4224-9da2-c72c54694c02 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:29.574431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 401/795] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:29.574431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70b5ff-aa15-4224-9da2-c72c54694c02 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:29.603949 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd47a6a-007d-49a5-8750-f38d2420fb92 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:29.619937 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:29.631761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:29.649819 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44650b-1896-417d-8637-4a2e9d28ee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:29.651435 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44650b-1896-417d-8637-4a2e9d28ee28 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:29.651986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44650b-1896-417d-8637-4a2e9d28ee28 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:29.652581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44650b-1896-417d-8637-4a2e9d28ee28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:29.669670 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44650b-1896-417d-8637-4a2e9d28ee28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-8c44650b-1896-417d-8637-4a2e9d28ee28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:29.680869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:29.682138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:29.683073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:29.689653 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 395/796] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:29.689653 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44650b-1896-417d-8637-4a2e9d28ee28 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:29.714883 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70b5ff-aa15-4224-9da2-c72c54694c02 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:29.761409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468f0ac-e3a1-4cf6-8dd2-5994011d7237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:29.762678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468f0ac-e3a1-4cf6-8dd2-5994011d7237 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:29.763395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468f0ac-e3a1-4cf6-8dd2-5994011d7237 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:29.764282 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468f0ac-e3a1-4cf6-8dd2-5994011d7237 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:29.765567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468f0ac-e3a1-4cf6-8dd2-5994011d7237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:29.766084 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468f0ac-e3a1-4cf6-8dd2-5994011d7237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:29.768504 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468f0ac-e3a1-4cf6-8dd2-5994011d7237 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:29.785097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 402/797] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:29.787032 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468f0ac-e3a1-4cf6-8dd2-5994011d7237 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:29.807713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44650b-1896-417d-8637-4a2e9d28ee28 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:29.853729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004bf46-02eb-4fe8-9536-5dd369a44624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:29.855046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004bf46-02eb-4fe8-9536-5dd369a44624 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:29.855787 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004bf46-02eb-4fe8-9536-5dd369a44624 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:29.856724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004bf46-02eb-4fe8-9536-5dd369a44624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:29.872881 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004bf46-02eb-4fe8-9536-5dd369a44624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:29.874666 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004bf46-02eb-4fe8-9536-5dd369a44624 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-0004bf46-02eb-4fe8-9536-5dd369a44624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:29.899912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:29.900990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:29.907222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:29.907222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:29.907222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:29.907222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:29.907222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 396/798] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:29.907222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004bf46-02eb-4fe8-9536-5dd369a44624 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:29.908212 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:29.911251 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468f0ac-e3a1-4cf6-8dd2-5994011d7237 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:29.947177 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:29.947381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b36093-e26c-4e00-8541-9eb08e3f3c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:29.948071 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b36093-e26c-4e00-8541-9eb08e3f3c18 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:29.948475 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b36093-e26c-4e00-8541-9eb08e3f3c18 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:29.948667 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b36093-e26c-4e00-8541-9eb08e3f3c18 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:29.949437 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b36093-e26c-4e00-8541-9eb08e3f3c18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:29.949994 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b36093-e26c-4e00-8541-9eb08e3f3c18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:29.951285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b36093-e26c-4e00-8541-9eb08e3f3c18 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:29.966228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 403/799] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:29:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:29.966228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b36093-e26c-4e00-8541-9eb08e3f3c18 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:30.057364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004bf46-02eb-4fe8-9536-5dd369a44624 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:30.090364 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b36093-e26c-4e00-8541-9eb08e3f3c18 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:30.104039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd55dc-c9ed-413b-860c-5c6fc9239b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:30.105928 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd55dc-c9ed-413b-860c-5c6fc9239b8c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.106436 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd55dc-c9ed-413b-860c-5c6fc9239b8c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.107203 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd55dc-c9ed-413b-860c-5c6fc9239b8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.118029 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd55dc-c9ed-413b-860c-5c6fc9239b8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:30.119508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd55dc-c9ed-413b-860c-5c6fc9239b8c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:30.131313 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cadb50-79cf-44a6-a4eb-bd7c383fae45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:30.131992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cadb50-79cf-44a6-a4eb-bd7c383fae45 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.132277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cadb50-79cf-44a6-a4eb-bd7c383fae45 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.132583 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cadb50-79cf-44a6-a4eb-bd7c383fae45 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.133319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cadb50-79cf-44a6-a4eb-bd7c383fae45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:30.133628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cadb50-79cf-44a6-a4eb-bd7c383fae45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:30.135051 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cadb50-79cf-44a6-a4eb-bd7c383fae45 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-fcdd55dc-c9ed-413b-860c-5c6fc9239b8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:30.139671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:30.140703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:30.141777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:30.141777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:30.141777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:30.141777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:30.141777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:30.141777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:30.141777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:30.141777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:30.145062 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 397/800] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:30.146713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd55dc-c9ed-413b-860c-5c6fc9239b8c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:30.153926 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 404/801] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:30.153926 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cadb50-79cf-44a6-a4eb-bd7c383fae45 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:30.178596 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:30.266882 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cadb50-79cf-44a6-a4eb-bd7c383fae45 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:30.278148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:30.300979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd55dc-c9ed-413b-860c-5c6fc9239b8c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:30.307185 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d6122-03c3-4cd9-81ed-a1c15b83f53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:30.308009 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d6122-03c3-4cd9-81ed-a1c15b83f53b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.308631 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d6122-03c3-4cd9-81ed-a1c15b83f53b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.309296 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d6122-03c3-4cd9-81ed-a1c15b83f53b None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.310288 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d6122-03c3-4cd9-81ed-a1c15b83f53b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:30.310980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d6122-03c3-4cd9-81ed-a1c15b83f53b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:30.312715 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d6122-03c3-4cd9-81ed-a1c15b83f53b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:30.329747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 405/802] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:30.330830 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d6122-03c3-4cd9-81ed-a1c15b83f53b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:30.348487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c5638-3b4a-48ae-aa8f-4def7b98bae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:30.349824 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c5638-3b4a-48ae-aa8f-4def7b98bae8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.350263 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c5638-3b4a-48ae-aa8f-4def7b98bae8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.351205 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c5638-3b4a-48ae-aa8f-4def7b98bae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.366665 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c5638-3b4a-48ae-aa8f-4def7b98bae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:30.369094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c5638-3b4a-48ae-aa8f-4def7b98bae8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c25c5638-3b4a-48ae-aa8f-4def7b98bae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:30.393083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:30.394298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:30.395511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:30.395511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:30.395511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:30.395511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:30.402594 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 398/803] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:30.402594 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c5638-3b4a-48ae-aa8f-4def7b98bae8 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:30.442954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:30.466364 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d6122-03c3-4cd9-81ed-a1c15b83f53b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:30.518513 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b34dd-f6fe-4457-9554-c3070c286da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:30.520014 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b34dd-f6fe-4457-9554-c3070c286da9 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.520831 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b34dd-f6fe-4457-9554-c3070c286da9 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.521730 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b34dd-f6fe-4457-9554-c3070c286da9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.534379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b34dd-f6fe-4457-9554-c3070c286da9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:30.537548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b34dd-f6fe-4457-9554-c3070c286da9 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:30.542232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c5638-3b4a-48ae-aa8f-4def7b98bae8 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-159b34dd-f6fe-4457-9554-c3070c286da9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:30.561820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:30.562924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:30.563992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:30.563992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:30.563992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:30.563992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:30.563992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:30.563992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:30.563992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:30.563992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:30.563992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:30.563992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:30.572092 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 406/804] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:29:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:30.572092 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b34dd-f6fe-4457-9554-c3070c286da9 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:30.589303 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d51d51-d67e-41a7-931d-b5b499fa43fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:30.590851 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d51d51-d67e-41a7-931d-b5b499fa43fa None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.591407 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d51d51-d67e-41a7-931d-b5b499fa43fa None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.591945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d51d51-d67e-41a7-931d-b5b499fa43fa None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/14034b78ec5e4999848a3fffb2964587/roles/8eaff9cd613a49b29548e90d7e3e9215` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.595872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d51d51-d67e-41a7-931d-b5b499fa43fa None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=14034b78ec5e4999848a3fffb2964587, role_id=8eaff9cd613a49b29548e90d7e3e9215)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:30.597633 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d51d51-d67e-41a7-931d-b5b499fa43fa None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:30.676500 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:30.681602 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 399/805] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:29:30 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/14034b78ec5e4999848a3fffb2964587/roles/8eaff9cd613a49b29548e90d7e3e9215 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:29:30.692839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d51d51-d67e-41a7-931d-b5b499fa43fa None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:30.692972 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b34dd-f6fe-4457-9554-c3070c286da9 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:30.724195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe4de8-dc0f-48dc-8c0a-bce42d832bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:30.725016 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe4de8-dc0f-48dc-8c0a-bce42d832bb4 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.725314 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe4de8-dc0f-48dc-8c0a-bce42d832bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.725595 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe4de8-dc0f-48dc-8c0a-bce42d832bb4 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.726345 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe4de8-dc0f-48dc-8c0a-bce42d832bb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:30.726664 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe4de8-dc0f-48dc-8c0a-bce42d832bb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:30.728084 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe4de8-dc0f-48dc-8c0a-bce42d832bb4 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:30.746238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 407/806] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:30.746238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f7edd-0896-4848-a031-1c6ef586a42a None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.746238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f7edd-0896-4848-a031-1c6ef586a42a None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.746238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f7edd-0896-4848-a031-1c6ef586a42a None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.746238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 408/807] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:30.746238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32a50e-038e-492e-9ea6-404beb93c0e8 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.746839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32a50e-038e-492e-9ea6-404beb93c0e8 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.746916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32a50e-038e-492e-9ea6-404beb93c0e8 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.748958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 409/808] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:30.749895 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32a50e-038e-492e-9ea6-404beb93c0e8 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:30.753946 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:30.807951 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d51d51-d67e-41a7-931d-b5b499fa43fa None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:30.851289 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9358ec-fd7d-44ab-bb23-222a648eb5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:30.851986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9358ec-fd7d-44ab-bb23-222a648eb5eb None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.852275 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9358ec-fd7d-44ab-bb23-222a648eb5eb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.852999 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9358ec-fd7d-44ab-bb23-222a648eb5eb None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.853473 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9358ec-fd7d-44ab-bb23-222a648eb5eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:30.853787 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9358ec-fd7d-44ab-bb23-222a648eb5eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:30.855282 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9358ec-fd7d-44ab-bb23-222a648eb5eb None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:30.866962 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32a50e-038e-492e-9ea6-404beb93c0e8 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:30.877414 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 400/809] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:29:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:30.878575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e3a82-5984-4e74-af9b-0712f46f2006 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.878873 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e3a82-5984-4e74-af9b-0712f46f2006 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.879150 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e3a82-5984-4e74-af9b-0712f46f2006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.900502 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-166e3a82-5984-4e74-af9b-0712f46f2006 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:30.901692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-166e3a82-5984-4e74-af9b-0712f46f2006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:30.913709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add932d-f313-4c87-8fd9-7fea9f42281c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:30.914506 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add932d-f313-4c87-8fd9-7fea9f42281c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.914807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add932d-f313-4c87-8fd9-7fea9f42281c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.915079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add932d-f313-4c87-8fd9-7fea9f42281c None admin] PATH_INFO: `/v3/users/nova` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.924361 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add932d-f313-4c87-8fd9-7fea9f42281c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:30.925954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add932d-f313-4c87-8fd9-7fea9f42281c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-1add932d-f313-4c87-8fd9-7fea9f42281c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:30.942230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:30.943409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:30.944494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:30.944494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:30.944494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:30.944494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:30.944494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:30.944494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:30.944494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:30.944494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:30.944494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:29:30.944494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:30.944494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 410/810] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:29:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:30.945162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c34c7-29c0-4d42-bde1-fd4ebf452c06 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:30.945162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c34c7-29c0-4d42-bde1-fd4ebf452c06 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:30.945162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c34c7-29c0-4d42-bde1-fd4ebf452c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:30.963051 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:30.970773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-350c34c7-29c0-4d42-bde1-fd4ebf452c06 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:30.974766 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-350c34c7-29c0-4d42-bde1-fd4ebf452c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:30.988531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 401/811] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:30.988531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e3a82-5984-4e74-af9b-0712f46f2006 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:31.045035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:31.076358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 411/812] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:31.076358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c34c7-29c0-4d42-bde1-fd4ebf452c06 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:31.088200 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e3a82-5984-4e74-af9b-0712f46f2006 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:31.125549 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c92a7-e1fa-4d06-8809-041bd75edf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:31.127754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c92a7-e1fa-4d06-8809-041bd75edf89 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:31.127754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c92a7-e1fa-4d06-8809-041bd75edf89 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:31.128049 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c92a7-e1fa-4d06-8809-041bd75edf89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:31.146024 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c92a7-e1fa-4d06-8809-041bd75edf89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:31.147501 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c92a7-e1fa-4d06-8809-041bd75edf89 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-7f0c92a7-e1fa-4d06-8809-041bd75edf89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:31.167607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:31.168754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:31.169753 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:31.169753 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:31.169753 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:31.169753 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:31.169753 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 402/813] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:31.169753 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c92a7-e1fa-4d06-8809-041bd75edf89 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:31.225034 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:31.225615 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c34c7-29c0-4d42-bde1-fd4ebf452c06 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:31.265235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e906d8-8c6b-4a6c-855a-fb54db4a70c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:31.266028 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e906d8-8c6b-4a6c-855a-fb54db4a70c1 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:31.266321 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e906d8-8c6b-4a6c-855a-fb54db4a70c1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:31.266603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e906d8-8c6b-4a6c-855a-fb54db4a70c1 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:31.268038 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e906d8-8c6b-4a6c-855a-fb54db4a70c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:31.268604 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e906d8-8c6b-4a6c-855a-fb54db4a70c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:31.269826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e906d8-8c6b-4a6c-855a-fb54db4a70c1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:31.292691 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 412/814] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:29:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:31.293742 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e906d8-8c6b-4a6c-855a-fb54db4a70c1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:31.303567 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c92a7-e1fa-4d06-8809-041bd75edf89 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:31.341866 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5b6d5-4ccb-40da-a5e9-8814ce44df4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:31.342675 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5b6d5-4ccb-40da-a5e9-8814ce44df4b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:31.342958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5b6d5-4ccb-40da-a5e9-8814ce44df4b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:31.343235 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5b6d5-4ccb-40da-a5e9-8814ce44df4b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:31.343881 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5b6d5-4ccb-40da-a5e9-8814ce44df4b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:31.345314 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5b6d5-4ccb-40da-a5e9-8814ce44df4b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:31.349958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-69c5b6d5-4ccb-40da-a5e9-8814ce44df4b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:31.358257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:31.364053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:31.365139 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 403/815] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 20:29:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:31.366229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5b6d5-4ccb-40da-a5e9-8814ce44df4b None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:31.408662 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e906d8-8c6b-4a6c-855a-fb54db4a70c1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:31.453144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a8da0-78ad-48df-b0c7-0c18318530e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:31.453896 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a8da0-78ad-48df-b0c7-0c18318530e1 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:31.454186 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a8da0-78ad-48df-b0c7-0c18318530e1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:31.455214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a8da0-78ad-48df-b0c7-0c18318530e1 None admin] PATH_INFO: `/v3/services/network` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:31.455768 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a8da0-78ad-48df-b0c7-0c18318530e1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:31.457166 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a8da0-78ad-48df-b0c7-0c18318530e1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:31.472616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5b6d5-4ccb-40da-a5e9-8814ce44df4b None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-d66a8da0-78ad-48df-b0c7-0c18318530e1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:31.473044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:31.474117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:29:31.475102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:31.475925 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:31.476002 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 413/816] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:31 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:31.476084 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a8da0-78ad-48df-b0c7-0c18318530e1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:31.517430 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cd9e8-f68c-41bc-a53d-beddb9ef5ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:31.518167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cd9e8-f68c-41bc-a53d-beddb9ef5ffc None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:31.518466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cd9e8-f68c-41bc-a53d-beddb9ef5ffc None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:31.518770 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cd9e8-f68c-41bc-a53d-beddb9ef5ffc None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:31.519484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6cd9e8-f68c-41bc-a53d-beddb9ef5ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:31.519794 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6cd9e8-f68c-41bc-a53d-beddb9ef5ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:31.521233 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6cd9e8-f68c-41bc-a53d-beddb9ef5ffc None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:31.534555 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 404/817] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:31.535503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cd9e8-f68c-41bc-a53d-beddb9ef5ffc None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:31.602024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a8da0-78ad-48df-b0c7-0c18318530e1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:31.616639 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:31.639152 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd51e6-cd3d-45c7-8fd7-0b8725368c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:31.639844 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd51e6-cd3d-45c7-8fd7-0b8725368c87 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:31.640132 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd51e6-cd3d-45c7-8fd7-0b8725368c87 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:31.640462 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd51e6-cd3d-45c7-8fd7-0b8725368c87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:31.654634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cd9e8-f68c-41bc-a53d-beddb9ef5ffc None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:31.692862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9ab5b-1e6f-4fc7-bafc-24a38ad48707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:31.693535 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ab5b-1e6f-4fc7-bafc-24a38ad48707 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:31.693821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ab5b-1e6f-4fc7-bafc-24a38ad48707 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:31.694218 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9ab5b-1e6f-4fc7-bafc-24a38ad48707 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:31.694928 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9ab5b-1e6f-4fc7-bafc-24a38ad48707 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:31.695239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9ab5b-1e6f-4fc7-bafc-24a38ad48707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:31.696703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9ab5b-1e6f-4fc7-bafc-24a38ad48707 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:31.706238 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 405/818] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:29:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:31.706238 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9ab5b-1e6f-4fc7-bafc-24a38ad48707 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:31.714775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd51e6-cd3d-45c7-8fd7-0b8725368c87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:31.716357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd51e6-cd3d-45c7-8fd7-0b8725368c87 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:31.735187 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:31.817228 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9ab5b-1e6f-4fc7-bafc-24a38ad48707 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:31.828422 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 414/819] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:29:31.829334 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd51e6-cd3d-45c7-8fd7-0b8725368c87 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:31.847816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:31.863095 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e924c-58e9-4a42-b366-6aeccefb0659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:31.863773 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e924c-58e9-4a42-b366-6aeccefb0659 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:31.864056 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e924c-58e9-4a42-b366-6aeccefb0659 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:31.864351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e924c-58e9-4a42-b366-6aeccefb0659 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:31.865078 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e924c-58e9-4a42-b366-6aeccefb0659 None admin] RBAC: Adding query filter params (name=network) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:31.865635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e924c-58e9-4a42-b366-6aeccefb0659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:31.866753 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e924c-58e9-4a42-b366-6aeccefb0659 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:31.871791 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 406/820] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:31.872507 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e924c-58e9-4a42-b366-6aeccefb0659 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:31.946264 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd51e6-cd3d-45c7-8fd7-0b8725368c87 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:31.976485 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:31.986964 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e924c-58e9-4a42-b366-6aeccefb0659 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:31.989004 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde852b-c713-4952-9b68-2dd8bddd418f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:31.989973 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde852b-c713-4952-9b68-2dd8bddd418f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:31.990109 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde852b-c713-4952-9b68-2dd8bddd418f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:31.990358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde852b-c713-4952-9b68-2dd8bddd418f None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:31.998189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde852b-c713-4952-9b68-2dd8bddd418f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:31.999366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde852b-c713-4952-9b68-2dd8bddd418f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-5fde852b-c713-4952-9b68-2dd8bddd418f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:32.013789 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:32.014810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:32.015793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:32.015793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:32.015793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:32.015793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:32.015793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:32.015793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:32.015793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:32.015793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:32.015793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 415/821] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:32.015793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde852b-c713-4952-9b68-2dd8bddd418f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:32.028075 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7b383-02ee-4932-bb9b-ccacfe86b862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:32.028803 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7b383-02ee-4932-bb9b-ccacfe86b862 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:32.029088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7b383-02ee-4932-bb9b-ccacfe86b862 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:32.029361 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7b383-02ee-4932-bb9b-ccacfe86b862 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:32.030068 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7b383-02ee-4932-bb9b-ccacfe86b862 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:32.030381 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7b383-02ee-4932-bb9b-ccacfe86b862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:32.031761 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7b383-02ee-4932-bb9b-ccacfe86b862 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:32.037695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 407/822] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:29:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:32.038543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7b383-02ee-4932-bb9b-ccacfe86b862 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:32.102703 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:32.127955 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde852b-c713-4952-9b68-2dd8bddd418f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:32.145250 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7b383-02ee-4932-bb9b-ccacfe86b862 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:32.163364 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5867b-4e2a-4d16-90d9-bb1671381ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:32.164253 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5867b-4e2a-4d16-90d9-bb1671381ade None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:32.164893 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5867b-4e2a-4d16-90d9-bb1671381ade None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:32.165031 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5867b-4e2a-4d16-90d9-bb1671381ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:32.177820 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5867b-4e2a-4d16-90d9-bb1671381ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:32.179327 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5867b-4e2a-4d16-90d9-bb1671381ade None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:32.182232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb467bf-e563-4710-9120-9dc86838f17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:32.182899 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb467bf-e563-4710-9120-9dc86838f17a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:32.183178 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb467bf-e563-4710-9120-9dc86838f17a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:32.183451 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb467bf-e563-4710-9120-9dc86838f17a None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:32.184153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb467bf-e563-4710-9120-9dc86838f17a None admin] RBAC: Adding query filter params (type=network) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:32.184492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb467bf-e563-4710-9120-9dc86838f17a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:32.185856 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb467bf-e563-4710-9120-9dc86838f17a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:32.191132 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 408/823] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:32.191918 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb467bf-e563-4710-9120-9dc86838f17a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-c7a5867b-4e2a-4d16-90d9-bb1671381ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:32.197916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:32.198889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:32.199879 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:32.199879 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:32.199879 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:32.199879 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:32.203582 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 416/824] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:32.204993 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5867b-4e2a-4d16-90d9-bb1671381ade None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:32.217737 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:32.307873 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb467bf-e563-4710-9120-9dc86838f17a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:32.323701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5867b-4e2a-4d16-90d9-bb1671381ade None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:32.347422 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c78e57-8eeb-432a-af7a-6b3d22d6ac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:32.348098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c78e57-8eeb-432a-af7a-6b3d22d6ac69 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:32.348410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c78e57-8eeb-432a-af7a-6b3d22d6ac69 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:32.348693 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c78e57-8eeb-432a-af7a-6b3d22d6ac69 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:32.349486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c78e57-8eeb-432a-af7a-6b3d22d6ac69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:32.349795 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c78e57-8eeb-432a-af7a-6b3d22d6ac69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:32.351439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c78e57-8eeb-432a-af7a-6b3d22d6ac69 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:32.351639 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:32.360851 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc60ec2-497e-497f-bba8-3635677ed034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:32.367303 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc60ec2-497e-497f-bba8-3635677ed034 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:32.367303 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc60ec2-497e-497f-bba8-3635677ed034 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:32.367303 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc60ec2-497e-497f-bba8-3635677ed034 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:32.367303 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc60ec2-497e-497f-bba8-3635677ed034 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:32.367694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 409/825] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:32.367795 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc60ec2-497e-497f-bba8-3635677ed034 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:32.367890 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c78e57-8eeb-432a-af7a-6b3d22d6ac69 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:32.450295 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:32.474503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c78e57-8eeb-432a-af7a-6b3d22d6ac69 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:32.507475 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6e030-bd3d-48c0-b94b-3f36fa207427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:32.508160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6e030-bd3d-48c0-b94b-3f36fa207427 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:32.508476 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6e030-bd3d-48c0-b94b-3f36fa207427 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:32.508750 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6e030-bd3d-48c0-b94b-3f36fa207427 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:32.509488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6e030-bd3d-48c0-b94b-3f36fa207427 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:32.510935 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6e030-bd3d-48c0-b94b-3f36fa207427 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:32.585006 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 417/826] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:29:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:32.585760 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc60ec2-497e-497f-bba8-3635677ed034 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:32.720355 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 410/827] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:29:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:32.722112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6e030-bd3d-48c0-b94b-3f36fa207427 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:32.733483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc60ec2-497e-497f-bba8-3635677ed034 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:32.800790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1610023-b90a-448d-91ba-10b05fe9aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:32.801508 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1610023-b90a-448d-91ba-10b05fe9aa64 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:32.801844 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1610023-b90a-448d-91ba-10b05fe9aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:32.802248 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1610023-b90a-448d-91ba-10b05fe9aa64 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:32.803169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1610023-b90a-448d-91ba-10b05fe9aa64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:32.803545 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1610023-b90a-448d-91ba-10b05fe9aa64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:32.805048 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1610023-b90a-448d-91ba-10b05fe9aa64 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:32.823740 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 418/828] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:32.830115 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1610023-b90a-448d-91ba-10b05fe9aa64 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:32.874681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:32.892541 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6e030-bd3d-48c0-b94b-3f36fa207427 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:32.939544 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da178fd2-9896-4dd0-9d23-2a50a23c0b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:32.940513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da178fd2-9896-4dd0-9d23-2a50a23c0b3b None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:32.940851 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da178fd2-9896-4dd0-9d23-2a50a23c0b3b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:32.941134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da178fd2-9896-4dd0-9d23-2a50a23c0b3b None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/5f5896afd9434adebe8c1d8db70b28bf/roles/ec2a4317dc7343208d439d5f2d44d6c3` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:32.943941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:32.962844 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da178fd2-9896-4dd0-9d23-2a50a23c0b3b None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=5f5896afd9434adebe8c1d8db70b28bf, role_id=ec2a4317dc7343208d439d5f2d44d6c3)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:32.964225 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da178fd2-9896-4dd0-9d23-2a50a23c0b3b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:32.967180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1610023-b90a-448d-91ba-10b05fe9aa64 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:33.010437 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02263891-3048-44d1-920b-5f52f9501f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:33.011385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02263891-3048-44d1-920b-5f52f9501f94 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:33.011676 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02263891-3048-44d1-920b-5f52f9501f94 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:33.011969 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02263891-3048-44d1-920b-5f52f9501f94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:33.020289 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02263891-3048-44d1-920b-5f52f9501f94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:33.021816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02263891-3048-44d1-920b-5f52f9501f94 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-02263891-3048-44d1-920b-5f52f9501f94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:33.040426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:33.041737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:33.042796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:33.042796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:33.042796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:33.042796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:33.042796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:33.042796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:33.042796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:33.042796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:33.042796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 419/829] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:33.043350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47043-cf30-42e4-bcd6-86e1c89ea78a None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:33.043425 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47043-cf30-42e4-bcd6-86e1c89ea78a None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:33.044140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a47043-cf30-42e4-bcd6-86e1c89ea78a None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:33.045129 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 420/830] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:33.046292 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 411/831] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:29:32 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/5f5896afd9434adebe8c1d8db70b28bf/roles/ec2a4317dc7343208d439d5f2d44d6c3 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:29:33.054378 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed081d5b-536e-44c0-a910-e45887691bf7 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:33.055016 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed081d5b-536e-44c0-a910-e45887691bf7 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:33.055521 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed081d5b-536e-44c0-a910-e45887691bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:33.057437 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da178fd2-9896-4dd0-9d23-2a50a23c0b3b None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:33.079158 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-ed081d5b-536e-44c0-a910-e45887691bf7 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:33.081383 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-ed081d5b-536e-44c0-a910-e45887691bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:33.186328 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 421/832] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:33.203620 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da178fd2-9896-4dd0-9d23-2a50a23c0b3b None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:33.214904 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed081d5b-536e-44c0-a910-e45887691bf7 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:33.244757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:33.249662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee889ee4-f8b4-4c4b-b7c0-6072fb56cf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:33.250699 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889ee4-f8b4-4c4b-b7c0-6072fb56cf54 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:33.251262 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889ee4-f8b4-4c4b-b7c0-6072fb56cf54 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:33.251533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889ee4-f8b4-4c4b-b7c0-6072fb56cf54 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:33.252364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee889ee4-f8b4-4c4b-b7c0-6072fb56cf54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:33.252760 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee889ee4-f8b4-4c4b-b7c0-6072fb56cf54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:33.254228 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee889ee4-f8b4-4c4b-b7c0-6072fb56cf54 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:33.267896 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 412/833] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:33.273932 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee889ee4-f8b4-4c4b-b7c0-6072fb56cf54 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:33.331919 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:33.344603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed081d5b-536e-44c0-a910-e45887691bf7 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:33.382924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3634f-233e-40ee-be64-d46c47351f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:33.383735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3634f-233e-40ee-be64-d46c47351f37 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:33.384021 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3634f-233e-40ee-be64-d46c47351f37 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:33.384300 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3634f-233e-40ee-be64-d46c47351f37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:33.396412 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3634f-233e-40ee-be64-d46c47351f37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:33.401497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee889ee4-f8b4-4c4b-b7c0-6072fb56cf54 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-b1b3634f-233e-40ee-be64-d46c47351f37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:33.404411 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:33.405440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:33.406368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 422/834] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:33.412026 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3634f-233e-40ee-be64-d46c47351f37 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:33.442105 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d90e0a-7586-46f0-b44a-1086b9419424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:33.443373 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d90e0a-7586-46f0-b44a-1086b9419424 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:33.443997 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d90e0a-7586-46f0-b44a-1086b9419424 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:33.444532 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d90e0a-7586-46f0-b44a-1086b9419424 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:33.445586 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d90e0a-7586-46f0-b44a-1086b9419424 None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=e6a982ae15a6443b9b52529df47aa837) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:33.446135 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d90e0a-7586-46f0-b44a-1086b9419424 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:33.448597 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d90e0a-7586-46f0-b44a-1086b9419424 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:33.464578 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 413/835] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:33 2022] GET /identity/v3/role_assignments?user.id=e6a982ae15a6443b9b52529df47aa837&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:33.520396 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:33.538811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3634f-233e-40ee-be64-d46c47351f37 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:33.579267 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d6e1f-da39-4c8d-8a5f-9e06656b5648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:33.580510 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d6e1f-da39-4c8d-8a5f-9e06656b5648 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:33.580801 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d6e1f-da39-4c8d-8a5f-9e06656b5648 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:33.581083 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d6e1f-da39-4c8d-8a5f-9e06656b5648 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:33.581898 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d6e1f-da39-4c8d-8a5f-9e06656b5648 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:33.582230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d6e1f-da39-4c8d-8a5f-9e06656b5648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:33.583646 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d6e1f-da39-4c8d-8a5f-9e06656b5648 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:33.593895 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 423/836] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:29:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:33.600063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d90e0a-7586-46f0-b44a-1086b9419424 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:33.720264 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d90e0a-7586-46f0-b44a-1086b9419424 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:33.730433 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:33.757154 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddbd34-4808-4bb7-9b8b-4a669f3ffd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:33.757963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddbd34-4808-4bb7-9b8b-4a669f3ffd23 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:33.758254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddbd34-4808-4bb7-9b8b-4a669f3ffd23 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:33.758531 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddbd34-4808-4bb7-9b8b-4a669f3ffd23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:33.769881 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddbd34-4808-4bb7-9b8b-4a669f3ffd23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:33.771470 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddbd34-4808-4bb7-9b8b-4a669f3ffd23 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c4ddbd34-4808-4bb7-9b8b-4a669f3ffd23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:33.789372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:33.790411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:33.791956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:33.791956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:33.791956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:33.791956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:33.791956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 414/837] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:33.799050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d6e1f-da39-4c8d-8a5f-9e06656b5648 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:33.906114 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d6e1f-da39-4c8d-8a5f-9e06656b5648 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:33.948250 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796bfc9-f0d4-43df-9498-bc42e5d510c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:33.948992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796bfc9-f0d4-43df-9498-bc42e5d510c6 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:33.949280 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796bfc9-f0d4-43df-9498-bc42e5d510c6 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:33.949561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f796bfc9-f0d4-43df-9498-bc42e5d510c6 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:33.950324 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796bfc9-f0d4-43df-9498-bc42e5d510c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:33.950639 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796bfc9-f0d4-43df-9498-bc42e5d510c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:33.952176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f796bfc9-f0d4-43df-9498-bc42e5d510c6 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:33.966549 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 424/838] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:33.972376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddbd34-4808-4bb7-9b8b-4a669f3ffd23 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:34.085091 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddbd34-4808-4bb7-9b8b-4a669f3ffd23 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:34.122053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6771f-5f33-4eea-b7ff-b9807ed3e6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:34.122906 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6771f-5f33-4eea-b7ff-b9807ed3e6ec None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:34.123187 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6771f-5f33-4eea-b7ff-b9807ed3e6ec None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:34.123460 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6771f-5f33-4eea-b7ff-b9807ed3e6ec None admin] PATH_INFO: `/v3/users/swift` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:34.131788 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6771f-5f33-4eea-b7ff-b9807ed3e6ec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:34.133273 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6771f-5f33-4eea-b7ff-b9807ed3e6ec None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-0dc6771f-5f33-4eea-b7ff-b9807ed3e6ec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:34.147976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:34.149072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:34.150362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:34.150362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:34.150362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:34.150362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:34.150362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:34.150362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:34.150362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:34.150362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:34.150362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:29:34.150362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:34.150362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 415/839] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:29:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:34.157846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796bfc9-f0d4-43df-9498-bc42e5d510c6 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:34.167769 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:34.268045 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f796bfc9-f0d4-43df-9498-bc42e5d510c6 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:34.307372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ba673-f6f8-4de1-8e9f-4bbe61421438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:34.308059 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ba673-f6f8-4de1-8e9f-4bbe61421438 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:34.308366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ba673-f6f8-4de1-8e9f-4bbe61421438 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:34.308642 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ba673-f6f8-4de1-8e9f-4bbe61421438 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:34.309493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ba673-f6f8-4de1-8e9f-4bbe61421438 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:34.309866 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ba673-f6f8-4de1-8e9f-4bbe61421438 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:34.311353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ba673-f6f8-4de1-8e9f-4bbe61421438 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:34.330942 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 425/840] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:29:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:34.336940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6771f-5f33-4eea-b7ff-b9807ed3e6ec None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:34.362857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:34.445160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6771f-5f33-4eea-b7ff-b9807ed3e6ec None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:34.487121 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b0735-966f-453c-af36-4e0f88db6b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:34.488193 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b0735-966f-453c-af36-4e0f88db6b1f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:34.488717 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b0735-966f-453c-af36-4e0f88db6b1f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:34.489199 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392b0735-966f-453c-af36-4e0f88db6b1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:34.563841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b0735-966f-453c-af36-4e0f88db6b1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:34.565349 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392b0735-966f-453c-af36-4e0f88db6b1f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:34.586987 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:34.669766 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 416/841] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:29:34.675510 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ba673-f6f8-4de1-8e9f-4bbe61421438 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:34.751794 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:34.798217 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ba673-f6f8-4de1-8e9f-4bbe61421438 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:34.834860 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a329d2f-b7e9-4943-bba5-60ef3db63a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:34.836046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a329d2f-b7e9-4943-bba5-60ef3db63a79 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:34.836575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a329d2f-b7e9-4943-bba5-60ef3db63a79 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:34.837057 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a329d2f-b7e9-4943-bba5-60ef3db63a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:34.848984 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a329d2f-b7e9-4943-bba5-60ef3db63a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:34.850738 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a329d2f-b7e9-4943-bba5-60ef3db63a79 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-9a329d2f-b7e9-4943-bba5-60ef3db63a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:34.869548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:34.870550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:34.871551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:34.871551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:34.871551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:34.871551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:34.879295 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 426/842] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:34.880983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b0735-966f-453c-af36-4e0f88db6b1f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:34.997545 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392b0735-966f-453c-af36-4e0f88db6b1f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:35.019220 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:35.037900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1534a-917e-4764-a6e6-e45587ef5871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:35.038867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1534a-917e-4764-a6e6-e45587ef5871 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.039312 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1534a-917e-4764-a6e6-e45587ef5871 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.039750 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1534a-917e-4764-a6e6-e45587ef5871 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.040712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1534a-917e-4764-a6e6-e45587ef5871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:35.041203 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1534a-917e-4764-a6e6-e45587ef5871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:35.043048 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1534a-917e-4764-a6e6-e45587ef5871 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:35.057646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 417/843] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:35.066406 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a329d2f-b7e9-4943-bba5-60ef3db63a79 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:35.137845 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918f8ac-7488-48c1-91c2-56e00ad5a94c None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.138041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918f8ac-7488-48c1-91c2-56e00ad5a94c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.138319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918f8ac-7488-48c1-91c2-56e00ad5a94c None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.140138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 418/844] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:29:35.145906 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e538e50-b80b-44aa-9a3f-045ed2e0f4c0 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.146191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e538e50-b80b-44aa-9a3f-045ed2e0f4c0 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.146470 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e538e50-b80b-44aa-9a3f-045ed2e0f4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.166988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-0e538e50-b80b-44aa-9a3f-045ed2e0f4c0 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:35.168677 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-0e538e50-b80b-44aa-9a3f-045ed2e0f4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:35.180512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a329d2f-b7e9-4943-bba5-60ef3db63a79 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:35.199561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:35.220361 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff1e76-1b82-4dff-8d96-73c93dda85bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:35.221168 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff1e76-1b82-4dff-8d96-73c93dda85bc None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.221453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff1e76-1b82-4dff-8d96-73c93dda85bc None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.221787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff1e76-1b82-4dff-8d96-73c93dda85bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.229650 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff1e76-1b82-4dff-8d96-73c93dda85bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:35.231207 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff1e76-1b82-4dff-8d96-73c93dda85bc None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-b1ff1e76-1b82-4dff-8d96-73c93dda85bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:35.247721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:35.248808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:35.249807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:35.249807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:35.249807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:35.249807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:35.249807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:35.249807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:35.249807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:35.249807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:35.256380 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 419/846] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:29:35.256545 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 427/846] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:35.256655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e538e50-b80b-44aa-9a3f-045ed2e0f4c0 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:35.274395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff1e76-1b82-4dff-8d96-73c93dda85bc None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:35.366189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e538e50-b80b-44aa-9a3f-045ed2e0f4c0 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:35.378323 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:35.406230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff1e76-1b82-4dff-8d96-73c93dda85bc None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:35.406423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf7a20-c258-47d3-952f-78d5e88a3607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:35.406423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf7a20-c258-47d3-952f-78d5e88a3607 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.406423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf7a20-c258-47d3-952f-78d5e88a3607 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.406423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf7a20-c258-47d3-952f-78d5e88a3607 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.406766 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf7a20-c258-47d3-952f-78d5e88a3607 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:35.406766 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf7a20-c258-47d3-952f-78d5e88a3607 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:35.406766 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf7a20-c258-47d3-952f-78d5e88a3607 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:35.420387 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 420/847] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:35.426768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf7a20-c258-47d3-952f-78d5e88a3607 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:35.441977 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb944bb4-f3fa-4bb1-814c-0874f90cf801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:35.442811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb944bb4-f3fa-4bb1-814c-0874f90cf801 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.443354 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb944bb4-f3fa-4bb1-814c-0874f90cf801 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.443477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb944bb4-f3fa-4bb1-814c-0874f90cf801 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.444102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb944bb4-f3fa-4bb1-814c-0874f90cf801 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:35.445501 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb944bb4-f3fa-4bb1-814c-0874f90cf801 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-cb944bb4-f3fa-4bb1-814c-0874f90cf801 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:35.462020 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:35.463144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:29:35.464081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:35.468469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 428/848] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:29:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:35.470071 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb944bb4-f3fa-4bb1-814c-0874f90cf801 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:35.516870 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:35.546163 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf7a20-c258-47d3-952f-78d5e88a3607 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:35.577684 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb944bb4-f3fa-4bb1-814c-0874f90cf801 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:35.583270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf10269-b44a-4b97-ac67-3bf901220b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:35.583901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf10269-b44a-4b97-ac67-3bf901220b6b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.584182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf10269-b44a-4b97-ac67-3bf901220b6b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.584487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf10269-b44a-4b97-ac67-3bf901220b6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.585318 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf10269-b44a-4b97-ac67-3bf901220b6b None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=14034b78ec5e4999848a3fffb2964587) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:35.585649 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf10269-b44a-4b97-ac67-3bf901220b6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:35.587135 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf10269-b44a-4b97-ac67-3bf901220b6b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:35.593627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 421/849] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:35 2022] GET /identity/v3/role_assignments?user.id=14034b78ec5e4999848a3fffb2964587&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:35.595431 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318cc0-879a-48a6-9dd8-ab0cc86e362d None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.595716 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318cc0-879a-48a6-9dd8-ab0cc86e362d None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.595991 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03318cc0-879a-48a6-9dd8-ab0cc86e362d None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.601546 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 422/850] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:35.603583 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42012c-ff98-4d6f-874b-d991da1839a8 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.603874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42012c-ff98-4d6f-874b-d991da1839a8 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.604148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42012c-ff98-4d6f-874b-d991da1839a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.622983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-9a42012c-ff98-4d6f-874b-d991da1839a8 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:35.623943 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8dea84-54a3-4365-8e38-8d90c8617cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:35.624211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-9a42012c-ff98-4d6f-874b-d991da1839a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:35.625111 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8dea84-54a3-4365-8e38-8d90c8617cba None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.625607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8dea84-54a3-4365-8e38-8d90c8617cba None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.626148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8dea84-54a3-4365-8e38-8d90c8617cba None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.627084 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8dea84-54a3-4365-8e38-8d90c8617cba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:35.627640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8dea84-54a3-4365-8e38-8d90c8617cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:35.629302 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8dea84-54a3-4365-8e38-8d90c8617cba None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:35.639190 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 429/851] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:29:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:35.642112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8dea84-54a3-4365-8e38-8d90c8617cba None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:35.659528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:35.712375 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 423/852] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:35.732569 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42012c-ff98-4d6f-874b-d991da1839a8 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:35.785846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8dea84-54a3-4365-8e38-8d90c8617cba None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:35.791829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:35.840592 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9214fdc-b268-4406-9323-f5b6d02d0925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:35.841549 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9214fdc-b268-4406-9323-f5b6d02d0925 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.841833 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9214fdc-b268-4406-9323-f5b6d02d0925 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.842108 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9214fdc-b268-4406-9323-f5b6d02d0925 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.842912 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9214fdc-b268-4406-9323-f5b6d02d0925 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:35.843225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9214fdc-b268-4406-9323-f5b6d02d0925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:35.845933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9214fdc-b268-4406-9323-f5b6d02d0925 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:35.851238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 430/853] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:29:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:35.887496 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42012c-ff98-4d6f-874b-d991da1839a8 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:35.929154 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c65abd-dcd7-43b2-a638-83c239e8b7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:35.930335 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c65abd-dcd7-43b2-a638-83c239e8b7a6 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.930823 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c65abd-dcd7-43b2-a638-83c239e8b7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.931301 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c65abd-dcd7-43b2-a638-83c239e8b7a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.944304 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c65abd-dcd7-43b2-a638-83c239e8b7a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:35.947662 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3d77d-1fff-4c18-b6e0-8e75f6607207 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.947955 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3d77d-1fff-4c18-b6e0-8e75f6607207 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.948235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3d77d-1fff-4c18-b6e0-8e75f6607207 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.949785 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 431/854] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-f5c65abd-dcd7-43b2-a638-83c239e8b7a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:35.952886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:35.954679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:29:35.955687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:35.956411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 424/855] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:29:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:35.962095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a410e-cd42-4fb3-bde9-a6e0dc90c3ba None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:35.962606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a410e-cd42-4fb3-bde9-a6e0dc90c3ba None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:35.962894 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a410e-cd42-4fb3-bde9-a6e0dc90c3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:35.963847 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c65abd-dcd7-43b2-a638-83c239e8b7a6 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:35.981123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-da5a410e-cd42-4fb3-bde9-a6e0dc90c3ba None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:35.982344 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-da5a410e-cd42-4fb3-bde9-a6e0dc90c3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:35.996157 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:36.071365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 432/856] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:36.092222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c65abd-dcd7-43b2-a638-83c239e8b7a6 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:36.093318 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a410e-cd42-4fb3-bde9-a6e0dc90c3ba None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:36.134273 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d4f29-8e6b-462d-bb60-cddfe016f84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:36.134962 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4f29-8e6b-462d-bb60-cddfe016f84e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:36.135245 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4f29-8e6b-462d-bb60-cddfe016f84e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:36.135523 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4f29-8e6b-462d-bb60-cddfe016f84e None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:36.136279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d4f29-8e6b-462d-bb60-cddfe016f84e None admin] RBAC: Adding query filter params (name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:36.136620 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d4f29-8e6b-462d-bb60-cddfe016f84e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:36.138017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d4f29-8e6b-462d-bb60-cddfe016f84e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:36.150639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 425/857] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:29:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:36.157966 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d4f29-8e6b-462d-bb60-cddfe016f84e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:36.229261 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:36.233824 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a410e-cd42-4fb3-bde9-a6e0dc90c3ba None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:36.260760 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:36.271254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d4f29-8e6b-462d-bb60-cddfe016f84e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:36.277951 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d950e-bf3d-42ca-8b6a-b07fb4a9bd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:36.278916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d950e-bf3d-42ca-8b6a-b07fb4a9bd56 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:36.279437 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d950e-bf3d-42ca-8b6a-b07fb4a9bd56 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:36.279891 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d950e-bf3d-42ca-8b6a-b07fb4a9bd56 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:36.280851 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d950e-bf3d-42ca-8b6a-b07fb4a9bd56 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:36.283260 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736d950e-bf3d-42ca-8b6a-b07fb4a9bd56 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-736d950e-bf3d-42ca-8b6a-b07fb4a9bd56 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:36.297529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:36.298856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:29:36.299987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:36.302595 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 433/858] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 20:29:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:36.308719 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d950e-bf3d-42ca-8b6a-b07fb4a9bd56 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:36.320194 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0972f-e32a-4105-9c25-783eb8f2c758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:36.321353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0972f-e32a-4105-9c25-783eb8f2c758 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:36.321794 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0972f-e32a-4105-9c25-783eb8f2c758 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:36.322255 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0972f-e32a-4105-9c25-783eb8f2c758 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:36.333958 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0972f-e32a-4105-9c25-783eb8f2c758 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:36.335627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0972f-e32a-4105-9c25-783eb8f2c758 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-5cd0972f-e32a-4105-9c25-783eb8f2c758 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:36.356084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:36.357376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:36.358707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:36.358707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:36.358707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:36.358707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:36.360755 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 426/859] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:36.365938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0972f-e32a-4105-9c25-783eb8f2c758 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:36.446130 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d950e-bf3d-42ca-8b6a-b07fb4a9bd56 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:36.488200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb22cea-eb9c-4534-952e-259239b61c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:36.489384 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb22cea-eb9c-4534-952e-259239b61c02 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:36.489880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb22cea-eb9c-4534-952e-259239b61c02 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:36.490362 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb22cea-eb9c-4534-952e-259239b61c02 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:36.491342 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb22cea-eb9c-4534-952e-259239b61c02 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:36.491862 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb22cea-eb9c-4534-952e-259239b61c02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:36.493525 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb22cea-eb9c-4534-952e-259239b61c02 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:36.499239 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 434/860] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:29:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:36.504835 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb22cea-eb9c-4534-952e-259239b61c02 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:36.520505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0972f-e32a-4105-9c25-783eb8f2c758 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:36.537165 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:36.568867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a79667-844e-4311-a591-eeeeebdbfa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:36.569579 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79667-844e-4311-a591-eeeeebdbfa39 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:36.569863 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79667-844e-4311-a591-eeeeebdbfa39 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:36.570141 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a79667-844e-4311-a591-eeeeebdbfa39 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:36.571128 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a79667-844e-4311-a591-eeeeebdbfa39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:36.571438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a79667-844e-4311-a591-eeeeebdbfa39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:36.573588 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a79667-844e-4311-a591-eeeeebdbfa39 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:36.585876 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:36.599169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 427/861] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:36.600402 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a79667-844e-4311-a591-eeeeebdbfa39 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:36.624871 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb22cea-eb9c-4534-952e-259239b61c02 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:36.663786 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e50fb-6fbc-404f-ae8e-1c73521ad1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:36.664677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e50fb-6fbc-404f-ae8e-1c73521ad1ed None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:36.665182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e50fb-6fbc-404f-ae8e-1c73521ad1ed None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:36.665426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e50fb-6fbc-404f-ae8e-1c73521ad1ed None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:36.666503 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e50fb-6fbc-404f-ae8e-1c73521ad1ed None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:36.666915 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e50fb-6fbc-404f-ae8e-1c73521ad1ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:36.668516 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e50fb-6fbc-404f-ae8e-1c73521ad1ed None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:36.680561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 435/862] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:29:36 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:36.716353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a79667-844e-4311-a591-eeeeebdbfa39 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:36.760665 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041713e-24fb-4f79-be8e-26b61c6a3eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:36.761967 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041713e-24fb-4f79-be8e-26b61c6a3eb4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:36.762406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041713e-24fb-4f79-be8e-26b61c6a3eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:36.762844 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041713e-24fb-4f79-be8e-26b61c6a3eb4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:36.772355 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041713e-24fb-4f79-be8e-26b61c6a3eb4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:36.774243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041713e-24fb-4f79-be8e-26b61c6a3eb4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c041713e-24fb-4f79-be8e-26b61c6a3eb4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:36.790725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:36.792290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:36.793551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:36.793551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:36.793551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:36.793551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:36.793551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:36.793551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:36.793551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:36.793551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:36.793551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:36.793551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:36.796042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 428/863] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:29:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:36.802757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e50fb-6fbc-404f-ae8e-1c73521ad1ed None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:36.911231 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:36.922050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e50fb-6fbc-404f-ae8e-1c73521ad1ed None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:36.960919 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc738a0-2a4b-4b6c-a237-b00156f321fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:36.962006 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc738a0-2a4b-4b6c-a237-b00156f321fe None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:36.962523 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc738a0-2a4b-4b6c-a237-b00156f321fe None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:36.962964 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc738a0-2a4b-4b6c-a237-b00156f321fe None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:36.963978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc738a0-2a4b-4b6c-a237-b00156f321fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:36.964490 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc738a0-2a4b-4b6c-a237-b00156f321fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:36.966152 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc738a0-2a4b-4b6c-a237-b00156f321fe None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:36.987227 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 436/864] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:29:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:36.993157 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041713e-24fb-4f79-be8e-26b61c6a3eb4 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:37.062244 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:37.110894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041713e-24fb-4f79-be8e-26b61c6a3eb4 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:37.147061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6b984-c15a-44a4-b851-3a249d74c0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:37.148025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6b984-c15a-44a4-b851-3a249d74c0f4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:37.148536 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6b984-c15a-44a4-b851-3a249d74c0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:37.148991 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6b984-c15a-44a4-b851-3a249d74c0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:37.223020 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6b984-c15a-44a4-b851-3a249d74c0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:37.224858 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e6b984-c15a-44a4-b851-3a249d74c0f4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:37.285909 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:37.330825 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 429/865] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:29:37.336486 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc738a0-2a4b-4b6c-a237-b00156f321fe None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:37.446403 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc738a0-2a4b-4b6c-a237-b00156f321fe None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:37.485666 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa4785-9c0d-4b5b-b6be-fe36c2476b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:37.486478 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa4785-9c0d-4b5b-b6be-fe36c2476b20 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:37.486766 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa4785-9c0d-4b5b-b6be-fe36c2476b20 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:37.487044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa4785-9c0d-4b5b-b6be-fe36c2476b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:37.498321 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa4785-9c0d-4b5b-b6be-fe36c2476b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:37.499867 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaa4785-9c0d-4b5b-b6be-fe36c2476b20 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:37.509213 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-abaa4785-9c0d-4b5b-b6be-fe36c2476b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:37.519767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:37.520819 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:37.522228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:37.522228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:37.522228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:37.522228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:37.522228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 437/866] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:37.526976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6b984-c15a-44a4-b851-3a249d74c0f4 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:37.633214 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6b984-c15a-44a4-b851-3a249d74c0f4 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:37.671457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d70ce-173f-4b53-a6a3-a1ba9639ac6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:37.672134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d70ce-173f-4b53-a6a3-a1ba9639ac6a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:37.672459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d70ce-173f-4b53-a6a3-a1ba9639ac6a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:37.672907 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d70ce-173f-4b53-a6a3-a1ba9639ac6a None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:37.673635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d70ce-173f-4b53-a6a3-a1ba9639ac6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:37.673948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d70ce-173f-4b53-a6a3-a1ba9639ac6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:37.675328 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3d70ce-173f-4b53-a6a3-a1ba9639ac6a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:37.687684 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 430/867] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:37.693226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa4785-9c0d-4b5b-b6be-fe36c2476b20 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:37.806669 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa4785-9c0d-4b5b-b6be-fe36c2476b20 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:37.847108 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79ad0c-2485-4829-b157-6a1d05f0cc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:37.848368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79ad0c-2485-4829-b157-6a1d05f0cc38 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:37.848892 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79ad0c-2485-4829-b157-6a1d05f0cc38 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:37.849382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79ad0c-2485-4829-b157-6a1d05f0cc38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:37.858151 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79ad0c-2485-4829-b157-6a1d05f0cc38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:37.860095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79ad0c-2485-4829-b157-6a1d05f0cc38 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-3b79ad0c-2485-4829-b157-6a1d05f0cc38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:37.876747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:37.877814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:37.878828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:37.878828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:37.878828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:37.878828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:37.878828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:37.878828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:37.878828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:37.878828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:37.881657 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 438/868] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:37.887930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d70ce-173f-4b53-a6a3-a1ba9639ac6a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:37.917258 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:37.999536 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3d70ce-173f-4b53-a6a3-a1ba9639ac6a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:38.039923 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e536ee-e09a-46c3-b4f3-446181b35b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:38.040627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e536ee-e09a-46c3-b4f3-446181b35b8e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:38.040912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e536ee-e09a-46c3-b4f3-446181b35b8e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:38.041440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e536ee-e09a-46c3-b4f3-446181b35b8e None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:38.042238 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e536ee-e09a-46c3-b4f3-446181b35b8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:38.042548 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e536ee-e09a-46c3-b4f3-446181b35b8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:38.043990 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e536ee-e09a-46c3-b4f3-446181b35b8e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:38.050380 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af0fc5-ba0e-4371-b758-442855ecafbd None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:38.050679 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af0fc5-ba0e-4371-b758-442855ecafbd None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:38.050958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af0fc5-ba0e-4371-b758-442855ecafbd None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:38.057354 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 439/869] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:38.058026 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 431/870] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:38.064280 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1cca8-c503-4bae-8164-2530934817e1 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:38.064706 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1cca8-c503-4bae-8164-2530934817e1 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:38.064963 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1cca8-c503-4bae-8164-2530934817e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:38.069316 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e536ee-e09a-46c3-b4f3-446181b35b8e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:38.091483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-a4d1cca8-c503-4bae-8164-2530934817e1 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:38.093573 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-a4d1cca8-c503-4bae-8164-2530934817e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:38.154778 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:38.166083 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:38.180389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 440/871] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:38.184728 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e536ee-e09a-46c3-b4f3-446181b35b8e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:38.190360 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3d7b-3e35-46bf-995a-2f673cacad6f None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:38.190654 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3d7b-3e35-46bf-995a-2f673cacad6f None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:38.190935 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef3d7b-3e35-46bf-995a-2f673cacad6f None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:38.192609 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 441/872] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:38.198115 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f61c64-5be5-483b-80ce-403ff9a80833 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:38.198417 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f61c64-5be5-483b-80ce-403ff9a80833 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:38.198699 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f61c64-5be5-483b-80ce-403ff9a80833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:38.217071 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-b0f61c64-5be5-483b-80ce-403ff9a80833 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:38.218336 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-b0f61c64-5be5-483b-80ce-403ff9a80833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:38.227681 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957522e-50af-4cb3-a0f7-aa65617c2d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:38.228373 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957522e-50af-4cb3-a0f7-aa65617c2d03 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:38.228663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957522e-50af-4cb3-a0f7-aa65617c2d03 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:38.228948 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957522e-50af-4cb3-a0f7-aa65617c2d03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:38.229789 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957522e-50af-4cb3-a0f7-aa65617c2d03 None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=5f5896afd9434adebe8c1d8db70b28bf) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:38.230115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957522e-50af-4cb3-a0f7-aa65617c2d03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:38.231559 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957522e-50af-4cb3-a0f7-aa65617c2d03 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:38.244370 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 432/873] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:38 2022] GET /identity/v3/role_assignments?user.id=5f5896afd9434adebe8c1d8db70b28bf&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:38.244370 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957522e-50af-4cb3-a0f7-aa65617c2d03 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:38.308530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 442/874] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:38.322456 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f61c64-5be5-483b-80ce-403ff9a80833 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:38.392170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957522e-50af-4cb3-a0f7-aa65617c2d03 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:38.444406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebea896-dc20-46b8-9b45-f659ec628499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:38.445207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea896-dc20-46b8-9b45-f659ec628499 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:38.445494 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea896-dc20-46b8-9b45-f659ec628499 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:38.445775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebea896-dc20-46b8-9b45-f659ec628499 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:38.446447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebea896-dc20-46b8-9b45-f659ec628499 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:38.448541 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebea896-dc20-46b8-9b45-f659ec628499 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-4ebea896-dc20-46b8-9b45-f659ec628499 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:38.468371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:38.469589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:38.470539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:38.471359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f61c64-5be5-483b-80ce-403ff9a80833 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:38.471447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 433/875] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:29:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:38.477953 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebea896-dc20-46b8-9b45-f659ec628499 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:38.490822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:38.497833 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:38.516228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0cfc2f-96bb-4fb7-9c5b-b1f5c0576778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:38.517019 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cfc2f-96bb-4fb7-9c5b-b1f5c0576778 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:38.517311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cfc2f-96bb-4fb7-9c5b-b1f5c0576778 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:38.517595 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0cfc2f-96bb-4fb7-9c5b-b1f5c0576778 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:38.518754 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0cfc2f-96bb-4fb7-9c5b-b1f5c0576778 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:38.520154 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0cfc2f-96bb-4fb7-9c5b-b1f5c0576778 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:38.594414 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebea896-dc20-46b8-9b45-f659ec628499 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:38.632069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcdddf-da0a-426c-8584-44afc1280058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:38.632848 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcdddf-da0a-426c-8584-44afc1280058 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:38.633141 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcdddf-da0a-426c-8584-44afc1280058 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:38.633409 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bcdddf-da0a-426c-8584-44afc1280058 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:38.634133 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcdddf-da0a-426c-8584-44afc1280058 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:38.634436 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcdddf-da0a-426c-8584-44afc1280058 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:38.635815 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bcdddf-da0a-426c-8584-44afc1280058 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:38.640803 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 434/876] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:29:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:38.646457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcdddf-da0a-426c-8584-44afc1280058 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:38.652269 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 443/877] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:29:38 2022] POST /identity/v3/services => generated 251 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:29:38.740406 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:38.776886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bcdddf-da0a-426c-8584-44afc1280058 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:38.824575 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300aada-ed7e-4724-928b-3de6c347f8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:38.825259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300aada-ed7e-4724-928b-3de6c347f8e1 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:38.825546 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300aada-ed7e-4724-928b-3de6c347f8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:38.825831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300aada-ed7e-4724-928b-3de6c347f8e1 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:38.826546 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300aada-ed7e-4724-928b-3de6c347f8e1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:38.826857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300aada-ed7e-4724-928b-3de6c347f8e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:38.828264 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300aada-ed7e-4724-928b-3de6c347f8e1 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:38.834132 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 435/878] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:29:38 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:39.081968 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd72f40-6636-4fc1-bcaf-ed5c7a873db7 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:39.082526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd72f40-6636-4fc1-bcaf-ed5c7a873db7 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:39.082526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd72f40-6636-4fc1-bcaf-ed5c7a873db7 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:39.084309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 444/879] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:39.090030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a733f-5f83-48ab-aef0-4018899425b0 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:39.090500 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a733f-5f83-48ab-aef0-4018899425b0 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:39.090583 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a733f-5f83-48ab-aef0-4018899425b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:39.109243 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-112a733f-5f83-48ab-aef0-4018899425b0 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:39.110465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-112a733f-5f83-48ab-aef0-4018899425b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:39.184505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 436/880] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:39 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:39.203418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd72f40-6636-4fc1-bcaf-ed5c7a873db7 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:39.319451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd72f40-6636-4fc1-bcaf-ed5c7a873db7 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:39.356298 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26303141-acfa-437f-85b2-58c68488d636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:39.356930 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26303141-acfa-437f-85b2-58c68488d636 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:39.357208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26303141-acfa-437f-85b2-58c68488d636 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:39.357479 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26303141-acfa-437f-85b2-58c68488d636 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:39.358223 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26303141-acfa-437f-85b2-58c68488d636 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:39.359645 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26303141-acfa-437f-85b2-58c68488d636 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:39.480791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 445/881] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:29:39 2022] POST /identity/v3/services => generated 269 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:40.664828 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b7b34-ef3c-4112-85d1-cd9e5e003bd3 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:40.665396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b7b34-ef3c-4112-85d1-cd9e5e003bd3 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:40.665396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b7b34-ef3c-4112-85d1-cd9e5e003bd3 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:40.667293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 437/882] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:40.673123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e689382-663a-4ef8-999c-bab996f24a7b None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:40.673572 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e689382-663a-4ef8-999c-bab996f24a7b None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:40.673729 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e689382-663a-4ef8-999c-bab996f24a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:40.691461 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-3e689382-663a-4ef8-999c-bab996f24a7b None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:40.692702 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-3e689382-663a-4ef8-999c-bab996f24a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:40.736009 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:40.776735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 446/883] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:29:40.801726 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b7b34-ef3c-4112-85d1-cd9e5e003bd3 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:40.869042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:40.918151 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b7b34-ef3c-4112-85d1-cd9e5e003bd3 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:40.954925 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb6f6d-8586-4a60-8d32-2c4e96994b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:40.955753 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb6f6d-8586-4a60-8d32-2c4e96994b7a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:40.956037 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb6f6d-8586-4a60-8d32-2c4e96994b7a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:40.956314 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb6f6d-8586-4a60-8d32-2c4e96994b7a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:40.967476 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb6f6d-8586-4a60-8d32-2c4e96994b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-babb6f6d-8586-4a60-8d32-2c4e96994b7a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:40.976071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:40.977239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:40.978598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 438/884] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:29:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:40.983494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e689382-663a-4ef8-999c-bab996f24a7b None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:41.040325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03036198-4ce3-4ca1-930a-4148f5810078 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.040685 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03036198-4ce3-4ca1-930a-4148f5810078 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.040915 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03036198-4ce3-4ca1-930a-4148f5810078 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.044622 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 439/885] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:41.048563 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580306-ca06-4748-9d75-3cff9234329f None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.048853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580306-ca06-4748-9d75-3cff9234329f None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.049126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d580306-ca06-4748-9d75-3cff9234329f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.073045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-0d580306-ca06-4748-9d75-3cff9234329f None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:41.074715 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-0d580306-ca06-4748-9d75-3cff9234329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:41.099518 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e689382-663a-4ef8-999c-bab996f24a7b None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:41.137265 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c43afe-bee3-45bc-92f1-57d8ea5f887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:41.138328 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43afe-bee3-45bc-92f1-57d8ea5f887c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.138837 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43afe-bee3-45bc-92f1-57d8ea5f887c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.139333 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c43afe-bee3-45bc-92f1-57d8ea5f887c None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.140313 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c43afe-bee3-45bc-92f1-57d8ea5f887c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:41.140898 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c43afe-bee3-45bc-92f1-57d8ea5f887c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:41.142872 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c43afe-bee3-45bc-92f1-57d8ea5f887c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:41.152672 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 447/886] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:29:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:41.158445 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c43afe-bee3-45bc-92f1-57d8ea5f887c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:41.176277 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 440/887] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:29:41.178300 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca7c1a-5932-4d7a-acf8-e613b102f83a None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.178595 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca7c1a-5932-4d7a-acf8-e613b102f83a None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.178876 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca7c1a-5932-4d7a-acf8-e613b102f83a None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.184164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 441/888] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:29:41.185417 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa659cdd-959a-4bf7-b59a-359bd83b41f8 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.185713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa659cdd-959a-4bf7-b59a-359bd83b41f8 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.185999 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa659cdd-959a-4bf7-b59a-359bd83b41f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.195083 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:41.203735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-aa659cdd-959a-4bf7-b59a-359bd83b41f8 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:41.213386 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-aa659cdd-959a-4bf7-b59a-359bd83b41f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:41.260895 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:41.273379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c43afe-bee3-45bc-92f1-57d8ea5f887c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:41.292477 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 442/889] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:41.292477 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa659cdd-959a-4bf7-b59a-359bd83b41f8 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:41.317525 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df46d1-fc44-41bb-9055-30acf814c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:41.318301 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df46d1-fc44-41bb-9055-30acf814c792 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.318586 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df46d1-fc44-41bb-9055-30acf814c792 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.318938 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df46d1-fc44-41bb-9055-30acf814c792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.330406 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df46d1-fc44-41bb-9055-30acf814c792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:41.331841 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df46d1-fc44-41bb-9055-30acf814c792 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-72df46d1-fc44-41bb-9055-30acf814c792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:41.350254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:41.351473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:41.352536 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:41.352536 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:41.352536 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:41.352536 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:41.352536 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 448/890] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:41.352536 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df46d1-fc44-41bb-9055-30acf814c792 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:41.407521 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa659cdd-959a-4bf7-b59a-359bd83b41f8 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:41.440483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:41.446357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52583497-9d3a-428b-a00a-af4155ba2ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:41.447351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52583497-9d3a-428b-a00a-af4155ba2ccc None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.447639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52583497-9d3a-428b-a00a-af4155ba2ccc None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.448010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52583497-9d3a-428b-a00a-af4155ba2ccc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.449171 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52583497-9d3a-428b-a00a-af4155ba2ccc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:41.451639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52583497-9d3a-428b-a00a-af4155ba2ccc None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-52583497-9d3a-428b-a00a-af4155ba2ccc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:41.470985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:41.472320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:41.473351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 443/891] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:29:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:41.474235 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52583497-9d3a-428b-a00a-af4155ba2ccc None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:41.487294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df46d1-fc44-41bb-9055-30acf814c792 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:41.534688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac1b64-d4f8-4a5f-a9e3-16b6760614f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:41.535343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:41.535460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac1b64-d4f8-4a5f-a9e3-16b6760614f1 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.535628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac1b64-d4f8-4a5f-a9e3-16b6760614f1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.536291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac1b64-d4f8-4a5f-a9e3-16b6760614f1 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.537007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac1b64-d4f8-4a5f-a9e3-16b6760614f1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:41.538394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac1b64-d4f8-4a5f-a9e3-16b6760614f1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:41.600546 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52583497-9d3a-428b-a00a-af4155ba2ccc None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:41.640118 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116ebaf-c3f2-4614-b299-f059d113f006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:41.640823 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116ebaf-c3f2-4614-b299-f059d113f006 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.641104 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116ebaf-c3f2-4614-b299-f059d113f006 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.642013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116ebaf-c3f2-4614-b299-f059d113f006 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.642739 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116ebaf-c3f2-4614-b299-f059d113f006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:41.643041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116ebaf-c3f2-4614-b299-f059d113f006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:41.644434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116ebaf-c3f2-4614-b299-f059d113f006 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:41.657056 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 444/892] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:41.657850 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116ebaf-c3f2-4614-b299-f059d113f006 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:41.670458 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 449/893] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:29:41 2022] POST /identity/v3/services => generated 241 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:41.671280 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac1b64-d4f8-4a5f-a9e3-16b6760614f1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:41.782533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116ebaf-c3f2-4614-b299-f059d113f006 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:41.787084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:41.827111 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac1b64-d4f8-4a5f-a9e3-16b6760614f1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:41.830700 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d57d5-5ea1-46f0-b9d1-f5405ebaf613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:41.831534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d57d5-5ea1-46f0-b9d1-f5405ebaf613 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.831896 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d57d5-5ea1-46f0-b9d1-f5405ebaf613 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.832229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d57d5-5ea1-46f0-b9d1-f5405ebaf613 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.833138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d57d5-5ea1-46f0-b9d1-f5405ebaf613 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:41.833555 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d57d5-5ea1-46f0-b9d1-f5405ebaf613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:41.835112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d57d5-5ea1-46f0-b9d1-f5405ebaf613 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:41.845140 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 445/894] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:29:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:41.846537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d57d5-5ea1-46f0-b9d1-f5405ebaf613 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:41.875195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6985d7c-46ff-4369-93a0-b12e9d0c0463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:41.876700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6985d7c-46ff-4369-93a0-b12e9d0c0463 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.877031 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6985d7c-46ff-4369-93a0-b12e9d0c0463 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.877320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6985d7c-46ff-4369-93a0-b12e9d0c0463 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.881037 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:41.889808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6985d7c-46ff-4369-93a0-b12e9d0c0463 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:41.895095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6985d7c-46ff-4369-93a0-b12e9d0c0463 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-b6985d7c-46ff-4369-93a0-b12e9d0c0463 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:41.914235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:41.915553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 450/895] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:29:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:41.917103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dd006-4c06-4cd9-92ad-fe99008d1903 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.917955 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dd006-4c06-4cd9-92ad-fe99008d1903 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.917955 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dd006-4c06-4cd9-92ad-fe99008d1903 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.919313 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 451/896] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:41.925240 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829e4c-a482-4804-ab5f-bd4d5db0fbab None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:41.926166 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829e4c-a482-4804-ab5f-bd4d5db0fbab None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:41.927366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39829e4c-a482-4804-ab5f-bd4d5db0fbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:41.945098 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-39829e4c-a482-4804-ab5f-bd4d5db0fbab None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:41.946298 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-39829e4c-a482-4804-ab5f-bd4d5db0fbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:41.971916 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d57d5-5ea1-46f0-b9d1-f5405ebaf613 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:42.008687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac29a01-4f1b-4c2b-8731-ef5c9e3a7bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:42.009365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac29a01-4f1b-4c2b-8731-ef5c9e3a7bd1 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:42.009652 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac29a01-4f1b-4c2b-8731-ef5c9e3a7bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:42.009935 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac29a01-4f1b-4c2b-8731-ef5c9e3a7bd1 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:42.010656 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac29a01-4f1b-4c2b-8731-ef5c9e3a7bd1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:42.010974 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac29a01-4f1b-4c2b-8731-ef5c9e3a7bd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:42.012372 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac29a01-4f1b-4c2b-8731-ef5c9e3a7bd1 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:42.017540 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 446/897] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:29:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:42.018327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac29a01-4f1b-4c2b-8731-ef5c9e3a7bd1 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:42.031715 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 452/898] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:29:42.032690 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39829e4c-a482-4804-ab5f-bd4d5db0fbab None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:42.065840 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:42.140556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac29a01-4f1b-4c2b-8731-ef5c9e3a7bd1 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:42.161308 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39829e4c-a482-4804-ab5f-bd4d5db0fbab None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:42.178213 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae59700-9ce5-4fcc-b385-b772de0d507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:42.178899 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59700-9ce5-4fcc-b385-b772de0d507c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:42.179190 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59700-9ce5-4fcc-b385-b772de0d507c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:42.179467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae59700-9ce5-4fcc-b385-b772de0d507c None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:42.180311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae59700-9ce5-4fcc-b385-b772de0d507c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:42.180647 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae59700-9ce5-4fcc-b385-b772de0d507c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:42.182083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae59700-9ce5-4fcc-b385-b772de0d507c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:42.193185 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:42.201636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02990646-1152-40fc-a590-f1a07fe00564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:42.201809 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 447/899] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:29:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:42.202458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae59700-9ce5-4fcc-b385-b772de0d507c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:42.205453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990646-1152-40fc-a590-f1a07fe00564 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:42.206960 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990646-1152-40fc-a590-f1a07fe00564 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:42.209555 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02990646-1152-40fc-a590-f1a07fe00564 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:42.209555 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02990646-1152-40fc-a590-f1a07fe00564 None admin] RBAC: Adding query filter params (interface=public, service_id=74ff6b8df403402f899817d6ab763419, region_id=RegionOne) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:42.209555 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02990646-1152-40fc-a590-f1a07fe00564 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:42.209822 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02990646-1152-40fc-a590-f1a07fe00564 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:42.220843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 453/900] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:29:42 2022] GET /identity/v3/endpoints?service_id=74ff6b8df403402f899817d6ab763419&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:42.220843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02990646-1152-40fc-a590-f1a07fe00564 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:42.302528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:42.319188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae59700-9ce5-4fcc-b385-b772de0d507c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:42.331725 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02990646-1152-40fc-a590-f1a07fe00564 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:42.360909 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093c940-42b8-41a9-a6e8-ec69a162daa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:42.361739 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093c940-42b8-41a9-a6e8-ec69a162daa0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:42.362025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093c940-42b8-41a9-a6e8-ec69a162daa0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:42.362300 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093c940-42b8-41a9-a6e8-ec69a162daa0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:42.362961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093c940-42b8-41a9-a6e8-ec69a162daa0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:42.364363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093c940-42b8-41a9-a6e8-ec69a162daa0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:42.368896 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b15fae-9bf5-4845-9550-677268eca9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:42.369535 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15fae-9bf5-4845-9550-677268eca9fe None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:42.369827 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15fae-9bf5-4845-9550-677268eca9fe None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:42.370109 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b15fae-9bf5-4845-9550-677268eca9fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-5093c940-42b8-41a9-a6e8-ec69a162daa0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:42.377191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:42.378345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:42.385113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 448/901] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 20:29:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:42.386137 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093c940-42b8-41a9-a6e8-ec69a162daa0 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:42.439928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:42.451500 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b15fae-9bf5-4845-9550-677268eca9fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:42.453084 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b15fae-9bf5-4845-9550-677268eca9fe None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:42.492960 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093c940-42b8-41a9-a6e8-ec69a162daa0 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:42.529631 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456d94-dcdd-4239-84bc-9e01eacf2d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:42.530435 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456d94-dcdd-4239-84bc-9e01eacf2d48 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:42.530819 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456d94-dcdd-4239-84bc-9e01eacf2d48 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:42.531001 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456d94-dcdd-4239-84bc-9e01eacf2d48 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:42.531692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd456d94-dcdd-4239-84bc-9e01eacf2d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:42.533680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd456d94-dcdd-4239-84bc-9e01eacf2d48 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:42.566243 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 454/902] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:42 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:29:42.567326 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b15fae-9bf5-4845-9550-677268eca9fe None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:42.575900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 449/903] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:29:42 2022] GET /identity/v3/services => generated 1497 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:29:42.576915 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456d94-dcdd-4239-84bc-9e01eacf2d48 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:42.604877 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:42.697380 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:42.713477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b15fae-9bf5-4845-9550-677268eca9fe None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:42.733951 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456d94-dcdd-4239-84bc-9e01eacf2d48 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:42.786553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d242d2e-139c-452e-a875-413d06666b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:42.787675 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d242d2e-139c-452e-a875-413d06666b8e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:42.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d242d2e-139c-452e-a875-413d06666b8e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:42.788667 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d242d2e-139c-452e-a875-413d06666b8e None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:42.789595 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d242d2e-139c-452e-a875-413d06666b8e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:42.789911 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d242d2e-139c-452e-a875-413d06666b8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:42.791569 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d242d2e-139c-452e-a875-413d06666b8e None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:42.797589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 455/904] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:29:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:42.801384 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f671e9-5706-4d77-93dd-a96ff1561d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:42.802998 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f671e9-5706-4d77-93dd-a96ff1561d08 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:42.803464 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f671e9-5706-4d77-93dd-a96ff1561d08 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:42.803598 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d242d2e-139c-452e-a875-413d06666b8e None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:42.804136 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f671e9-5706-4d77-93dd-a96ff1561d08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:42.816838 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f671e9-5706-4d77-93dd-a96ff1561d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:42.818555 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f671e9-5706-4d77-93dd-a96ff1561d08 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-b2f671e9-5706-4d77-93dd-a96ff1561d08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:42.838584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:42.839814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:42.841110 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:42.841110 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:42.841110 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:42.841110 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:42.843212 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 450/905] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:42.849945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f671e9-5706-4d77-93dd-a96ff1561d08 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:42.929294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d242d2e-139c-452e-a875-413d06666b8e None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:42.937468 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:42.976381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddcefd-9bf3-4f7e-8820-067c871a1340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:42.977085 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddcefd-9bf3-4f7e-8820-067c871a1340 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:42.977374 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddcefd-9bf3-4f7e-8820-067c871a1340 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:42.980358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddcefd-9bf3-4f7e-8820-067c871a1340 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:42.980358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddcefd-9bf3-4f7e-8820-067c871a1340 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:42.980358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddcefd-9bf3-4f7e-8820-067c871a1340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:42.980358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddcefd-9bf3-4f7e-8820-067c871a1340 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:42.987443 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f671e9-5706-4d77-93dd-a96ff1561d08 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:42.991514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 456/906] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:29:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:42.992562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddcefd-9bf3-4f7e-8820-067c871a1340 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:43.015475 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:43.037922 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dea8d4-ab9f-431e-92a3-32891799a30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:43.039343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dea8d4-ab9f-431e-92a3-32891799a30e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:43.039814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dea8d4-ab9f-431e-92a3-32891799a30e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:43.040577 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dea8d4-ab9f-431e-92a3-32891799a30e None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:43.042043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dea8d4-ab9f-431e-92a3-32891799a30e None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:43.042510 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dea8d4-ab9f-431e-92a3-32891799a30e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:43.044686 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dea8d4-ab9f-431e-92a3-32891799a30e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:43.066432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 451/907] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:43.071803 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dea8d4-ab9f-431e-92a3-32891799a30e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:43.112373 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddcefd-9bf3-4f7e-8820-067c871a1340 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:43.151196 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd5d2f-3977-4359-95ae-74605c7fb63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:43.152238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd5d2f-3977-4359-95ae-74605c7fb63d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:43.152784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd5d2f-3977-4359-95ae-74605c7fb63d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:43.153293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd5d2f-3977-4359-95ae-74605c7fb63d None admin] PATH_INFO: `/v3/endpoints` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:43.154322 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd5d2f-3977-4359-95ae-74605c7fb63d None admin] RBAC: Adding query filter params (interface=public, service_id=cff07afc16d14fef9e07c78ac60a6991, region_id=RegionOne) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:43.154853 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd5d2f-3977-4359-95ae-74605c7fb63d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:43.156490 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd5d2f-3977-4359-95ae-74605c7fb63d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:43.163933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 457/908] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:29:42 2022] GET /identity/v3/endpoints?service_id=cff07afc16d14fef9e07c78ac60a6991&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:43.169995 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd5d2f-3977-4359-95ae-74605c7fb63d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:43.214813 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dea8d4-ab9f-431e-92a3-32891799a30e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:43.245378 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:43.255751 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18ddf2-91b4-4592-a882-af860ec7cf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:43.256891 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18ddf2-91b4-4592-a882-af860ec7cf8f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:43.257427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18ddf2-91b4-4592-a882-af860ec7cf8f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:43.257913 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18ddf2-91b4-4592-a882-af860ec7cf8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:43.265312 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18ddf2-91b4-4592-a882-af860ec7cf8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:43.266897 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:43.267185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18ddf2-91b4-4592-a882-af860ec7cf8f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-4b18ddf2-91b4-4592-a882-af860ec7cf8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:43.283106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:43.285246 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:43.286525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:43.286525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:43.286525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:43.286525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:43.286525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:43.286525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:43.286525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:43.286525 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:43.287630 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 452/909] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:43.292245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd5d2f-3977-4359-95ae-74605c7fb63d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:43.294107 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18ddf2-91b4-4592-a882-af860ec7cf8f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:43.340224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b7b60-2ef3-43c2-8fd2-d6e4e9468090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:43.341534 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b7b60-2ef3-43c2-8fd2-d6e4e9468090 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:43.341824 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b7b60-2ef3-43c2-8fd2-d6e4e9468090 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:43.342101 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b7b60-2ef3-43c2-8fd2-d6e4e9468090 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:43.342797 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b7b60-2ef3-43c2-8fd2-d6e4e9468090 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:43.345168 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b7b60-2ef3-43c2-8fd2-d6e4e9468090 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:43.356426 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 458/910] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:29:43 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 20:29:43.420944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18ddf2-91b4-4592-a882-af860ec7cf8f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:43.473119 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126c3ad-f37b-4ac0-8a0f-c0ddf5cd6876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:43.474271 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c3ad-f37b-4ac0-8a0f-c0ddf5cd6876 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:43.474779 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c3ad-f37b-4ac0-8a0f-c0ddf5cd6876 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:43.475276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c3ad-f37b-4ac0-8a0f-c0ddf5cd6876 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:43.476434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126c3ad-f37b-4ac0-8a0f-c0ddf5cd6876 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:43.477045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126c3ad-f37b-4ac0-8a0f-c0ddf5cd6876 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:43.479023 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126c3ad-f37b-4ac0-8a0f-c0ddf5cd6876 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:43.494865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 453/911] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:43.501500 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b7b60-2ef3-43c2-8fd2-d6e4e9468090 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:43.613924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b7b60-2ef3-43c2-8fd2-d6e4e9468090 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:43.650962 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbb5c9-5cf2-4d34-8980-2fa4f4614a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:43.651614 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbb5c9-5cf2-4d34-8980-2fa4f4614a04 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:43.651954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbb5c9-5cf2-4d34-8980-2fa4f4614a04 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:43.652235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbb5c9-5cf2-4d34-8980-2fa4f4614a04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:43.653219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbb5c9-5cf2-4d34-8980-2fa4f4614a04 None admin] RBAC: Adding query filter params (role.id=bc67bfe0fbca4ce2bbb378ea4a046931, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=5f5896afd9434adebe8c1d8db70b28bf) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:43.653538 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbb5c9-5cf2-4d34-8980-2fa4f4614a04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:43.654958 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbb5c9-5cf2-4d34-8980-2fa4f4614a04 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:43.661845 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 459/912] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:43 2022] GET /identity/v3/role_assignments?user.id=5f5896afd9434adebe8c1d8db70b28bf&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=bc67bfe0fbca4ce2bbb378ea4a046931 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:44.084153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d91d3-99c7-4430-99a4-d2c0c14ec29c None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:44.084752 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d91d3-99c7-4430-99a4-d2c0c14ec29c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:44.084752 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635d91d3-99c7-4430-99a4-d2c0c14ec29c None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:44.086369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 454/913] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:44.092061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09a527-9140-4e4a-900c-d5e2b9e37b2d None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:44.092533 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09a527-9140-4e4a-900c-d5e2b9e37b2d None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:44.092765 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09a527-9140-4e4a-900c-d5e2b9e37b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:44.111406 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-ed09a527-9140-4e4a-900c-d5e2b9e37b2d None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:44.112634 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-ed09a527-9140-4e4a-900c-d5e2b9e37b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:44.116565 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:44.190052 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 460/914] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:44.212874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d91d3-99c7-4430-99a4-d2c0c14ec29c None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:44.267924 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:44.332780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635d91d3-99c7-4430-99a4-d2c0c14ec29c None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:44.368997 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b4ba4-703c-4c06-8b77-e657f95e9b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:44.370172 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b4ba4-703c-4c06-8b77-e657f95e9b34 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:44.370692 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b4ba4-703c-4c06-8b77-e657f95e9b34 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:44.371189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b4ba4-703c-4c06-8b77-e657f95e9b34 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:44.372051 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b4ba4-703c-4c06-8b77-e657f95e9b34 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:44.373669 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b4ba4-703c-4c06-8b77-e657f95e9b34 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-8b4b4ba4-703c-4c06-8b77-e657f95e9b34 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:44.387063 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:44.388185 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:44.389161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:44.392117 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 455/915] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:29:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:44.400383 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09a527-9140-4e4a-900c-d5e2b9e37b2d None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:44.511735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09a527-9140-4e4a-900c-d5e2b9e37b2d None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:44.547953 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50396647-3b86-48d8-987f-b9475d5eea1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:44.548687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50396647-3b86-48d8-987f-b9475d5eea1a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:44.549032 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50396647-3b86-48d8-987f-b9475d5eea1a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:44.549310 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50396647-3b86-48d8-987f-b9475d5eea1a None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:44.550080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50396647-3b86-48d8-987f-b9475d5eea1a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:44.550390 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50396647-3b86-48d8-987f-b9475d5eea1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:44.551984 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50396647-3b86-48d8-987f-b9475d5eea1a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:44.557463 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 461/916] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:29:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:44.563867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b4ba4-703c-4c06-8b77-e657f95e9b34 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:44.672043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b4ba4-703c-4c06-8b77-e657f95e9b34 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:44.684010 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:44.708626 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa12693-8ad1-4c8b-b698-923c483d6a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:44.709311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa12693-8ad1-4c8b-b698-923c483d6a90 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:44.709597 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa12693-8ad1-4c8b-b698-923c483d6a90 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:44.709877 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa12693-8ad1-4c8b-b698-923c483d6a90 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:44.710650 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa12693-8ad1-4c8b-b698-923c483d6a90 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:44.710963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa12693-8ad1-4c8b-b698-923c483d6a90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:44.712367 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa12693-8ad1-4c8b-b698-923c483d6a90 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:44.717928 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 456/917] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:29:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:44.724147 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50396647-3b86-48d8-987f-b9475d5eea1a None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:44.728186 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:44.845801 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50396647-3b86-48d8-987f-b9475d5eea1a None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:44.885431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843c005-e269-489f-a189-afafab43c9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:44.886101 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843c005-e269-489f-a189-afafab43c9c4 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:44.886389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843c005-e269-489f-a189-afafab43c9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:44.886662 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843c005-e269-489f-a189-afafab43c9c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:44.887459 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843c005-e269-489f-a189-afafab43c9c4 None admin] RBAC: Adding query filter params (interface=public, service_id=ce2c2068ba1842419209fcafe29ff538, region_id=RegionOne) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:44.887772 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843c005-e269-489f-a189-afafab43c9c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:44.889154 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843c005-e269-489f-a189-afafab43c9c4 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:44.895073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 462/918] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:29:44 2022] GET /identity/v3/endpoints?service_id=ce2c2068ba1842419209fcafe29ff538&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:44.900293 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa12693-8ad1-4c8b-b698-923c483d6a90 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:44.982393 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bef3c7-5486-4d45-94b3-dc863f0b554c None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:44.982575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bef3c7-5486-4d45-94b3-dc863f0b554c None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:44.982856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bef3c7-5486-4d45-94b3-dc863f0b554c None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:44.984561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 463/919] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:29:44.989816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851c942-6218-49f1-a538-415c93466510 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:44.990116 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851c942-6218-49f1-a538-415c93466510 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:44.990397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851c942-6218-49f1-a538-415c93466510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:45.007218 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa12693-8ad1-4c8b-b698-923c483d6a90 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:45.007909 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-b851c942-6218-49f1-a538-415c93466510 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:45.009139 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-b851c942-6218-49f1-a538-415c93466510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:45.049604 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7684326-1ce9-42c9-a71b-dd658e75487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:45.050280 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684326-1ce9-42c9-a71b-dd658e75487f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:45.050567 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684326-1ce9-42c9-a71b-dd658e75487f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:45.050845 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684326-1ce9-42c9-a71b-dd658e75487f None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:45.051570 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7684326-1ce9-42c9-a71b-dd658e75487f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:45.053018 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7684326-1ce9-42c9-a71b-dd658e75487f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:45.068383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 457/920] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:29:44 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:29:45.095562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:45.108513 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 464/921] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:29:45.126576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7684326-1ce9-42c9-a71b-dd658e75487f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:45.177345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:45.251299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7684326-1ce9-42c9-a71b-dd658e75487f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:45.290599 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf00f77-e391-4405-80d4-474dbd27524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:45.291680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf00f77-e391-4405-80d4-474dbd27524b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:45.291965 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf00f77-e391-4405-80d4-474dbd27524b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:45.292242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf00f77-e391-4405-80d4-474dbd27524b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:45.292930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf00f77-e391-4405-80d4-474dbd27524b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:45.294320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf00f77-e391-4405-80d4-474dbd27524b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-9bf00f77-e391-4405-80d4-474dbd27524b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:45.306920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:45.308042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:45.309189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 458/922] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:29:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:45.314399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851c942-6218-49f1-a538-415c93466510 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:45.424388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851c942-6218-49f1-a538-415c93466510 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:45.474769 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c74751-6e6e-4962-aa87-ca91e681fe42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:45.475770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c74751-6e6e-4962-aa87-ca91e681fe42 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:45.476271 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c74751-6e6e-4962-aa87-ca91e681fe42 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:45.476780 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c74751-6e6e-4962-aa87-ca91e681fe42 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:45.477685 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c74751-6e6e-4962-aa87-ca91e681fe42 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:45.478271 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c74751-6e6e-4962-aa87-ca91e681fe42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:45.479851 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c74751-6e6e-4962-aa87-ca91e681fe42 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:45.486400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 465/923] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:29:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:45.494966 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf00f77-e391-4405-80d4-474dbd27524b None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:45.611194 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf00f77-e391-4405-80d4-474dbd27524b None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:45.612996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:45.662831 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f23573-bc17-4434-8abe-780dd5a0bc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:45.663837 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f23573-bc17-4434-8abe-780dd5a0bc6b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:45.664253 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f23573-bc17-4434-8abe-780dd5a0bc6b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:45.664683 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f23573-bc17-4434-8abe-780dd5a0bc6b None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:45.665786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f23573-bc17-4434-8abe-780dd5a0bc6b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:45.666523 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f23573-bc17-4434-8abe-780dd5a0bc6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:45.668162 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f23573-bc17-4434-8abe-780dd5a0bc6b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:45.679180 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 459/924] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:29:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:45.680566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c74751-6e6e-4962-aa87-ca91e681fe42 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:45.746148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:45.793442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c74751-6e6e-4962-aa87-ca91e681fe42 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:45.824362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fc863-aee9-4fcb-b2fb-c678a12f206a None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:45.824712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fc863-aee9-4fcb-b2fb-c678a12f206a None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:45.824944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16fc863-aee9-4fcb-b2fb-c678a12f206a None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:45.826646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 460/925] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:45.832539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9428772-ad18-4366-83fd-c0ed1cdd97a6 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:45.832853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9428772-ad18-4366-83fd-c0ed1cdd97a6 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:45.833107 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9428772-ad18-4366-83fd-c0ed1cdd97a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:45.834304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a619242-bcd4-4ec3-b508-a2fbdddf6836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:45.835146 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a619242-bcd4-4ec3-b508-a2fbdddf6836 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:45.835592 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a619242-bcd4-4ec3-b508-a2fbdddf6836 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:45.836028 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a619242-bcd4-4ec3-b508-a2fbdddf6836 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:45.837007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a619242-bcd4-4ec3-b508-a2fbdddf6836 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:45.838628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a619242-bcd4-4ec3-b508-a2fbdddf6836 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:45.850897 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-b9428772-ad18-4366-83fd-c0ed1cdd97a6 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:45.852283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-b9428772-ad18-4366-83fd-c0ed1cdd97a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:45.918524 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 466/926] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:29:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:45.950327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 461/927] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:45.974895 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a619242-bcd4-4ec3-b508-a2fbdddf6836 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:46.066148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085653cf-ca7e-426c-b4ca-3527118ba688 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:46.066445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085653cf-ca7e-426c-b4ca-3527118ba688 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:46.066768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085653cf-ca7e-426c-b4ca-3527118ba688 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:46.068534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 462/928] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:29:46.074325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46781da3-8f56-42d9-b8e1-1cf19acb08b5 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:46.074624 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46781da3-8f56-42d9-b8e1-1cf19acb08b5 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:46.074888 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46781da3-8f56-42d9-b8e1-1cf19acb08b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:46.093417 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-46781da3-8f56-42d9-b8e1-1cf19acb08b5 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:46.095376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-46781da3-8f56-42d9-b8e1-1cf19acb08b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:46.114435 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a619242-bcd4-4ec3-b508-a2fbdddf6836 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:46.133402 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:46.160887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed181f8-e9ff-4d01-a80c-102b0feed971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:46.161781 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed181f8-e9ff-4d01-a80c-102b0feed971 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:46.162061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed181f8-e9ff-4d01-a80c-102b0feed971 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:46.162334 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed181f8-e9ff-4d01-a80c-102b0feed971 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:46.162992 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed181f8-e9ff-4d01-a80c-102b0feed971 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:46.165128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed181f8-e9ff-4d01-a80c-102b0feed971 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-bed181f8-e9ff-4d01-a80c-102b0feed971 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:46.179779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:46.180925 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:46.181861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 467/929] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 20:29:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:46.185949 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 463/930] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:29:46.187036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed181f8-e9ff-4d01-a80c-102b0feed971 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:46.210941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46781da3-8f56-42d9-b8e1-1cf19acb08b5 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:46.253690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:46.300295 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed181f8-e9ff-4d01-a80c-102b0feed971 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:46.329721 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46781da3-8f56-42d9-b8e1-1cf19acb08b5 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:46.335535 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a08c7d-6d63-4fd6-b219-32a883857c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:46.336246 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08c7d-6d63-4fd6-b219-32a883857c6d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:46.336565 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08c7d-6d63-4fd6-b219-32a883857c6d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:46.336840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a08c7d-6d63-4fd6-b219-32a883857c6d None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:46.337575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a08c7d-6d63-4fd6-b219-32a883857c6d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:46.337889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a08c7d-6d63-4fd6-b219-32a883857c6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:46.339442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a08c7d-6d63-4fd6-b219-32a883857c6d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:46.347168 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 468/931] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:29:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:46.352631 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a08c7d-6d63-4fd6-b219-32a883857c6d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:46.381133 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d26d29-6b27-4269-9921-567ea5c53e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:46.382338 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d26d29-6b27-4269-9921-567ea5c53e35 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:46.382842 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d26d29-6b27-4269-9921-567ea5c53e35 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:46.383493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d26d29-6b27-4269-9921-567ea5c53e35 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:46.397816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d26d29-6b27-4269-9921-567ea5c53e35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-78d26d29-6b27-4269-9921-567ea5c53e35 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:46.407444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:46.408585 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:29:46.409514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:46.411384 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:46.413421 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 464/932] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:29:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:46.421290 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d26d29-6b27-4269-9921-567ea5c53e35 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:46.489113 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a08c7d-6d63-4fd6-b219-32a883857c6d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:46.530214 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:46.545272 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228d561-7f2e-4276-b4e4-a748d84d0515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:46.546720 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d561-7f2e-4276-b4e4-a748d84d0515 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:46.547591 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d561-7f2e-4276-b4e4-a748d84d0515 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:46.548071 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228d561-7f2e-4276-b4e4-a748d84d0515 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:46.549297 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228d561-7f2e-4276-b4e4-a748d84d0515 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:46.556969 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228d561-7f2e-4276-b4e4-a748d84d0515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:46.556969 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b228d561-7f2e-4276-b4e4-a748d84d0515 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:46.557195 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d26d29-6b27-4269-9921-567ea5c53e35 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:46.559265 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 469/933] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:29:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:46.565697 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228d561-7f2e-4276-b4e4-a748d84d0515 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:46.596238 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6238b-e6b2-4bc1-a2a4-26569052e0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:46.597043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6238b-e6b2-4bc1-a2a4-26569052e0d9 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:46.597336 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6238b-e6b2-4bc1-a2a4-26569052e0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:46.597619 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6238b-e6b2-4bc1-a2a4-26569052e0d9 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:46.598379 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6238b-e6b2-4bc1-a2a4-26569052e0d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:46.598709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6238b-e6b2-4bc1-a2a4-26569052e0d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:46.600115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6238b-e6b2-4bc1-a2a4-26569052e0d9 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:46.613705 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 465/934] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:29:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:46.616090 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6238b-e6b2-4bc1-a2a4-26569052e0d9 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:46.696142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b228d561-7f2e-4276-b4e4-a748d84d0515 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:46.708394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:46.735851 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25775a5-37eb-478a-b15a-40d2663cbea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:46.738274 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25775a5-37eb-478a-b15a-40d2663cbea1 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:46.739016 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25775a5-37eb-478a-b15a-40d2663cbea1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:46.739542 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25775a5-37eb-478a-b15a-40d2663cbea1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:46.740765 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25775a5-37eb-478a-b15a-40d2663cbea1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:46.742749 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25775a5-37eb-478a-b15a-40d2663cbea1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:46.758417 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6238b-e6b2-4bc1-a2a4-26569052e0d9 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:46.799445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187abf1-d5fa-428c-94f3-f722b1c93363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:46.800394 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187abf1-d5fa-428c-94f3-f722b1c93363 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:46.800756 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187abf1-d5fa-428c-94f3-f722b1c93363 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:46.801097 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187abf1-d5fa-428c-94f3-f722b1c93363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:46.813533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187abf1-d5fa-428c-94f3-f722b1c93363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:46.815211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187abf1-d5fa-428c-94f3-f722b1c93363 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:46.818503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-e187abf1-d5fa-428c-94f3-f722b1c93363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:46.835013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:46.836073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:46.841493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:46.841493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:46.841493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:46.841493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:46.841493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 466/935] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:46.843018 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187abf1-d5fa-428c-94f3-f722b1c93363 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:46.866256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 470/936] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:29:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:29:46.970584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187abf1-d5fa-428c-94f3-f722b1c93363 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:47.015309 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db00d0-86b5-4e60-85ce-d2bfcae53fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:47.016098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db00d0-86b5-4e60-85ce-d2bfcae53fe4 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:47.016422 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db00d0-86b5-4e60-85ce-d2bfcae53fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:47.016708 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db00d0-86b5-4e60-85ce-d2bfcae53fe4 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:47.017453 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db00d0-86b5-4e60-85ce-d2bfcae53fe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:47.017766 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db00d0-86b5-4e60-85ce-d2bfcae53fe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:47.019174 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db00d0-86b5-4e60-85ce-d2bfcae53fe4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:47.033603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 467/937] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:47.039239 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25775a5-37eb-478a-b15a-40d2663cbea1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:47.157878 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25775a5-37eb-478a-b15a-40d2663cbea1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:47.197124 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a81f08-fd10-4d96-ac0c-c25bf6b9fdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:47.198326 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81f08-fd10-4d96-ac0c-c25bf6b9fdeb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:47.198826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81f08-fd10-4d96-ac0c-c25bf6b9fdeb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:47.199366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81f08-fd10-4d96-ac0c-c25bf6b9fdeb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:47.209028 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a81f08-fd10-4d96-ac0c-c25bf6b9fdeb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:47.210839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a81f08-fd10-4d96-ac0c-c25bf6b9fdeb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-a3a81f08-fd10-4d96-ac0c-c25bf6b9fdeb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:47.227721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:47.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:47.229965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:47.229965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:47.229965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:47.229965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:47.229965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:47.229965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:47.229965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:47.229965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:47.229965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:47.229965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:47.232751 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 471/938] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:29:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:47.238739 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db00d0-86b5-4e60-85ce-d2bfcae53fe4 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:47.297839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:47.353633 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db00d0-86b5-4e60-85ce-d2bfcae53fe4 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:47.389678 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31167797-fb1a-44fa-a2f6-d0271d6dc2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:47.390743 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31167797-fb1a-44fa-a2f6-d0271d6dc2f7 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:47.391803 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31167797-fb1a-44fa-a2f6-d0271d6dc2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:47.392361 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31167797-fb1a-44fa-a2f6-d0271d6dc2f7 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:47.393452 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31167797-fb1a-44fa-a2f6-d0271d6dc2f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:47.393972 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31167797-fb1a-44fa-a2f6-d0271d6dc2f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:47.395625 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31167797-fb1a-44fa-a2f6-d0271d6dc2f7 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:47.415308 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 468/939] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:29:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:47.421454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a81f08-fd10-4d96-ac0c-c25bf6b9fdeb None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:47.463864 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:47.505069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb919b57-2ac2-48cf-a224-7ba2c8c1f43c None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:47.505378 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb919b57-2ac2-48cf-a224-7ba2c8c1f43c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:47.505941 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb919b57-2ac2-48cf-a224-7ba2c8c1f43c None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:47.512359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 469/940] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:47.518482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e1fb-cbae-4dcb-9a9c-5815d8333c9e None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:47.518482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e1fb-cbae-4dcb-9a9c-5815d8333c9e None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:47.518482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53e1fb-cbae-4dcb-9a9c-5815d8333c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:47.532989 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-2a53e1fb-cbae-4dcb-9a9c-5815d8333c9e None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:47.534591 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-2a53e1fb-cbae-4dcb-9a9c-5815d8333c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:47.550232 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a81f08-fd10-4d96-ac0c-c25bf6b9fdeb None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:47.596101 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d09388-6fc3-40e0-a496-58f48d2922d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:47.596937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d09388-6fc3-40e0-a496-58f48d2922d1 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:47.597220 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d09388-6fc3-40e0-a496-58f48d2922d1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:47.597494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d09388-6fc3-40e0-a496-58f48d2922d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:47.611563 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d09388-6fc3-40e0-a496-58f48d2922d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:47.613267 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d09388-6fc3-40e0-a496-58f48d2922d1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:47.624701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 470/941] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-c3d09388-6fc3-40e0-a496-58f48d2922d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:47.632859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:47.633836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:47.635370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:47.635370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:47.635370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:47.635370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:47.635370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 472/942] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:47.640079 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53e1fb-cbae-4dcb-9a9c-5815d8333c9e None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:47.645609 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d09388-6fc3-40e0-a496-58f48d2922d1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:47.699603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:47.740438 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:47.770450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d09388-6fc3-40e0-a496-58f48d2922d1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:47.776709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53e1fb-cbae-4dcb-9a9c-5815d8333c9e None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:47.814676 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15c96b-21cf-48ae-b8cc-76bc24856c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:47.815790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15c96b-21cf-48ae-b8cc-76bc24856c6e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:47.816305 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15c96b-21cf-48ae-b8cc-76bc24856c6e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:47.816597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca10a7b-05ad-42f9-904d-fbacf5b819f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:47.817127 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15c96b-21cf-48ae-b8cc-76bc24856c6e None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:47.817461 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10a7b-05ad-42f9-904d-fbacf5b819f7 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:47.817748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10a7b-05ad-42f9-904d-fbacf5b819f7 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:47.818026 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca10a7b-05ad-42f9-904d-fbacf5b819f7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:47.818699 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca10a7b-05ad-42f9-904d-fbacf5b819f7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:47.819087 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15c96b-21cf-48ae-b8cc-76bc24856c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:47.819597 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15c96b-21cf-48ae-b8cc-76bc24856c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:47.820077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca10a7b-05ad-42f9-904d-fbacf5b819f7 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:47.821472 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15c96b-21cf-48ae-b8cc-76bc24856c6e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-7ca10a7b-05ad-42f9-904d-fbacf5b819f7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:47.835567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:47.836790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:47.843395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 473/943] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:29:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:47.844389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca10a7b-05ad-42f9-904d-fbacf5b819f7 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:47.844470 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 471/944] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:47.849522 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15c96b-21cf-48ae-b8cc-76bc24856c6e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:47.976577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca10a7b-05ad-42f9-904d-fbacf5b819f7 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:47.982390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:47.990646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15c96b-21cf-48ae-b8cc-76bc24856c6e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:48.045271 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534d767-10ed-423d-ae86-eea0d6276034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:48.046330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d767-10ed-423d-ae86-eea0d6276034 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:48.046827 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d767-10ed-423d-ae86-eea0d6276034 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:48.047872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534d767-10ed-423d-ae86-eea0d6276034 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:48.048875 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534d767-10ed-423d-ae86-eea0d6276034 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:48.049397 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534d767-10ed-423d-ae86-eea0d6276034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:48.051170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534d767-10ed-423d-ae86-eea0d6276034 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:48.052010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e906355-f0c4-4f08-9550-5a4da9d694e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:48.052863 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e906355-f0c4-4f08-9550-5a4da9d694e0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:48.053149 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e906355-f0c4-4f08-9550-5a4da9d694e0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:48.053513 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e906355-f0c4-4f08-9550-5a4da9d694e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:48.064400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e906355-f0c4-4f08-9550-5a4da9d694e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:48.064947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e906355-f0c4-4f08-9550-5a4da9d694e0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:48.065258 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 472/945] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:29:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:48.067282 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534d767-10ed-423d-ae86-eea0d6276034 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-4e906355-f0c4-4f08-9550-5a4da9d694e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:48.080070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:48.081153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:48.082606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:48.082606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:48.082606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:48.082606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:48.082606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:48.082606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:48.082606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:48.082606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:48.082606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 474/946] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:48.088182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e906355-f0c4-4f08-9550-5a4da9d694e0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:48.091430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:48.197547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534d767-10ed-423d-ae86-eea0d6276034 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:48.229417 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e906355-f0c4-4f08-9550-5a4da9d694e0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:48.237483 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db7442-0138-4de8-bd89-e52ebb965ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:48.238504 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db7442-0138-4de8-bd89-e52ebb965ae9 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:48.239004 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db7442-0138-4de8-bd89-e52ebb965ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:48.239931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65db7442-0138-4de8-bd89-e52ebb965ae9 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:48.241012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db7442-0138-4de8-bd89-e52ebb965ae9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:48.241556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db7442-0138-4de8-bd89-e52ebb965ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:48.243157 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65db7442-0138-4de8-bd89-e52ebb965ae9 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:48.253850 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 473/947] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:29:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:48.255481 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db7442-0138-4de8-bd89-e52ebb965ae9 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:48.271541 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b3a97-3bf9-4ee7-8cc7-cb9194a88908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:48.272225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b3a97-3bf9-4ee7-8cc7-cb9194a88908 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:48.272548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b3a97-3bf9-4ee7-8cc7-cb9194a88908 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:48.272827 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b3a97-3bf9-4ee7-8cc7-cb9194a88908 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:48.273629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b3a97-3bf9-4ee7-8cc7-cb9194a88908 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:48.274214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b3a97-3bf9-4ee7-8cc7-cb9194a88908 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:48.275363 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b3a97-3bf9-4ee7-8cc7-cb9194a88908 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:48.288860 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:48.292551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 475/948] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:48.295262 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b3a97-3bf9-4ee7-8cc7-cb9194a88908 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:48.359338 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:48.369870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65db7442-0138-4de8-bd89-e52ebb965ae9 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:48.400813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b3a97-3bf9-4ee7-8cc7-cb9194a88908 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:48.406970 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262cb49-62fb-4630-94eb-8807c5188fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:48.408059 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262cb49-62fb-4630-94eb-8807c5188fea None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:48.408590 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262cb49-62fb-4630-94eb-8807c5188fea None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:48.409106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262cb49-62fb-4630-94eb-8807c5188fea None admin] PATH_INFO: `/v3/endpoints` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:48.410177 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262cb49-62fb-4630-94eb-8807c5188fea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:48.411662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262cb49-62fb-4630-94eb-8807c5188fea None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:48.436192 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f4c2f-74b4-4937-a953-185b1ee71da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:48.437287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f4c2f-74b4-4937-a953-185b1ee71da3 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:48.437576 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f4c2f-74b4-4937-a953-185b1ee71da3 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:48.438406 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f4c2f-74b4-4937-a953-185b1ee71da3 None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/5f5896afd9434adebe8c1d8db70b28bf/roles/bc67bfe0fbca4ce2bbb378ea4a046931` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:48.440671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f4c2f-74b4-4937-a953-185b1ee71da3 None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=5f5896afd9434adebe8c1d8db70b28bf, role_id=bc67bfe0fbca4ce2bbb378ea4a046931)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:48.442087 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f4c2f-74b4-4937-a953-185b1ee71da3 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:48.536208 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 474/949] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:29:48 2022] POST /identity/v3/endpoints => generated 349 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:48.537667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c1505-8564-4f13-bc3c-7c1d7575a4cd None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:48.537961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c1505-8564-4f13-bc3c-7c1d7575a4cd None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:48.538239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c1505-8564-4f13-bc3c-7c1d7575a4cd None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:48.539718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 475/950] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:48.545888 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a96eb-ce44-4fc7-9826-9ea6f1130097 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:48.546253 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a96eb-ce44-4fc7-9826-9ea6f1130097 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:48.546461 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a96eb-ce44-4fc7-9826-9ea6f1130097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:48.565381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 476/951] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:29:48 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/5f5896afd9434adebe8c1d8db70b28bf/roles/bc67bfe0fbca4ce2bbb378ea4a046931 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:29:48.573351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-1f7a96eb-ce44-4fc7-9826-9ea6f1130097 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:48.574695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-1f7a96eb-ce44-4fc7-9826-9ea6f1130097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:48.672307 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 476/952] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:48.693346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f4c2f-74b4-4937-a953-185b1ee71da3 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:48.818402 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f4c2f-74b4-4937-a953-185b1ee71da3 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:48.842494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:48.853869 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae74b1b-1088-40ae-9dae-be485f29a170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:48.856439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae74b1b-1088-40ae-9dae-be485f29a170 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:48.856439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae74b1b-1088-40ae-9dae-be485f29a170 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:48.856439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae74b1b-1088-40ae-9dae-be485f29a170 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:48.856439 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae74b1b-1088-40ae-9dae-be485f29a170 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:48.857225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae74b1b-1088-40ae-9dae-be485f29a170 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:48.858599 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: WARNING keystone.server.flask.common [None req-dae74b1b-1088-40ae-9dae-be485f29a170 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:29:48.907186 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.sql.core [None req-dae74b1b-1088-40ae-9dae-be485f29a170 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:29:48.907186 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:29:48.907186 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [parameters: {'id': 'eebad86de4d944a59d0880a5b940c6a5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:29:48.907186 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=208581) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-dae74b1b-1088-40ae-9dae-be485f29a170 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eebad86de4d944a59d0880a5b940c6a5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eebad86de4d944a59d0880a5b940c6a5. May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application conn.query(q) May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:29:48.926023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application result.read() May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 20:29:48.926991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application next(self.gen) May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application yield resource May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application next(self.gen) May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise self.value May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application session.commit() May 19 20:29:48.928095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self.session.flush() May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application compat.raise_( May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:29:48.929169 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:29:48.931416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application conn.query(q) May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application result.read() May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:29:48.937465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [parameters: {'id': 'eebad86de4d944a59d0880a5b940c6a5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:48.940372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:48.949095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:48.951530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:48.951530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 20:29:48.951530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 20:29:48.951530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:48.951530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:48.951530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 20:29:48.951530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 20:29:48.951530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eebad86de4d944a59d0880a5b940c6a5. May 19 20:29:48.951530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:48.951530 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 477/953] 10.222.0.88 () {60 vars in 1140 bytes} [Thu May 19 20:29:48 2022] POST /identity/v3/projects => generated 262 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:48.953034 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a96eb-ce44-4fc7-9826-9ea6f1130097 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:48.953034 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:49.050609 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a96eb-ce44-4fc7-9826-9ea6f1130097 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:49.088756 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4915f-6111-40c8-9757-7c021d1cbff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:49.089961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4915f-6111-40c8-9757-7c021d1cbff3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:49.090480 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4915f-6111-40c8-9757-7c021d1cbff3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:49.090970 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4915f-6111-40c8-9757-7c021d1cbff3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:49.098888 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4915f-6111-40c8-9757-7c021d1cbff3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:49.100684 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4915f-6111-40c8-9757-7c021d1cbff3 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-62d4915f-6111-40c8-9757-7c021d1cbff3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:49.115724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:49.116817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:49.117816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:49.117816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:49.117816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:49.117816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:49.117816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:49.117816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:49.117816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:49.117816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:49.120500 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 477/954] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:49.126396 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae74b1b-1088-40ae-9dae-be485f29a170 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:49.243547 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae74b1b-1088-40ae-9dae-be485f29a170 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:49.279581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61a9d8-172a-4347-abb1-e04d4d0ff849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:49.280318 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61a9d8-172a-4347-abb1-e04d4d0ff849 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:49.280636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61a9d8-172a-4347-abb1-e04d4d0ff849 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:49.280969 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61a9d8-172a-4347-abb1-e04d4d0ff849 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:49.282305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61a9d8-172a-4347-abb1-e04d4d0ff849 None admin] RBAC: Adding query filter params (name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:49.282619 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61a9d8-172a-4347-abb1-e04d4d0ff849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:49.284284 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61a9d8-172a-4347-abb1-e04d4d0ff849 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:49.312497 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 478/955] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:49.330930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3f7ca-a2b1-4b9d-8122-3e3a492cdc20 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:49.331502 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3f7ca-a2b1-4b9d-8122-3e3a492cdc20 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:49.331502 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3f7ca-a2b1-4b9d-8122-3e3a492cdc20 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:49.333483 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 478/956] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:49.339454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1089-ba02-4b7b-98db-547109194a8c None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:49.339772 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1089-ba02-4b7b-98db-547109194a8c None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:49.340019 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d1089-ba02-4b7b-98db-547109194a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:49.359119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-db8d1089-ba02-4b7b-98db-547109194a8c None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:49.361250 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-db8d1089-ba02-4b7b-98db-547109194a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:49.393975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:49.439535 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 479/957] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:49.463215 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3f7ca-a2b1-4b9d-8122-3e3a492cdc20 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:49.549062 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:49.587591 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3f7ca-a2b1-4b9d-8122-3e3a492cdc20 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:49.625996 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbbedc-99bc-477a-b49c-8cd2350d26bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:49.626806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbbedc-99bc-477a-b49c-8cd2350d26bd None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:49.627159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbbedc-99bc-477a-b49c-8cd2350d26bd None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:49.627376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbbedc-99bc-477a-b49c-8cd2350d26bd None admin] PATH_INFO: `/v3/services/compute` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:49.628016 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbbedc-99bc-477a-b49c-8cd2350d26bd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:49.629428 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbbedc-99bc-477a-b49c-8cd2350d26bd None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c6dbbedc-99bc-477a-b49c-8cd2350d26bd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:49.642030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:49.643219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:49.644210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 479/958] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:49.650460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d1089-ba02-4b7b-98db-547109194a8c None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:49.758604 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d1089-ba02-4b7b-98db-547109194a8c None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:49.795440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408cc1ab-ed46-42a5-9edd-4fe5274da59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:49.796129 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cc1ab-ed46-42a5-9edd-4fe5274da59c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:49.796450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cc1ab-ed46-42a5-9edd-4fe5274da59c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:49.796735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408cc1ab-ed46-42a5-9edd-4fe5274da59c None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:49.798129 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cc1ab-ed46-42a5-9edd-4fe5274da59c None admin] RBAC: Adding query filter params (name=compute) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:49.798451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cc1ab-ed46-42a5-9edd-4fe5274da59c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:49.799887 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408cc1ab-ed46-42a5-9edd-4fe5274da59c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:49.805690 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 480/959] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:49.811701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbbedc-99bc-477a-b49c-8cd2350d26bd None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:49.926664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbbedc-99bc-477a-b49c-8cd2350d26bd None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:49.964658 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c6915-96f0-43b5-b14d-b83b65e974dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:49.965627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c6915-96f0-43b5-b14d-b83b65e974dd None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:49.966088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c6915-96f0-43b5-b14d-b83b65e974dd None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:49.966518 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c6915-96f0-43b5-b14d-b83b65e974dd None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:49.967419 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c6915-96f0-43b5-b14d-b83b65e974dd None admin] RBAC: Adding query filter params (type=compute) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:49.967903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c6915-96f0-43b5-b14d-b83b65e974dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:49.970116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c6915-96f0-43b5-b14d-b83b65e974dd None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:49.971834 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:49.976035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 480/960] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:49 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:50.965659 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196ad2f-3d49-4e74-a13f-a317df7c3ad6 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:50.966225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196ad2f-3d49-4e74-a13f-a317df7c3ad6 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:50.966225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196ad2f-3d49-4e74-a13f-a317df7c3ad6 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:50.967848 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 481/961] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:50.974161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33e699-1777-43b8-9918-558a0a1d0419 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:50.974631 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33e699-1777-43b8-9918-558a0a1d0419 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:50.974717 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33e699-1777-43b8-9918-558a0a1d0419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:50.992701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-1e33e699-1777-43b8-9918-558a0a1d0419 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:50.993894 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-1e33e699-1777-43b8-9918-558a0a1d0419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:51.036229 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4054-2b78-4e19-b036-6ddb1cbbdc7b None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.036593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4054-2b78-4e19-b036-6ddb1cbbdc7b None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.036848 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c4054-2b78-4e19-b036-6ddb1cbbdc7b None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.038495 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 482/962] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:51.044009 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a9931-88bf-4f3c-9df1-7a0a473d91ac None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.044299 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a9931-88bf-4f3c-9df1-7a0a473d91ac None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.044609 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a9931-88bf-4f3c-9df1-7a0a473d91ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.061907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-563a9931-88bf-4f3c-9df1-7a0a473d91ac None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:51.063334 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-563a9931-88bf-4f3c-9df1-7a0a473d91ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:51.071229 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 481/963] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:51.092933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:51.096850 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33e699-1777-43b8-9918-558a0a1d0419 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:51.158353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 483/964] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:29:51.177203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a9931-88bf-4f3c-9df1-7a0a473d91ac None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:51.218399 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33e699-1777-43b8-9918-558a0a1d0419 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:51.247814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:51.256984 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94579ba9-813e-48be-a331-51b74062ad9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:51.257953 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94579ba9-813e-48be-a331-51b74062ad9c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.258338 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94579ba9-813e-48be-a331-51b74062ad9c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.258711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94579ba9-813e-48be-a331-51b74062ad9c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.259499 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94579ba9-813e-48be-a331-51b74062ad9c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:51.261000 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94579ba9-813e-48be-a331-51b74062ad9c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-94579ba9-813e-48be-a331-51b74062ad9c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:51.274311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:51.275645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:51.277283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 482/965] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:29:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:51.283765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94579ba9-813e-48be-a331-51b74062ad9c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:51.316637 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a9931-88bf-4f3c-9df1-7a0a473d91ac None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:51.363328 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab364f4-5e38-48be-a92b-a4be83bbc2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:51.364149 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab364f4-5e38-48be-a92b-a4be83bbc2e8 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.364511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab364f4-5e38-48be-a92b-a4be83bbc2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.365025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab364f4-5e38-48be-a92b-a4be83bbc2e8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.378577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab364f4-5e38-48be-a92b-a4be83bbc2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-7ab364f4-5e38-48be-a92b-a4be83bbc2e8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:29:51.389127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:51.390194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:29:51.391203 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:51.395576 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 484/966] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:29:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:51.396465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab364f4-5e38-48be-a92b-a4be83bbc2e8 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:51.413892 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94579ba9-813e-48be-a331-51b74062ad9c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:51.432039 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:51.458645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27323c-4ca9-43bf-a644-e16ad2dc01ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:51.460174 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27323c-4ca9-43bf-a644-e16ad2dc01ff None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.460485 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27323c-4ca9-43bf-a644-e16ad2dc01ff None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.460756 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27323c-4ca9-43bf-a644-e16ad2dc01ff None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.461548 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27323c-4ca9-43bf-a644-e16ad2dc01ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:51.461856 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27323c-4ca9-43bf-a644-e16ad2dc01ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:51.463242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27323c-4ca9-43bf-a644-e16ad2dc01ff None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:51.477642 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 483/967] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:29:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:51.478592 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27323c-4ca9-43bf-a644-e16ad2dc01ff None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:51.536708 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab364f4-5e38-48be-a92b-a4be83bbc2e8 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:51.563890 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:51.575295 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defb447-be9d-49fb-b417-bb6651f6d91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:51.576043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defb447-be9d-49fb-b417-bb6651f6d91e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.576322 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defb447-be9d-49fb-b417-bb6651f6d91e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.576665 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defb447-be9d-49fb-b417-bb6651f6d91e None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.577418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defb447-be9d-49fb-b417-bb6651f6d91e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:51.577723 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defb447-be9d-49fb-b417-bb6651f6d91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:51.579202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defb447-be9d-49fb-b417-bb6651f6d91e None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:51.587951 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 485/968] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:29:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:51.594416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defb447-be9d-49fb-b417-bb6651f6d91e None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:51.628314 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27323c-4ca9-43bf-a644-e16ad2dc01ff None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:51.689392 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d42f-f18a-4534-bedd-3c19dc13423c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:51.690388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d42f-f18a-4534-bedd-3c19dc13423c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.691287 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d42f-f18a-4534-bedd-3c19dc13423c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.691752 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d42f-f18a-4534-bedd-3c19dc13423c None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.693173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d42f-f18a-4534-bedd-3c19dc13423c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:51.693955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d42f-f18a-4534-bedd-3c19dc13423c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:51.695805 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d42f-f18a-4534-bedd-3c19dc13423c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:51.700975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:51.707508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 484/969] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:29:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:51.708678 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d42f-f18a-4534-bedd-3c19dc13423c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:51.723224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defb447-be9d-49fb-b417-bb6651f6d91e None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:51.770812 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a3d16-faac-4f6f-a5c7-43c83767d578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:51.772376 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a3d16-faac-4f6f-a5c7-43c83767d578 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.773120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a3d16-faac-4f6f-a5c7-43c83767d578 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.773555 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a3d16-faac-4f6f-a5c7-43c83767d578 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.789215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a3d16-faac-4f6f-a5c7-43c83767d578 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:51.791741 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a3d16-faac-4f6f-a5c7-43c83767d578 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-293a3d16-faac-4f6f-a5c7-43c83767d578 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:51.815161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:51.816195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:51.817306 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:51.817306 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:51.817306 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:51.817306 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:51.817306 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 486/970] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:51.818230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb9ff1-8f3d-4e6a-aa01-e02590dbaad7 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.818230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb9ff1-8f3d-4e6a-aa01-e02590dbaad7 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.818498 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb9ff1-8f3d-4e6a-aa01-e02590dbaad7 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.819871 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 487/971] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:51.822229 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb9ff1-8f3d-4e6a-aa01-e02590dbaad7 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:51.834009 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d42f-f18a-4534-bedd-3c19dc13423c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:51.859015 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:51.876009 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5a324-f78c-4449-8bb2-694054f57e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:51.876787 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5a324-f78c-4449-8bb2-694054f57e55 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.877082 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5a324-f78c-4449-8bb2-694054f57e55 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.877365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5a324-f78c-4449-8bb2-694054f57e55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.878161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5a324-f78c-4449-8bb2-694054f57e55 None admin] RBAC: Adding query filter params (interface=admin, service_id=ce2c2068ba1842419209fcafe29ff538, region_id=RegionOne) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:51.878474 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5a324-f78c-4449-8bb2-694054f57e55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:51.880083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5a324-f78c-4449-8bb2-694054f57e55 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:51.890128 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 485/972] 10.222.0.88 () {56 vars in 1249 bytes} [Thu May 19 20:29:51 2022] GET /identity/v3/endpoints?service_id=ce2c2068ba1842419209fcafe29ff538&interface=admin®ion_id=RegionOne => generated 193 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:51.890128 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef87844-d568-457c-a024-4027699b6f6c None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.890128 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef87844-d568-457c-a024-4027699b6f6c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.890128 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef87844-d568-457c-a024-4027699b6f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.905433 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-5ef87844-d568-457c-a024-4027699b6f6c None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:51.907317 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-5ef87844-d568-457c-a024-4027699b6f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:51.942227 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb9ff1-8f3d-4e6a-aa01-e02590dbaad7 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:51.984943 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136d1f0-13e8-4238-bd45-87a678b5041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:51.985667 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d1f0-13e8-4238-bd45-87a678b5041b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:51.985920 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d1f0-13e8-4238-bd45-87a678b5041b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:51.986201 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136d1f0-13e8-4238-bd45-87a678b5041b None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:51.986977 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136d1f0-13e8-4238-bd45-87a678b5041b None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:51.987319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136d1f0-13e8-4238-bd45-87a678b5041b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:51.989471 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136d1f0-13e8-4238-bd45-87a678b5041b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:51.992833 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:51.997041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 486/973] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:51.998385 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef87844-d568-457c-a024-4027699b6f6c None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:52.018701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 488/974] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:52.024309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136d1f0-13e8-4238-bd45-87a678b5041b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:52.124721 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef87844-d568-457c-a024-4027699b6f6c None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:52.139394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136d1f0-13e8-4238-bd45-87a678b5041b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:52.145892 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:52.168960 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf19fb0-fb61-4ba9-91b4-8cb7074b7b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:52.169732 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf19fb0-fb61-4ba9-91b4-8cb7074b7b43 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:52.169989 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf19fb0-fb61-4ba9-91b4-8cb7074b7b43 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:52.170267 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf19fb0-fb61-4ba9-91b4-8cb7074b7b43 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:52.171122 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf19fb0-fb61-4ba9-91b4-8cb7074b7b43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:52.172365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf19fb0-fb61-4ba9-91b4-8cb7074b7b43 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:52.176012 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15787ae2-c823-48e5-9ddf-fbf837cc804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:52.176860 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787ae2-c823-48e5-9ddf-fbf837cc804a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:52.177148 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787ae2-c823-48e5-9ddf-fbf837cc804a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:52.177417 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15787ae2-c823-48e5-9ddf-fbf837cc804a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:52.184733 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 487/975] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:29:51 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:29:52.186063 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15787ae2-c823-48e5-9ddf-fbf837cc804a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:52.186465 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf19fb0-fb61-4ba9-91b4-8cb7074b7b43 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:52.188131 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15787ae2-c823-48e5-9ddf-fbf837cc804a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-15787ae2-c823-48e5-9ddf-fbf837cc804a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:52.205743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:52.206906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:52.208578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:52.208578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:52.208578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:52.208578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:52.208578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:52.208578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:52.208578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:52.208578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:52.208578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:29:52.208578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:52.212160 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 489/976] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:29:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:52.213192 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15787ae2-c823-48e5-9ddf-fbf837cc804a None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:52.291688 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:52.321815 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf19fb0-fb61-4ba9-91b4-8cb7074b7b43 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:52.344391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15787ae2-c823-48e5-9ddf-fbf837cc804a None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:52.361570 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4137343-0713-4b0e-9ae6-4c12490ad8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:52.362389 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4137343-0713-4b0e-9ae6-4c12490ad8b6 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:52.362679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4137343-0713-4b0e-9ae6-4c12490ad8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:52.362962 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4137343-0713-4b0e-9ae6-4c12490ad8b6 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:52.363680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4137343-0713-4b0e-9ae6-4c12490ad8b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:52.365163 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4137343-0713-4b0e-9ae6-4c12490ad8b6 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:52.366489 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: WARNING keystone.server.flask.common [None req-b4137343-0713-4b0e-9ae6-4c12490ad8b6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:29:52.386050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85926fe-0f37-4d39-8a0d-10e4ef939045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:52.386740 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85926fe-0f37-4d39-8a0d-10e4ef939045 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:52.387029 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85926fe-0f37-4d39-8a0d-10e4ef939045 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:52.387309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85926fe-0f37-4d39-8a0d-10e4ef939045 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:52.388165 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85926fe-0f37-4d39-8a0d-10e4ef939045 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:52.388567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85926fe-0f37-4d39-8a0d-10e4ef939045 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:52.390005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85926fe-0f37-4d39-8a0d-10e4ef939045 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:52.409286 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 490/977] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:29:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:52.410616 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931d0d1-7595-42b5-8df9-16db1ffa519b None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:52.410913 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931d0d1-7595-42b5-8df9-16db1ffa519b None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:52.411197 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931d0d1-7595-42b5-8df9-16db1ffa519b None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:52.412638 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 491/978] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:52.415266 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931d0d1-7595-42b5-8df9-16db1ffa519b None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:52.420788 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.sql.core [None req-b4137343-0713-4b0e-9ae6-4c12490ad8b6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:29:52.420788 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:29:52.420788 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [parameters: {'user_id': '8bedd97079ed4deca4f3f86ccd9a1673', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:29:52.420788 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=208582) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-b4137343-0713-4b0e-9ae6-4c12490ad8b6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application conn.query(q) May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application result.read() May 19 20:29:52.428403 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:29:52.429667 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application next(self.gen) May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application yield resource May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application next(self.gen) May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise self.value May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application session.commit() May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:29:52.430724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self.session.flush() May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application compat.raise_( May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:29:52.431711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:29:52.432712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application conn.query(q) May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application result.read() May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:29:52.433674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [parameters: {'user_id': '8bedd97079ed4deca4f3f86ccd9a1673', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:52.434806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 20:29:52.436044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 20:29:52.437383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 20:29:52.437383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 20:29:52.437383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:29:52.437383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:29:52.437383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:29:52.437383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:52.437383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 488/979] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:29:52 2022] POST /identity/v3/users => generated 160 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:52.437383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cad15-bc90-494a-be78-68dc81966480 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:52.437383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cad15-bc90-494a-be78-68dc81966480 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:52.437383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2cad15-bc90-494a-be78-68dc81966480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:52.455401 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-2c2cad15-bc90-494a-be78-68dc81966480 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:52.457004 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-2c2cad15-bc90-494a-be78-68dc81966480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:52.503839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:52.537414 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 489/980] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:52.549662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cad15-bc90-494a-be78-68dc81966480 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:52.549850 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931d0d1-7595-42b5-8df9-16db1ffa519b None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:52.565568 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:52.586745 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8574634-bf27-4e22-a271-64cfe0c3630d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:52.587402 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8574634-bf27-4e22-a271-64cfe0c3630d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:52.587684 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8574634-bf27-4e22-a271-64cfe0c3630d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:52.587955 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8574634-bf27-4e22-a271-64cfe0c3630d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:52.657240 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8574634-bf27-4e22-a271-64cfe0c3630d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:52.658675 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8574634-bf27-4e22-a271-64cfe0c3630d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:52.661931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2cad15-bc90-494a-be78-68dc81966480 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:52.699736 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34326e09-eee3-45d6-8b46-afc01758c9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:52.700594 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34326e09-eee3-45d6-8b46-afc01758c9ef None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:52.700915 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34326e09-eee3-45d6-8b46-afc01758c9ef None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:52.701192 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34326e09-eee3-45d6-8b46-afc01758c9ef None admin] PATH_INFO: `/v3/users/cinder` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:52.708783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34326e09-eee3-45d6-8b46-afc01758c9ef None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:52.710473 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34326e09-eee3-45d6-8b46-afc01758c9ef None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-34326e09-eee3-45d6-8b46-afc01758c9ef None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:29:52.725487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:52.726786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 490/981] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:29:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:52.727846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34326e09-eee3-45d6-8b46-afc01758c9ef None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:52.764041 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:52.777955 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 492/982] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:29:52 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:29:52.778886 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8574634-bf27-4e22-a271-64cfe0c3630d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:52.796394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:52.850896 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34326e09-eee3-45d6-8b46-afc01758c9ef None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:52.891272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e08379-7d20-4fc9-8ef8-c4a385f35509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:52.892270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e08379-7d20-4fc9-8ef8-c4a385f35509 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:52.892730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e08379-7d20-4fc9-8ef8-c4a385f35509 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:52.893012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e08379-7d20-4fc9-8ef8-c4a385f35509 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:52.893488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8574634-bf27-4e22-a271-64cfe0c3630d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:52.893759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e08379-7d20-4fc9-8ef8-c4a385f35509 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:52.895123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e08379-7d20-4fc9-8ef8-c4a385f35509 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:52.929195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbec1c-3294-47db-a150-4d744558629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:52.929864 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbec1c-3294-47db-a150-4d744558629d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:52.930147 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbec1c-3294-47db-a150-4d744558629d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:52.930421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fbec1c-3294-47db-a150-4d744558629d None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:52.931220 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbec1c-3294-47db-a150-4d744558629d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:52.931524 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbec1c-3294-47db-a150-4d744558629d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:52.932985 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fbec1c-3294-47db-a150-4d744558629d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:52.969690 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 493/983] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:29:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:52.970531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbec1c-3294-47db-a150-4d744558629d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:52.991218 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 491/984] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:29:52 2022] POST /identity/v3/services => generated 242 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:53.080056 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:53.119373 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fbec1c-3294-47db-a150-4d744558629d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:53.170030 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35f509-87f0-41d7-b688-5e36a774e2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:53.171154 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35f509-87f0-41d7-b688-5e36a774e2e7 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:53.171603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35f509-87f0-41d7-b688-5e36a774e2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:53.172092 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35f509-87f0-41d7-b688-5e36a774e2e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:53.185726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35f509-87f0-41d7-b688-5e36a774e2e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:53.188105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f35f509-87f0-41d7-b688-5e36a774e2e7 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-1f35f509-87f0-41d7-b688-5e36a774e2e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:29:53.208182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:53.209488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:53.210693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:29:53.210693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:29:53.210693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:29:53.210693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:53.212502 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 494/985] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:53.219112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e08379-7d20-4fc9-8ef8-c4a385f35509 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:53.325066 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:53.335827 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e08379-7d20-4fc9-8ef8-c4a385f35509 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:53.373704 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb891fd3-f707-4954-86af-0985abd58055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:53.374425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb891fd3-f707-4954-86af-0985abd58055 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:53.374944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb891fd3-f707-4954-86af-0985abd58055 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:53.375851 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb891fd3-f707-4954-86af-0985abd58055 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:53.376856 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb891fd3-f707-4954-86af-0985abd58055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:53.377240 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb891fd3-f707-4954-86af-0985abd58055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:53.379011 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb891fd3-f707-4954-86af-0985abd58055 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:53.391703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 492/986] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:53.397552 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35f509-87f0-41d7-b688-5e36a774e2e7 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:53.513526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f35f509-87f0-41d7-b688-5e36a774e2e7 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:53.534081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:53.549351 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c9874-1567-4a82-bd15-5a1ee39dcf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:53.550156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c9874-1567-4a82-bd15-5a1ee39dcf06 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:53.550441 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c9874-1567-4a82-bd15-5a1ee39dcf06 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:53.550716 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c9874-1567-4a82-bd15-5a1ee39dcf06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:53.557658 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c9874-1567-4a82-bd15-5a1ee39dcf06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:53.559160 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c9874-1567-4a82-bd15-5a1ee39dcf06 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-c09c9874-1567-4a82-bd15-5a1ee39dcf06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:29:53.575398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:53.576476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:53.577466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:29:53.577466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:29:53.577466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:29:53.577466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:29:53.577466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:29:53.577466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:29:53.577466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:29:53.577466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:53.577466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 495/987] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:53.584562 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb891fd3-f707-4954-86af-0985abd58055 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:53.690866 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb891fd3-f707-4954-86af-0985abd58055 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:53.726869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6706cb-2f37-43fc-95da-ee0282297218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:53.727582 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6706cb-2f37-43fc-95da-ee0282297218 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:53.727870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6706cb-2f37-43fc-95da-ee0282297218 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:53.728141 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6706cb-2f37-43fc-95da-ee0282297218 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:53.728961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6706cb-2f37-43fc-95da-ee0282297218 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:53.729278 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6706cb-2f37-43fc-95da-ee0282297218 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:53.730716 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6706cb-2f37-43fc-95da-ee0282297218 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:53.747984 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 493/988] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:29:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:53.749257 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c9874-1567-4a82-bd15-5a1ee39dcf06 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:53.860034 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c9874-1567-4a82-bd15-5a1ee39dcf06 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:53.894890 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc5bca-e514-4b8a-820b-400597d1191b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:53.895563 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5bca-e514-4b8a-820b-400597d1191b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:53.895852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5bca-e514-4b8a-820b-400597d1191b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:53.896129 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc5bca-e514-4b8a-820b-400597d1191b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:53.897010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc5bca-e514-4b8a-820b-400597d1191b None admin] RBAC: Adding query filter params (role.id=bc67bfe0fbca4ce2bbb378ea4a046931, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=5f5896afd9434adebe8c1d8db70b28bf) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:53.897331 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc5bca-e514-4b8a-820b-400597d1191b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:53.898756 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc5bca-e514-4b8a-820b-400597d1191b None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:53.905236 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 496/989] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:29:53 2022] GET /identity/v3/role_assignments?user.id=5f5896afd9434adebe8c1d8db70b28bf&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=bc67bfe0fbca4ce2bbb378ea4a046931 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:54.461463 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5621d-b8ed-4951-b761-122785272d49 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:54.461984 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5621d-b8ed-4951-b761-122785272d49 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:54.462078 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5621d-b8ed-4951-b761-122785272d49 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:54.463626 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 494/990] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:54.468787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d822f-504d-45ee-8c33-a3be923cbe50 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:54.469243 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d822f-504d-45ee-8c33-a3be923cbe50 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:54.469346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d822f-504d-45ee-8c33-a3be923cbe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:54.486155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-f90d822f-504d-45ee-8c33-a3be923cbe50 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:54.487352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-f90d822f-504d-45ee-8c33-a3be923cbe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:54.530467 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:54.557957 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 497/991] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:54.581538 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5621d-b8ed-4951-b761-122785272d49 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:54.594789 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:54.686511 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5621d-b8ed-4951-b761-122785272d49 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:54.721581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a1d3b-16d2-49c3-a553-baf6799ac6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:54.722315 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a1d3b-16d2-49c3-a553-baf6799ac6e2 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:54.722683 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a1d3b-16d2-49c3-a553-baf6799ac6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:54.722960 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a1d3b-16d2-49c3-a553-baf6799ac6e2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:54.723636 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a1d3b-16d2-49c3-a553-baf6799ac6e2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:54.725207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a1d3b-16d2-49c3-a553-baf6799ac6e2 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-0a8a1d3b-16d2-49c3-a553-baf6799ac6e2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:54.739369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:54.740646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:29:54.741680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:54.743214 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 495/992] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:29:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:54.746732 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d822f-504d-45ee-8c33-a3be923cbe50 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:54.850796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d822f-504d-45ee-8c33-a3be923cbe50 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:54.885634 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a5286-aa20-462c-b056-c8889bd87885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:54.886309 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a5286-aa20-462c-b056-c8889bd87885 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:54.886591 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a5286-aa20-462c-b056-c8889bd87885 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:54.886878 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a5286-aa20-462c-b056-c8889bd87885 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:54.887568 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a5286-aa20-462c-b056-c8889bd87885 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:54.887875 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a5286-aa20-462c-b056-c8889bd87885 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:54.889268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17a5286-aa20-462c-b056-c8889bd87885 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:54.899064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 498/993] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:29:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:54.899921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a1d3b-16d2-49c3-a553-baf6799ac6e2 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:54.985712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:55.011503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a1d3b-16d2-49c3-a553-baf6799ac6e2 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:55.046759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d5808-9756-4710-a94f-a7660b8cb1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:55.047949 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d5808-9756-4710-a94f-a7660b8cb1e0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:55.048232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d5808-9756-4710-a94f-a7660b8cb1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:55.048527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d5808-9756-4710-a94f-a7660b8cb1e0 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:55.049232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d5808-9756-4710-a94f-a7660b8cb1e0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:55.049541 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d5808-9756-4710-a94f-a7660b8cb1e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:55.050943 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d5808-9756-4710-a94f-a7660b8cb1e0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:55.060709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 496/994] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:29:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:29:55.061677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a5286-aa20-462c-b056-c8889bd87885 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:55.105227 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:55.171536 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a5286-aa20-462c-b056-c8889bd87885 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:55.211005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c280006-b682-48c3-9354-92f0f391c7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:55.211705 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c280006-b682-48c3-9354-92f0f391c7ff None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:55.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c280006-b682-48c3-9354-92f0f391c7ff None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:55.212254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c280006-b682-48c3-9354-92f0f391c7ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:55.213062 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c280006-b682-48c3-9354-92f0f391c7ff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:55.214478 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c280006-b682-48c3-9354-92f0f391c7ff None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:55.292363 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 499/995] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:29:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:29:55.386858 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311088ea-9034-49d0-ac14-de15b5a6f9f4 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:55.387235 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311088ea-9034-49d0-ac14-de15b5a6f9f4 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:55.387426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311088ea-9034-49d0-ac14-de15b5a6f9f4 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:55.389384 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 497/996] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:55.395077 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca69586-309f-4b68-8b51-319d88028f92 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:55.395383 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca69586-309f-4b68-8b51-319d88028f92 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:55.395660 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca69586-309f-4b68-8b51-319d88028f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:55.413822 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-3ca69586-309f-4b68-8b51-319d88028f92 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:55.415057 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-3ca69586-309f-4b68-8b51-319d88028f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:55.504932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 500/997] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:55 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:55.528037 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311088ea-9034-49d0-ac14-de15b5a6f9f4 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:55.639262 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311088ea-9034-49d0-ac14-de15b5a6f9f4 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:55.675519 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c4756-96a5-47fa-ae31-5e5b2fca0a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:55.676521 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c4756-96a5-47fa-ae31-5e5b2fca0a5a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:55.676808 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c4756-96a5-47fa-ae31-5e5b2fca0a5a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:55.677084 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c4756-96a5-47fa-ae31-5e5b2fca0a5a None admin] PATH_INFO: `/v3/services/compute` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:55.677761 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c4756-96a5-47fa-ae31-5e5b2fca0a5a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:55.679123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c4756-96a5-47fa-ae31-5e5b2fca0a5a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-0f1c4756-96a5-47fa-ae31-5e5b2fca0a5a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:55.691628 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:55.692947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:29:55.693942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:55.695593 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 498/998] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:55.699378 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca69586-309f-4b68-8b51-319d88028f92 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:55.726399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:55.805935 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca69586-309f-4b68-8b51-319d88028f92 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:55.843085 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16796e9f-5017-4c1c-b6d8-0668671e6563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:55.844518 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16796e9f-5017-4c1c-b6d8-0668671e6563 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:55.844813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16796e9f-5017-4c1c-b6d8-0668671e6563 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:55.845094 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16796e9f-5017-4c1c-b6d8-0668671e6563 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:55.845817 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16796e9f-5017-4c1c-b6d8-0668671e6563 None admin] RBAC: Adding query filter params (name=compute) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:55.846131 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16796e9f-5017-4c1c-b6d8-0668671e6563 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:55.847574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16796e9f-5017-4c1c-b6d8-0668671e6563 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:55.852817 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 501/999] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:55.858348 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c4756-96a5-47fa-ae31-5e5b2fca0a5a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:55.861378 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:55.962746 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c4756-96a5-47fa-ae31-5e5b2fca0a5a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:55.997837 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bfd8e-65d8-48e9-b16f-7b852861ca3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:55.998829 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bfd8e-65d8-48e9-b16f-7b852861ca3f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:55.999268 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bfd8e-65d8-48e9-b16f-7b852861ca3f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:55.999718 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bfd8e-65d8-48e9-b16f-7b852861ca3f None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:56.001234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bfd8e-65d8-48e9-b16f-7b852861ca3f None admin] RBAC: Adding query filter params (type=compute) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:56.001791 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bfd8e-65d8-48e9-b16f-7b852861ca3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:56.003434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bfd8e-65d8-48e9-b16f-7b852861ca3f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:56.014891 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 499/1000] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:56.017146 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16796e9f-5017-4c1c-b6d8-0668671e6563 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:56.113430 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:56.126143 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16796e9f-5017-4c1c-b6d8-0668671e6563 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:56.161658 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b137fa3-dd6d-48fb-b384-87ab20d6fd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:56.162327 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b137fa3-dd6d-48fb-b384-87ab20d6fd8a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:56.162616 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b137fa3-dd6d-48fb-b384-87ab20d6fd8a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:56.162895 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b137fa3-dd6d-48fb-b384-87ab20d6fd8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:56.163699 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b137fa3-dd6d-48fb-b384-87ab20d6fd8a None admin] RBAC: Adding query filter params (interface=public, service_id=d7e09adc1d274fa1a3e95adf9b854eab, region_id=RegionOne) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:56.164015 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b137fa3-dd6d-48fb-b384-87ab20d6fd8a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:56.165414 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b137fa3-dd6d-48fb-b384-87ab20d6fd8a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:56.176357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 502/1001] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:29:56 2022] GET /identity/v3/endpoints?service_id=d7e09adc1d274fa1a3e95adf9b854eab&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:56.178075 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bfd8e-65d8-48e9-b16f-7b852861ca3f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:56.200018 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a16f3e-575e-4b96-a441-2d92e37acd98 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:56.200460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a16f3e-575e-4b96-a441-2d92e37acd98 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:56.200743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a16f3e-575e-4b96-a441-2d92e37acd98 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:56.204389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 503/1002] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:29:56.207970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79dd90-061c-431d-ada0-f8cc3bc636fc None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:56.208262 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79dd90-061c-431d-ada0-f8cc3bc636fc None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:56.208909 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79dd90-061c-431d-ada0-f8cc3bc636fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:56.226295 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-6c79dd90-061c-431d-ada0-f8cc3bc636fc None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:56.227425 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-6c79dd90-061c-431d-ada0-f8cc3bc636fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:56.254562 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:56.290483 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bfd8e-65d8-48e9-b16f-7b852861ca3f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:56.301988 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 504/1003] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:29:56.324064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79dd90-061c-431d-ada0-f8cc3bc636fc None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:56.329401 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a61b2-2329-4e2e-acc4-4604dffd4a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:56.330080 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a61b2-2329-4e2e-acc4-4604dffd4a59 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:56.330360 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a61b2-2329-4e2e-acc4-4604dffd4a59 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:56.330630 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a61b2-2329-4e2e-acc4-4604dffd4a59 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:56.331308 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a61b2-2329-4e2e-acc4-4604dffd4a59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:56.332709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a61b2-2329-4e2e-acc4-4604dffd4a59 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:56.348373 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 500/1004] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:29:56 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:29:56.427639 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:56.456292 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79dd90-061c-431d-ada0-f8cc3bc636fc None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:56.497064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0b344-1d88-4eb0-8f82-dc191cfe3e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:56.497996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b344-1d88-4eb0-8f82-dc191cfe3e13 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:56.498383 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b344-1d88-4eb0-8f82-dc191cfe3e13 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:56.498761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0b344-1d88-4eb0-8f82-dc191cfe3e13 None admin] PATH_INFO: `/v3/services/image` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:56.499546 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0b344-1d88-4eb0-8f82-dc191cfe3e13 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:56.501385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0b344-1d88-4eb0-8f82-dc191cfe3e13 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-fca0b344-1d88-4eb0-8f82-dc191cfe3e13 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:56.514512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:56.517970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:29:56.518933 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 505/1005] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 20:29:56 2022] GET /identity/v3/services/image => generated 86 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:29:56.523896 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a61b2-2329-4e2e-acc4-4604dffd4a59 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:56.659276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a61b2-2329-4e2e-acc4-4604dffd4a59 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:56.689909 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:56.696354 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68809158-34b9-42cd-9a8c-8eb909a09d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:56.703972 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809158-34b9-42cd-9a8c-8eb909a09d21 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:56.703972 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809158-34b9-42cd-9a8c-8eb909a09d21 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:56.703972 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68809158-34b9-42cd-9a8c-8eb909a09d21 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:56.703972 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68809158-34b9-42cd-9a8c-8eb909a09d21 None admin] RBAC: Adding query filter params (name=image) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:56.703972 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68809158-34b9-42cd-9a8c-8eb909a09d21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:56.703972 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68809158-34b9-42cd-9a8c-8eb909a09d21 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:56.706995 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 501/1006] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:56.712943 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0b344-1d88-4eb0-8f82-dc191cfe3e13 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:56.820349 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0b344-1d88-4eb0-8f82-dc191cfe3e13 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:56.854857 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab045047-3108-442b-82ae-cfbf02b679ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:56.855534 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045047-3108-442b-82ae-cfbf02b679ae None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:56.855821 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045047-3108-442b-82ae-cfbf02b679ae None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:56.856091 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab045047-3108-442b-82ae-cfbf02b679ae None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:56.856837 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab045047-3108-442b-82ae-cfbf02b679ae None admin] RBAC: Adding query filter params (type=image) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:56.857212 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab045047-3108-442b-82ae-cfbf02b679ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:56.858712 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab045047-3108-442b-82ae-cfbf02b679ae None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:56.865477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 506/1007] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:57.578072 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965b20-b59f-4f51-a55c-9a7a5e07b31c None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:57.578605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965b20-b59f-4f51-a55c-9a7a5e07b31c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:57.578605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69965b20-b59f-4f51-a55c-9a7a5e07b31c None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:57.580216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 502/1008] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:57.585574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885502d0-b9b3-4b03-a1bf-816963475078 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:57.586049 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885502d0-b9b3-4b03-a1bf-816963475078 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:57.586153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885502d0-b9b3-4b03-a1bf-816963475078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:57.603455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-885502d0-b9b3-4b03-a1bf-816963475078 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:57.604668 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-885502d0-b9b3-4b03-a1bf-816963475078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:57.607566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:57.676836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 507/1009] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:57 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:57.700162 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69965b20-b59f-4f51-a55c-9a7a5e07b31c None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:57.805184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69965b20-b59f-4f51-a55c-9a7a5e07b31c None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:57.838956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f7dba-ebe5-4f3e-a026-44e451c42494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:57.839750 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f7dba-ebe5-4f3e-a026-44e451c42494 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:57.840032 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f7dba-ebe5-4f3e-a026-44e451c42494 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:57.840306 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f7dba-ebe5-4f3e-a026-44e451c42494 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:57.841516 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f7dba-ebe5-4f3e-a026-44e451c42494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:57.842953 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f7dba-ebe5-4f3e-a026-44e451c42494 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:57.845383 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 503/1010] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:29:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:57.850947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885502d0-b9b3-4b03-a1bf-816963475078 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:57.950193 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885502d0-b9b3-4b03-a1bf-816963475078 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:57.985555 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db11dfe-0eac-4af8-a2b9-cf3e4b223469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:57.986225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db11dfe-0eac-4af8-a2b9-cf3e4b223469 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:57.986508 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db11dfe-0eac-4af8-a2b9-cf3e4b223469 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:57.986781 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db11dfe-0eac-4af8-a2b9-cf3e4b223469 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:57.987518 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db11dfe-0eac-4af8-a2b9-cf3e4b223469 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:57.989043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db11dfe-0eac-4af8-a2b9-cf3e4b223469 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:58.000263 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:58.100012 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 508/1011] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:29:57 2022] POST /identity/v3/projects => generated 311 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:58.604180 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2939cb-e423-49f9-aaf2-62f6df71bfca None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:58.604664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2939cb-e423-49f9-aaf2-62f6df71bfca None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:58.604953 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2939cb-e423-49f9-aaf2-62f6df71bfca None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:58.606387 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 504/1012] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:29:58.611865 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0474405-9fcf-41f5-8181-09c70670c70e None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:58.612381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0474405-9fcf-41f5-8181-09c70670c70e None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:58.612493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0474405-9fcf-41f5-8181-09c70670c70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:58.630229 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-c0474405-9fcf-41f5-8181-09c70670c70e None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:58.631489 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-c0474405-9fcf-41f5-8181-09c70670c70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:58.712450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 509/1013] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:58.729007 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2939cb-e423-49f9-aaf2-62f6df71bfca None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:58.807381 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:58.852986 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2939cb-e423-49f9-aaf2-62f6df71bfca None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:58.902201 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cdec3-a250-4ece-b250-07883a5626e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:58.903183 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cdec3-a250-4ece-b250-07883a5626e2 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:58.903479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cdec3-a250-4ece-b250-07883a5626e2 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:58.903769 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0cdec3-a250-4ece-b250-07883a5626e2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:58.904468 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cdec3-a250-4ece-b250-07883a5626e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:58.905929 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0cdec3-a250-4ece-b250-07883a5626e2 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-be0cdec3-a250-4ece-b250-07883a5626e2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:29:58.919893 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:29:58.921148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:29:58.928042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 505/1014] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:29:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:29:58.930754 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0474405-9fcf-41f5-8181-09c70670c70e None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:59.043166 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0474405-9fcf-41f5-8181-09c70670c70e None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:59.068052 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:59.080241 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b903d6-2cfb-49a3-9c82-b94b25a96c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:59.082104 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b903d6-2cfb-49a3-9c82-b94b25a96c6a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:59.082104 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b903d6-2cfb-49a3-9c82-b94b25a96c6a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:59.082104 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b903d6-2cfb-49a3-9c82-b94b25a96c6a None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:59.082370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b903d6-2cfb-49a3-9c82-b94b25a96c6a None admin] RBAC: Adding query filter params (name=compute) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:59.082673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b903d6-2cfb-49a3-9c82-b94b25a96c6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:59.084038 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b903d6-2cfb-49a3-9c82-b94b25a96c6a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:59.094117 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 510/1015] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:59.095712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cdec3-a250-4ece-b250-07883a5626e2 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:59.181830 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b205-c4c5-432b-9806-2388fe877c93 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:59.182192 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b205-c4c5-432b-9806-2388fe877c93 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:59.182400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b205-c4c5-432b-9806-2388fe877c93 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:59.183965 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 511/1016] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:29:59.189123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f92e0c-4fff-4cf6-9ed5-277e5817223b None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:59.189419 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f92e0c-4fff-4cf6-9ed5-277e5817223b None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:59.189706 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f92e0c-4fff-4cf6-9ed5-277e5817223b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:59.207496 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-57f92e0c-4fff-4cf6-9ed5-277e5817223b None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:29:59.208750 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-57f92e0c-4fff-4cf6-9ed5-277e5817223b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:29:59.209226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0cdec3-a250-4ece-b250-07883a5626e2 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:59.240918 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:59.251111 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8ce18-97e4-494c-91f5-db9816e0d5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:59.255430 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8ce18-97e4-494c-91f5-db9816e0d5da None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:59.255430 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8ce18-97e4-494c-91f5-db9816e0d5da None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:59.255430 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8ce18-97e4-494c-91f5-db9816e0d5da None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:59.255430 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8ce18-97e4-494c-91f5-db9816e0d5da None admin] RBAC: Adding query filter params (type=compute) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:29:59.255430 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8ce18-97e4-494c-91f5-db9816e0d5da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:59.256429 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8ce18-97e4-494c-91f5-db9816e0d5da None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:59.268294 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 506/1017] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:29:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:29:59.270140 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8ce18-97e4-494c-91f5-db9816e0d5da None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:59.282116 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 512/1018] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:29:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:29:59.304883 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f92e0c-4fff-4cf6-9ed5-277e5817223b None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:29:59.387438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8ce18-97e4-494c-91f5-db9816e0d5da None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:59.423927 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f92e0c-4fff-4cf6-9ed5-277e5817223b None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:29:59.426591 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149c800-a28e-4e7f-a339-bcd31bd2e883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:59.427282 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149c800-a28e-4e7f-a339-bcd31bd2e883 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:59.427572 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149c800-a28e-4e7f-a339-bcd31bd2e883 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:59.428135 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149c800-a28e-4e7f-a339-bcd31bd2e883 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:59.429503 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149c800-a28e-4e7f-a339-bcd31bd2e883 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:59.430398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149c800-a28e-4e7f-a339-bcd31bd2e883 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:59.446873 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:29:59.464472 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82815b9a-2f0a-4955-97e1-a0214316a787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:29:59.465159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82815b9a-2f0a-4955-97e1-a0214316a787 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:29:59.465455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82815b9a-2f0a-4955-97e1-a0214316a787 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:29:59.465730 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82815b9a-2f0a-4955-97e1-a0214316a787 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:29:59.466461 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82815b9a-2f0a-4955-97e1-a0214316a787 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:29:59.467904 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82815b9a-2f0a-4955-97e1-a0214316a787 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:29:59.526108 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 507/1019] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:29:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:29:59.565489 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 513/1020] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:29:59 2022] POST /identity/v3/services => generated 242 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:30:00.463212 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fc421-b201-4e44-8514-28a6d8dede0b None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:00.463641 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fc421-b201-4e44-8514-28a6d8dede0b None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:00.463782 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820fc421-b201-4e44-8514-28a6d8dede0b None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:00.465378 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 508/1021] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:00.470599 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7d0ee-a41c-45bc-b665-1fd9eca5e895 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:00.471019 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7d0ee-a41c-45bc-b665-1fd9eca5e895 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:00.471204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7d0ee-a41c-45bc-b665-1fd9eca5e895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:00.488122 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-78b7d0ee-a41c-45bc-b665-1fd9eca5e895 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:00.489364 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-78b7d0ee-a41c-45bc-b665-1fd9eca5e895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:00.563327 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 514/1022] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:00.588350 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fc421-b201-4e44-8514-28a6d8dede0b None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:00.637758 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:00.696065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820fc421-b201-4e44-8514-28a6d8dede0b None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:00.737353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d72154-ab6b-4aac-b4f5-a1295ff30246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:00.738081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d72154-ab6b-4aac-b4f5-a1295ff30246 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:00.738362 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d72154-ab6b-4aac-b4f5-a1295ff30246 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:00.738634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d72154-ab6b-4aac-b4f5-a1295ff30246 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:00.739826 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d72154-ab6b-4aac-b4f5-a1295ff30246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:00.741310 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d72154-ab6b-4aac-b4f5-a1295ff30246 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:00.743106 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 509/1023] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:00.748646 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7d0ee-a41c-45bc-b665-1fd9eca5e895 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:00.854744 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7d0ee-a41c-45bc-b665-1fd9eca5e895 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:00.856298 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:00.891122 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe55bd6-6682-4d14-801e-39004948b9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:00.891803 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe55bd6-6682-4d14-801e-39004948b9a3 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:00.892085 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe55bd6-6682-4d14-801e-39004948b9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:00.892378 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe55bd6-6682-4d14-801e-39004948b9a3 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:00.893115 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe55bd6-6682-4d14-801e-39004948b9a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:00.894814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe55bd6-6682-4d14-801e-39004948b9a3 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:00.994348 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 515/1024] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:30:00 2022] POST /identity/v3/users => generated 285 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:30:01.848522 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569382e1-57d5-48d0-9680-0467b34e676f None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:01.849057 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569382e1-57d5-48d0-9680-0467b34e676f None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:01.849057 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569382e1-57d5-48d0-9680-0467b34e676f None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:01.850594 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 510/1025] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:01.855744 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c909e-8891-46cd-b062-8eeb951bf09e None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:01.856179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c909e-8891-46cd-b062-8eeb951bf09e None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:01.856378 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c909e-8891-46cd-b062-8eeb951bf09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:01.874567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-713c909e-8891-46cd-b062-8eeb951bf09e None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:01.875980 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-713c909e-8891-46cd-b062-8eeb951bf09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:01.908757 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9cb32-54c6-4ecc-bde1-724c1ea6cb99 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:01.909107 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9cb32-54c6-4ecc-bde1-724c1ea6cb99 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:01.909330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9cb32-54c6-4ecc-bde1-724c1ea6cb99 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:01.910728 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 511/1026] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:01.915780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e570-f92d-41e9-9e26-88b5a79172a5 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:01.916454 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e570-f92d-41e9-9e26-88b5a79172a5 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:01.916747 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e570-f92d-41e9-9e26-88b5a79172a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:01.934001 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-dc32e570-f92d-41e9-9e26-88b5a79172a5 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:01.935212 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-dc32e570-f92d-41e9-9e26-88b5a79172a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:01.953586 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 516/1027] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:01.976663 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c909e-8891-46cd-b062-8eeb951bf09e None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:02.009723 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 512/1028] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:30:02.033425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32e570-f92d-41e9-9e26-88b5a79172a5 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:02.086660 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:02.091167 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:02.106803 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c909e-8891-46cd-b062-8eeb951bf09e None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:02.146715 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff8e84-e127-4e56-bf66-aa8d73c8f652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:02.147211 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32e570-f92d-41e9-9e26-88b5a79172a5 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:02.147564 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff8e84-e127-4e56-bf66-aa8d73c8f652 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:02.147854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff8e84-e127-4e56-bf66-aa8d73c8f652 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:02.148133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff8e84-e127-4e56-bf66-aa8d73c8f652 None admin] PATH_INFO: `/v3/services/image` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:02.148842 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff8e84-e127-4e56-bf66-aa8d73c8f652 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:02.150225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff8e84-e127-4e56-bf66-aa8d73c8f652 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-afff8e84-e127-4e56-bf66-aa8d73c8f652 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:02.163784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:02.165352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:02.166577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 517/1029] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 20:30:01 2022] GET /identity/v3/services/image => generated 86 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:30:02.172525 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff8e84-e127-4e56-bf66-aa8d73c8f652 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:02.190679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa57383-5400-4f48-878f-d6e2316f3452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:02.191505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57383-5400-4f48-878f-d6e2316f3452 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:02.191813 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57383-5400-4f48-878f-d6e2316f3452 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:02.192086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57383-5400-4f48-878f-d6e2316f3452 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:02.204056 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa57383-5400-4f48-878f-d6e2316f3452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-8aa57383-5400-4f48-878f-d6e2316f3452 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:30:02.212605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:02.213695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:02.217438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 513/1030] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:30:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:30:02.220452 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa57383-5400-4f48-878f-d6e2316f3452 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:02.296589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff8e84-e127-4e56-bf66-aa8d73c8f652 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:02.333267 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f5eb5-a763-4c7d-99d1-59f7ebaefa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:02.333943 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f5eb5-a763-4c7d-99d1-59f7ebaefa96 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:02.334231 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f5eb5-a763-4c7d-99d1-59f7ebaefa96 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:02.334510 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f5eb5-a763-4c7d-99d1-59f7ebaefa96 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:02.335227 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f5eb5-a763-4c7d-99d1-59f7ebaefa96 None admin] RBAC: Adding query filter params (name=image) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:02.335541 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f5eb5-a763-4c7d-99d1-59f7ebaefa96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:02.336970 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f5eb5-a763-4c7d-99d1-59f7ebaefa96 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:02.337648 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa57383-5400-4f48-878f-d6e2316f3452 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:02.340480 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:02.342836 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 518/1031] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:02.349655 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f5eb5-a763-4c7d-99d1-59f7ebaefa96 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:02.370801 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:02.377543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71993a-5e88-4e85-bd99-298337b5dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:02.378543 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71993a-5e88-4e85-bd99-298337b5dcec None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:02.378826 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71993a-5e88-4e85-bd99-298337b5dcec None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:02.381703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71993a-5e88-4e85-bd99-298337b5dcec None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:02.381703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71993a-5e88-4e85-bd99-298337b5dcec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:02.381703 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71993a-5e88-4e85-bd99-298337b5dcec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:02.382634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71993a-5e88-4e85-bd99-298337b5dcec None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:02.392433 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 514/1032] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:30:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:02.398661 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71993a-5e88-4e85-bd99-298337b5dcec None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:02.477123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f5eb5-a763-4c7d-99d1-59f7ebaefa96 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:02.513880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bdbea-5032-44e8-95b0-08262012fbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:02.514560 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bdbea-5032-44e8-95b0-08262012fbb6 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:02.514849 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bdbea-5032-44e8-95b0-08262012fbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:02.515646 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bdbea-5032-44e8-95b0-08262012fbb6 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:02.516405 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bdbea-5032-44e8-95b0-08262012fbb6 None admin] RBAC: Adding query filter params (type=image) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:02.516724 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bdbea-5032-44e8-95b0-08262012fbb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:02.518106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bdbea-5032-44e8-95b0-08262012fbb6 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:02.521388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71993a-5e88-4e85-bd99-298337b5dcec None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:02.523287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 519/1033] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:02.529124 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bdbea-5032-44e8-95b0-08262012fbb6 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:02.565289 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775671e-4776-4880-9a7b-52e209e7521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:02.566096 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775671e-4776-4880-9a7b-52e209e7521b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:02.566387 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775671e-4776-4880-9a7b-52e209e7521b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:02.566662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775671e-4776-4880-9a7b-52e209e7521b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:02.579324 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775671e-4776-4880-9a7b-52e209e7521b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:02.580851 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775671e-4776-4880-9a7b-52e209e7521b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-1775671e-4776-4880-9a7b-52e209e7521b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:30:02.599094 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:02.600170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:02.601720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:30:02.601720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:30:02.601720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:02.601720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:02.601720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 515/1034] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:30:02.605899 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775671e-4776-4880-9a7b-52e209e7521b None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:02.622327 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:02.629700 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:02.643000 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bdbea-5032-44e8-95b0-08262012fbb6 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:02.677722 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab06348-78f8-419b-b10f-cb53b53f00b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:02.678397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab06348-78f8-419b-b10f-cb53b53f00b1 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:02.678683 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab06348-78f8-419b-b10f-cb53b53f00b1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:02.678961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab06348-78f8-419b-b10f-cb53b53f00b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:02.679764 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab06348-78f8-419b-b10f-cb53b53f00b1 None admin] RBAC: Adding query filter params (interface=public, service_id=5d26407451064c06b6ddc966dbfddf3e, region_id=RegionOne) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:02.680106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab06348-78f8-419b-b10f-cb53b53f00b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:02.681489 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab06348-78f8-419b-b10f-cb53b53f00b1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:02.686625 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 520/1035] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:30:02 2022] GET /identity/v3/endpoints?service_id=5d26407451064c06b6ddc966dbfddf3e&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:02.693166 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab06348-78f8-419b-b10f-cb53b53f00b1 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:02.725414 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775671e-4776-4880-9a7b-52e209e7521b None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:02.765050 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4cee9-448c-428a-ba65-0ea5f5c834ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:02.766304 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4cee9-448c-428a-ba65-0ea5f5c834ad None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:02.766589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4cee9-448c-428a-ba65-0ea5f5c834ad None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:02.766870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4cee9-448c-428a-ba65-0ea5f5c834ad None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:02.767605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4cee9-448c-428a-ba65-0ea5f5c834ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:02.767917 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4cee9-448c-428a-ba65-0ea5f5c834ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:02.769390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4cee9-448c-428a-ba65-0ea5f5c834ad None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:02.781706 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 516/1036] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:02.787102 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4cee9-448c-428a-ba65-0ea5f5c834ad None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:02.814699 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab06348-78f8-419b-b10f-cb53b53f00b1 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:02.851599 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d4fa4-737c-43c3-9f47-6acdec99a2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:02.852278 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d4fa4-737c-43c3-9f47-6acdec99a2d3 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:02.852588 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d4fa4-737c-43c3-9f47-6acdec99a2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:02.852879 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d4fa4-737c-43c3-9f47-6acdec99a2d3 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:02.853623 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d4fa4-737c-43c3-9f47-6acdec99a2d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:02.854989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d4fa4-737c-43c3-9f47-6acdec99a2d3 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:02.872471 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 521/1037] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:30:02 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 20:30:02.890935 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:02.914606 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4cee9-448c-428a-ba65-0ea5f5c834ad None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:02.958989 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650139b-ba7f-446e-8c6b-3d7ac9d42461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:02.959826 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650139b-ba7f-446e-8c6b-3d7ac9d42461 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:02.960114 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650139b-ba7f-446e-8c6b-3d7ac9d42461 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:02.960425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650139b-ba7f-446e-8c6b-3d7ac9d42461 None admin] PATH_INFO: `/v3/users/nova` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:02.969617 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650139b-ba7f-446e-8c6b-3d7ac9d42461 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:02.971222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650139b-ba7f-446e-8c6b-3d7ac9d42461 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-2650139b-ba7f-446e-8c6b-3d7ac9d42461 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:30:02.987448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:02.988749 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:02.996542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:02.996542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:02.996542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:30:02.996542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:30:02.996542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:30:02.996542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:30:02.996542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:30:02.996542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:30:02.996542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:30:02.996542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:02.996542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 517/1038] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:30:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:30:02.997267 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d4fa4-737c-43c3-9f47-6acdec99a2d3 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:03.009616 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:03.121994 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d4fa4-737c-43c3-9f47-6acdec99a2d3 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:03.162039 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d592bdb-56b2-4d8e-8a13-0475bdd6079e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:03.162657 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592bdb-56b2-4d8e-8a13-0475bdd6079e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:03.162941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592bdb-56b2-4d8e-8a13-0475bdd6079e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:03.163215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592bdb-56b2-4d8e-8a13-0475bdd6079e None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:03.164068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d592bdb-56b2-4d8e-8a13-0475bdd6079e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:03.164401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d592bdb-56b2-4d8e-8a13-0475bdd6079e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:03.166106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d592bdb-56b2-4d8e-8a13-0475bdd6079e None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:03.188693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 522/1039] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:30:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:03.194599 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650139b-ba7f-446e-8c6b-3d7ac9d42461 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:03.299389 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650139b-ba7f-446e-8c6b-3d7ac9d42461 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:03.328746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf60b9-2809-40b1-b7ba-21fdc4ac2f84 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:03.329035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf60b9-2809-40b1-b7ba-21fdc4ac2f84 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:03.329316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf60b9-2809-40b1-b7ba-21fdc4ac2f84 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:03.330699 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 523/1040] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:03.335388 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6040c-f4fe-4438-8f6b-e4202fcc4e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:03.335796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482fa9a-6f05-470c-ab5f-8cee022278be None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:03.336082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482fa9a-6f05-470c-ab5f-8cee022278be None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:03.336383 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482fa9a-6f05-470c-ab5f-8cee022278be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:03.336881 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6040c-f4fe-4438-8f6b-e4202fcc4e36 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:03.337320 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6040c-f4fe-4438-8f6b-e4202fcc4e36 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:03.337757 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6040c-f4fe-4438-8f6b-e4202fcc4e36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:03.347622 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:03.353493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-3482fa9a-6f05-470c-ab5f-8cee022278be None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:03.354672 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-3482fa9a-6f05-470c-ab5f-8cee022278be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:03.414600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6040c-f4fe-4438-8f6b-e4202fcc4e36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:03.417077 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb6040c-f4fe-4438-8f6b-e4202fcc4e36 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:03.420635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:03.429464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 524/1041] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:30:03.456266 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482fa9a-6f05-470c-ab5f-8cee022278be None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:03.528120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 518/1042] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:30:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:30:03.534567 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6040c-f4fe-4438-8f6b-e4202fcc4e36 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:03.566329 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482fa9a-6f05-470c-ab5f-8cee022278be None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:03.578674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:03.606050 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45feb21-d1c7-4640-bd9f-b9601c059d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:03.607112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45feb21-d1c7-4640-bd9f-b9601c059d18 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:03.607553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45feb21-d1c7-4640-bd9f-b9601c059d18 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:03.608023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45feb21-d1c7-4640-bd9f-b9601c059d18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:03.619955 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45feb21-d1c7-4640-bd9f-b9601c059d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-a45feb21-d1c7-4640-bd9f-b9601c059d18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:30:03.627940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:03.629208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:03.630353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:03.633149 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 525/1043] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:30:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:30:03.639726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45feb21-d1c7-4640-bd9f-b9601c059d18 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:03.645533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6040c-f4fe-4438-8f6b-e4202fcc4e36 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:03.687492 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6270a-5aad-4f29-9b42-97e3fa04ae74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:03.688677 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6270a-5aad-4f29-9b42-97e3fa04ae74 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:03.688961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6270a-5aad-4f29-9b42-97e3fa04ae74 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:03.689235 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6270a-5aad-4f29-9b42-97e3fa04ae74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:03.700872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6270a-5aad-4f29-9b42-97e3fa04ae74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:03.702404 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6270a-5aad-4f29-9b42-97e3fa04ae74 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:03.705872 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-99a6270a-5aad-4f29-9b42-97e3fa04ae74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:30:03.723799 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:03.731729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:03.732822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:30:03.732822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:30:03.732822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:03.732822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:03.732822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 519/1044] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:03.732822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6270a-5aad-4f29-9b42-97e3fa04ae74 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:03.751885 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45feb21-d1c7-4640-bd9f-b9601c059d18 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:03.789190 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4ef33-9557-45af-8f24-ad814f451093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:03.790037 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4ef33-9557-45af-8f24-ad814f451093 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:03.790466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4ef33-9557-45af-8f24-ad814f451093 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:03.791101 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4ef33-9557-45af-8f24-ad814f451093 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:03.792019 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4ef33-9557-45af-8f24-ad814f451093 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:03.792358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4ef33-9557-45af-8f24-ad814f451093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:03.793759 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4ef33-9557-45af-8f24-ad814f451093 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:03.804362 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 526/1045] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:30:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:03.809556 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4ef33-9557-45af-8f24-ad814f451093 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:03.839936 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:03.849871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6270a-5aad-4f29-9b42-97e3fa04ae74 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:03.890886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0569d2d-df71-41ae-bb82-234c6409270f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:03.891556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0569d2d-df71-41ae-bb82-234c6409270f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:03.892105 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0569d2d-df71-41ae-bb82-234c6409270f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:03.892105 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0569d2d-df71-41ae-bb82-234c6409270f None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:03.892828 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0569d2d-df71-41ae-bb82-234c6409270f None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:03.893131 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0569d2d-df71-41ae-bb82-234c6409270f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:03.894497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0569d2d-df71-41ae-bb82-234c6409270f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:03.906612 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 520/1046] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:03.912068 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0569d2d-df71-41ae-bb82-234c6409270f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:03.926119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4ef33-9557-45af-8f24-ad814f451093 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:03.946312 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:03.964198 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c4d01-4fd4-4be6-9690-366a7586058f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:03.965107 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c4d01-4fd4-4be6-9690-366a7586058f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:03.965403 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c4d01-4fd4-4be6-9690-366a7586058f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:03.965683 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c4d01-4fd4-4be6-9690-366a7586058f None admin] PATH_INFO: `/v3/users/58d5579030044f1e91dab8efdca5f951` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:03.979389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c4d01-4fd4-4be6-9690-366a7586058f None admin] RBAC: Authorizing `identity:get_user(user_id=58d5579030044f1e91dab8efdca5f951)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:03.980864 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c4d01-4fd4-4be6-9690-366a7586058f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:03.984379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 527/1047] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:03 2022] GET /identity/v3/users/58d5579030044f1e91dab8efdca5f951 => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:03.988734 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c4d01-4fd4-4be6-9690-366a7586058f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:04.031082 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0569d2d-df71-41ae-bb82-234c6409270f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:04.071750 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50391173-e18a-4d72-8609-e2c9a433fbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:04.072661 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391173-e18a-4d72-8609-e2c9a433fbb7 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:04.072951 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391173-e18a-4d72-8609-e2c9a433fbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:04.073238 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50391173-e18a-4d72-8609-e2c9a433fbb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:04.080482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50391173-e18a-4d72-8609-e2c9a433fbb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:04.081931 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50391173-e18a-4d72-8609-e2c9a433fbb7 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:04.096557 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c4d01-4fd4-4be6-9690-366a7586058f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-50391173-e18a-4d72-8609-e2c9a433fbb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:30:04.097123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:04.098086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:04.099088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:30:04.099088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:30:04.099088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:30:04.099088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:30:04.099088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:30:04.099088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:30:04.099088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:30:04.099088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:04.099088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 521/1048] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:30:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:04.104232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50391173-e18a-4d72-8609-e2c9a433fbb7 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:04.118925 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:04.140185 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303414e-61dc-41d7-a0b6-8c14c6461805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:04.140899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303414e-61dc-41d7-a0b6-8c14c6461805 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:04.141286 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303414e-61dc-41d7-a0b6-8c14c6461805 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:04.141523 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303414e-61dc-41d7-a0b6-8c14c6461805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:04.190529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:04.213922 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303414e-61dc-41d7-a0b6-8c14c6461805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:04.215286 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303414e-61dc-41d7-a0b6-8c14c6461805 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:04.221204 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50391173-e18a-4d72-8609-e2c9a433fbb7 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:04.256886 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da59c0-74f7-4885-8bd5-6d4bf38465bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:04.257549 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da59c0-74f7-4885-8bd5-6d4bf38465bf None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:04.257832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da59c0-74f7-4885-8bd5-6d4bf38465bf None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:04.258109 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da59c0-74f7-4885-8bd5-6d4bf38465bf None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:04.258898 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da59c0-74f7-4885-8bd5-6d4bf38465bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:04.259208 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da59c0-74f7-4885-8bd5-6d4bf38465bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:04.260664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da59c0-74f7-4885-8bd5-6d4bf38465bf None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:04.273158 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 522/1049] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:30:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:04.278809 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da59c0-74f7-4885-8bd5-6d4bf38465bf None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:04.326049 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 528/1050] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:30:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:30:04.331874 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303414e-61dc-41d7-a0b6-8c14c6461805 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:04.354880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:04.389317 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da59c0-74f7-4885-8bd5-6d4bf38465bf None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:04.415737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:04.428954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd11c5-3797-43d3-8338-a43a70dc8842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:04.429705 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd11c5-3797-43d3-8338-a43a70dc8842 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:04.429927 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd11c5-3797-43d3-8338-a43a70dc8842 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:04.430170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd11c5-3797-43d3-8338-a43a70dc8842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:04.431014 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd11c5-3797-43d3-8338-a43a70dc8842 None admin] RBAC: Adding query filter params (role.id=bc67bfe0fbca4ce2bbb378ea4a046931, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=e6a982ae15a6443b9b52529df47aa837) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:04.431337 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd11c5-3797-43d3-8338-a43a70dc8842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:04.432817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd11c5-3797-43d3-8338-a43a70dc8842 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:04.438759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 523/1051] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:30:04 2022] GET /identity/v3/role_assignments?user.id=e6a982ae15a6443b9b52529df47aa837&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=bc67bfe0fbca4ce2bbb378ea4a046931 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:04.454492 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303414e-61dc-41d7-a0b6-8c14c6461805 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:04.496762 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a2aa1-0469-4b27-bd1c-16ed8201ddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:04.497566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a2aa1-0469-4b27-bd1c-16ed8201ddfb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:04.497864 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a2aa1-0469-4b27-bd1c-16ed8201ddfb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:04.498145 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a2aa1-0469-4b27-bd1c-16ed8201ddfb None admin] PATH_INFO: `/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:04.499473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a2aa1-0469-4b27-bd1c-16ed8201ddfb None admin] RBAC: Authorizing `identity:get_project(project_id=69a68dc9693b41d5bd21ac323cdbe8a6)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:04.501005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a2aa1-0469-4b27-bd1c-16ed8201ddfb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:04.503155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 529/1052] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:30:04 2022] GET /identity/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:04.509344 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd11c5-3797-43d3-8338-a43a70dc8842 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:04.625193 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd11c5-3797-43d3-8338-a43a70dc8842 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:04.668706 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933682b-fed9-4c3c-9dbe-c36a81186762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:04.669687 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933682b-fed9-4c3c-9dbe-c36a81186762 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:04.669813 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933682b-fed9-4c3c-9dbe-c36a81186762 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:04.670001 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933682b-fed9-4c3c-9dbe-c36a81186762 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:04.670840 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933682b-fed9-4c3c-9dbe-c36a81186762 None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=69a68dc9693b41d5bd21ac323cdbe8a6, user.id=58d5579030044f1e91dab8efdca5f951) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:04.671152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933682b-fed9-4c3c-9dbe-c36a81186762 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:04.672699 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933682b-fed9-4c3c-9dbe-c36a81186762 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:04.674671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:04.686882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 524/1053] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:30:04 2022] GET /identity/v3/role_assignments?user.id=58d5579030044f1e91dab8efdca5f951&scope.project.id=69a68dc9693b41d5bd21ac323cdbe8a6&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:05.181521 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bb257-e3f4-4e15-838c-20654d4f2f8b None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:05.182079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bb257-e3f4-4e15-838c-20654d4f2f8b None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:05.182333 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bb257-e3f4-4e15-838c-20654d4f2f8b None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:05.183792 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 530/1054] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:05.190337 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44e68d-4a85-434d-b5fd-3818017a719f None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:05.190337 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44e68d-4a85-434d-b5fd-3818017a719f None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:05.190337 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f44e68d-4a85-434d-b5fd-3818017a719f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:05.207477 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-2f44e68d-4a85-434d-b5fd-3818017a719f None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:05.208709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-2f44e68d-4a85-434d-b5fd-3818017a719f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:05.282726 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 525/1055] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:05.305806 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bb257-e3f4-4e15-838c-20654d4f2f8b None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:05.410608 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bb257-e3f4-4e15-838c-20654d4f2f8b None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:05.454060 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b964aa0-c7f8-43ee-9358-ed97cd2cc7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:05.455121 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b964aa0-c7f8-43ee-9358-ed97cd2cc7d9 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:05.455571 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b964aa0-c7f8-43ee-9358-ed97cd2cc7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:05.456010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b964aa0-c7f8-43ee-9358-ed97cd2cc7d9 None admin] PATH_INFO: `/v3/services/image` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:05.456857 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b964aa0-c7f8-43ee-9358-ed97cd2cc7d9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:05.458449 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b964aa0-c7f8-43ee-9358-ed97cd2cc7d9 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-5b964aa0-c7f8-43ee-9358-ed97cd2cc7d9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:05.471700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:05.473068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:30:05.474355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:05.476681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 531/1056] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 20:30:05 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:05.482956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44e68d-4a85-434d-b5fd-3818017a719f None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:05.589733 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f44e68d-4a85-434d-b5fd-3818017a719f None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:05.618116 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:05.627162 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a874d-353f-45c0-acff-45ff7a2971dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:05.627938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a874d-353f-45c0-acff-45ff7a2971dd None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:05.628221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a874d-353f-45c0-acff-45ff7a2971dd None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:05.628523 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a874d-353f-45c0-acff-45ff7a2971dd None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:05.635710 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a874d-353f-45c0-acff-45ff7a2971dd None admin] RBAC: Adding query filter params (name=image) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:05.635710 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a874d-353f-45c0-acff-45ff7a2971dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:05.635710 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a874d-353f-45c0-acff-45ff7a2971dd None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:05.641325 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 526/1057] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:05.642583 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b964aa0-c7f8-43ee-9358-ed97cd2cc7d9 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:05.655008 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:05.748080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b964aa0-c7f8-43ee-9358-ed97cd2cc7d9 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:05.782766 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c84b72-5bcf-46a0-b5e6-bc5795c7c82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:05.783460 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84b72-5bcf-46a0-b5e6-bc5795c7c82a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:05.783744 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84b72-5bcf-46a0-b5e6-bc5795c7c82a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:05.784013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c84b72-5bcf-46a0-b5e6-bc5795c7c82a None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:05.784783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c84b72-5bcf-46a0-b5e6-bc5795c7c82a None admin] RBAC: Adding query filter params (type=image) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:05.785093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c84b72-5bcf-46a0-b5e6-bc5795c7c82a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:05.786497 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c84b72-5bcf-46a0-b5e6-bc5795c7c82a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:05.793127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 532/1058] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:05.799076 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a874d-353f-45c0-acff-45ff7a2971dd None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:05.904093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a874d-353f-45c0-acff-45ff7a2971dd None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:05.939278 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682dc75-8b9a-4850-9042-30647f1685b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:05.939944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682dc75-8b9a-4850-9042-30647f1685b1 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:05.940224 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682dc75-8b9a-4850-9042-30647f1685b1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:05.940522 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682dc75-8b9a-4850-9042-30647f1685b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:05.941259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682dc75-8b9a-4850-9042-30647f1685b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:05.942625 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682dc75-8b9a-4850-9042-30647f1685b1 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:06.026009 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 527/1059] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:30:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:30:06.688979 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021c3a-45e8-4b4c-aead-54bddb7e7188 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:06.689553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021c3a-45e8-4b4c-aead-54bddb7e7188 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:06.689683 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb021c3a-45e8-4b4c-aead-54bddb7e7188 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:06.691164 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 533/1060] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:06.696484 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68594e8d-8421-4c49-9d10-029bcd10c341 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:06.696914 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68594e8d-8421-4c49-9d10-029bcd10c341 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:06.697095 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68594e8d-8421-4c49-9d10-029bcd10c341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:06.714877 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-68594e8d-8421-4c49-9d10-029bcd10c341 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:06.716073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-68594e8d-8421-4c49-9d10-029bcd10c341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:06.766276 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:06.799042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 528/1061] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:06.822012 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb021c3a-45e8-4b4c-aead-54bddb7e7188 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:06.898999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:06.922232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0c5a3-e1ae-494e-a400-bc66748eb0f6 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:06.922532 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0c5a3-e1ae-494e-a400-bc66748eb0f6 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:06.922807 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0c5a3-e1ae-494e-a400-bc66748eb0f6 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:06.924353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 529/1062] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:06.929161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204be546-e6e6-4020-af62-c9740a575d55 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:06.929463 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204be546-e6e6-4020-af62-c9740a575d55 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:06.929742 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204be546-e6e6-4020-af62-c9740a575d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:06.932504 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb021c3a-45e8-4b4c-aead-54bddb7e7188 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:06.947469 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-204be546-e6e6-4020-af62-c9740a575d55 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:06.949021 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-204be546-e6e6-4020-af62-c9740a575d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:06.970615 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951a0d0-c3a9-43da-b8f1-180c91595650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:06.971654 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951a0d0-c3a9-43da-b8f1-180c91595650 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:06.971654 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951a0d0-c3a9-43da-b8f1-180c91595650 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:06.971979 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951a0d0-c3a9-43da-b8f1-180c91595650 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:06.983060 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5951a0d0-c3a9-43da-b8f1-180c91595650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-5951a0d0-c3a9-43da-b8f1-180c91595650 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:30:06.991755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:06.992961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:30:06.993954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:06.996764 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 534/1063] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:30:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:06.999015 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951a0d0-c3a9-43da-b8f1-180c91595650 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:07.036846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 530/1064] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:30:07.047481 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204be546-e6e6-4020-af62-c9740a575d55 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:07.108301 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951a0d0-c3a9-43da-b8f1-180c91595650 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:07.140218 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:07.146254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806ee4a-9fe6-4874-9a52-17cf670bfd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:07.146937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806ee4a-9fe6-4874-9a52-17cf670bfd6c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:07.147224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806ee4a-9fe6-4874-9a52-17cf670bfd6c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:07.147508 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806ee4a-9fe6-4874-9a52-17cf670bfd6c None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:07.148250 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806ee4a-9fe6-4874-9a52-17cf670bfd6c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:07.148700 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806ee4a-9fe6-4874-9a52-17cf670bfd6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:07.150171 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806ee4a-9fe6-4874-9a52-17cf670bfd6c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:07.164383 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 535/1065] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:30:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:07.166468 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806ee4a-9fe6-4874-9a52-17cf670bfd6c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:07.176009 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204be546-e6e6-4020-af62-c9740a575d55 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:07.178733 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:07.213954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6a2b1-c436-436d-83c1-15186f4bc47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:07.214764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6a2b1-c436-436d-83c1-15186f4bc47e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:07.215051 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6a2b1-c436-436d-83c1-15186f4bc47e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:07.215331 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6a2b1-c436-436d-83c1-15186f4bc47e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:07.227633 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6a2b1-c436-436d-83c1-15186f4bc47e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-e6c6a2b1-c436-436d-83c1-15186f4bc47e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:30:07.235775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:07.236879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:07.237816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 531/1066] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:30:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:30:07.243101 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6a2b1-c436-436d-83c1-15186f4bc47e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:07.283081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806ee4a-9fe6-4874-9a52-17cf670bfd6c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:07.324183 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a50c0-fe99-4f89-b9df-f0557386c56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:07.325006 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a50c0-fe99-4f89-b9df-f0557386c56e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:07.325285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a50c0-fe99-4f89-b9df-f0557386c56e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:07.325553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a50c0-fe99-4f89-b9df-f0557386c56e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:07.337988 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a50c0-fe99-4f89-b9df-f0557386c56e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:07.340090 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a50c0-fe99-4f89-b9df-f0557386c56e None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:07.357557 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6a2b1-c436-436d-83c1-15186f4bc47e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-c74a50c0-fe99-4f89-b9df-f0557386c56e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:30:07.360661 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:07.361773 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:07.362756 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:30:07.362756 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:30:07.362756 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:07.362756 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:07.362756 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 536/1067] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:30:07.367350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a50c0-fe99-4f89-b9df-f0557386c56e None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:07.400050 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5e3a0-ea33-4279-88ce-0716d05c1cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:07.400750 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5e3a0-ea33-4279-88ce-0716d05c1cb0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:07.401034 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5e3a0-ea33-4279-88ce-0716d05c1cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:07.401310 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5e3a0-ea33-4279-88ce-0716d05c1cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:07.402178 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5e3a0-ea33-4279-88ce-0716d05c1cb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:07.403048 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5e3a0-ea33-4279-88ce-0716d05c1cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:07.404060 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb5e3a0-ea33-4279-88ce-0716d05c1cb0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:07.414101 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 532/1068] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:30:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:07.419331 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:07.420608 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5e3a0-ea33-4279-88ce-0716d05c1cb0 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:07.458238 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:07.490890 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a50c0-fe99-4f89-b9df-f0557386c56e None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:07.529305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961837e-7c43-4001-ac73-97cdbb62ec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:07.530261 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961837e-7c43-4001-ac73-97cdbb62ec8a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:07.531378 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961837e-7c43-4001-ac73-97cdbb62ec8a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:07.531657 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961837e-7c43-4001-ac73-97cdbb62ec8a None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:07.532410 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961837e-7c43-4001-ac73-97cdbb62ec8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:07.532812 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961837e-7c43-4001-ac73-97cdbb62ec8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:07.534061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961837e-7c43-4001-ac73-97cdbb62ec8a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:07.534900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5e3a0-ea33-4279-88ce-0716d05c1cb0 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:07.552563 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 537/1069] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:07.553603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961837e-7c43-4001-ac73-97cdbb62ec8a None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:07.575299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78324ae4-cfc4-48de-9bd4-ba9aa624d849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:07.576166 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78324ae4-cfc4-48de-9bd4-ba9aa624d849 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:07.576482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78324ae4-cfc4-48de-9bd4-ba9aa624d849 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:07.576758 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78324ae4-cfc4-48de-9bd4-ba9aa624d849 None admin] PATH_INFO: `/v3/users/58d5579030044f1e91dab8efdca5f951` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:07.578496 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78324ae4-cfc4-48de-9bd4-ba9aa624d849 None admin] RBAC: Authorizing `identity:get_user(user_id=58d5579030044f1e91dab8efdca5f951)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:07.579926 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78324ae4-cfc4-48de-9bd4-ba9aa624d849 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:07.586493 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 533/1070] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:07 2022] GET /identity/v3/users/58d5579030044f1e91dab8efdca5f951 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:07.587360 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78324ae4-cfc4-48de-9bd4-ba9aa624d849 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:07.668091 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961837e-7c43-4001-ac73-97cdbb62ec8a None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:07.676380 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:07.690636 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:07.699804 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78324ae4-cfc4-48de-9bd4-ba9aa624d849 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:07.707464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402064b0-72cb-4c3d-bf33-e9fb1dbc0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:07.708268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402064b0-72cb-4c3d-bf33-e9fb1dbc0950 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:07.708894 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402064b0-72cb-4c3d-bf33-e9fb1dbc0950 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:07.708894 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402064b0-72cb-4c3d-bf33-e9fb1dbc0950 None admin] PATH_INFO: `/v3/users/nova` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:07.717028 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402064b0-72cb-4c3d-bf33-e9fb1dbc0950 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:07.719311 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402064b0-72cb-4c3d-bf33-e9fb1dbc0950 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-402064b0-72cb-4c3d-bf33-e9fb1dbc0950 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:30:07.734733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:07.741947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:07.743024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:07.743024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:07.743024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:30:07.743024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:30:07.743024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:30:07.743024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:30:07.743024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:30:07.743024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:30:07.743024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:30:07.743024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:07.743024 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 538/1071] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:30:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:07.743676 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9092b-7641-428f-8b9a-9b78b5610ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:07.743676 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9092b-7641-428f-8b9a-9b78b5610ff8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:07.743676 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9092b-7641-428f-8b9a-9b78b5610ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:07.743676 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9092b-7641-428f-8b9a-9b78b5610ff8 None admin] PATH_INFO: `/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:07.743985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9092b-7641-428f-8b9a-9b78b5610ff8 None admin] RBAC: Authorizing `identity:get_project(project_id=69a68dc9693b41d5bd21ac323cdbe8a6)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:07.743985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9092b-7641-428f-8b9a-9b78b5610ff8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:07.744127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402064b0-72cb-4c3d-bf33-e9fb1dbc0950 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:07.744207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 534/1072] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:30:07 2022] GET /identity/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:07.750345 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9092b-7641-428f-8b9a-9b78b5610ff8 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:07.862247 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402064b0-72cb-4c3d-bf33-e9fb1dbc0950 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:07.869195 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9092b-7641-428f-8b9a-9b78b5610ff8 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:07.900090 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901794f3-a947-42f4-88e5-67e196d834d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:07.900912 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901794f3-a947-42f4-88e5-67e196d834d0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:07.901319 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901794f3-a947-42f4-88e5-67e196d834d0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:07.901601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901794f3-a947-42f4-88e5-67e196d834d0 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:07.902470 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901794f3-a947-42f4-88e5-67e196d834d0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:07.902786 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901794f3-a947-42f4-88e5-67e196d834d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:07.904213 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901794f3-a947-42f4-88e5-67e196d834d0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:07.906630 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b28a91-9a78-4e46-bf76-a56b14bcbc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:07.907529 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b28a91-9a78-4e46-bf76-a56b14bcbc59 None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:07.907812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b28a91-9a78-4e46-bf76-a56b14bcbc59 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:07.908086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b28a91-9a78-4e46-bf76-a56b14bcbc59 None admin] PATH_INFO: `/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6/users/58d5579030044f1e91dab8efdca5f951/roles/8eaff9cd613a49b29548e90d7e3e9215` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:07.910104 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b28a91-9a78-4e46-bf76-a56b14bcbc59 None admin] RBAC: Authorizing `identity:create_grant(project_id=69a68dc9693b41d5bd21ac323cdbe8a6, user_id=58d5579030044f1e91dab8efdca5f951, role_id=8eaff9cd613a49b29548e90d7e3e9215)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:07.911540 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b28a91-9a78-4e46-bf76-a56b14bcbc59 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:07.924174 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 539/1073] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:30:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:07.929806 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901794f3-a947-42f4-88e5-67e196d834d0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:07.955245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:08.019438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 535/1074] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:30:07 2022] PUT /identity/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6/users/58d5579030044f1e91dab8efdca5f951/roles/8eaff9cd613a49b29548e90d7e3e9215 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:30:08.032717 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901794f3-a947-42f4-88e5-67e196d834d0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:08.073841 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06333690-c7c6-42f8-9b21-bfa5e813fadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:08.074675 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06333690-c7c6-42f8-9b21-bfa5e813fadf None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:08.074961 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06333690-c7c6-42f8-9b21-bfa5e813fadf None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:08.075234 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06333690-c7c6-42f8-9b21-bfa5e813fadf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:08.087788 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06333690-c7c6-42f8-9b21-bfa5e813fadf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:08.089296 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06333690-c7c6-42f8-9b21-bfa5e813fadf None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-06333690-c7c6-42f8-9b21-bfa5e813fadf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:30:08.109187 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:08.110359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:08.117061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:30:08.117061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:30:08.117061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:08.117061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:08.117061 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 540/1075] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:30:08.118754 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b28a91-9a78-4e46-bf76-a56b14bcbc59 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:08.149232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:08.238114 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b28a91-9a78-4e46-bf76-a56b14bcbc59 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:08.275988 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26331e-95d9-48a4-a337-c99d178384c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:08.276616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26331e-95d9-48a4-a337-c99d178384c3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:08.276903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26331e-95d9-48a4-a337-c99d178384c3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:08.277179 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26331e-95d9-48a4-a337-c99d178384c3 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:08.277907 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26331e-95d9-48a4-a337-c99d178384c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:08.278262 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26331e-95d9-48a4-a337-c99d178384c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:08.279699 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26331e-95d9-48a4-a337-c99d178384c3 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:08.292430 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 536/1076] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:08.298283 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06333690-c7c6-42f8-9b21-bfa5e813fadf None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:08.307043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d202d7-a707-4f2f-8ebb-40473df52195 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:08.307339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d202d7-a707-4f2f-8ebb-40473df52195 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:08.307623 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d202d7-a707-4f2f-8ebb-40473df52195 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:08.309186 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 537/1077] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:08.314748 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f2854-b393-4423-b903-9be0c42adc15 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:08.315044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f2854-b393-4423-b903-9be0c42adc15 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:08.315357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f2854-b393-4423-b903-9be0c42adc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:08.332121 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-6d5f2854-b393-4423-b903-9be0c42adc15 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:08.333352 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-6d5f2854-b393-4423-b903-9be0c42adc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:08.387157 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:08.410655 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06333690-c7c6-42f8-9b21-bfa5e813fadf None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:08.416622 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 538/1078] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:30:08.430166 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f2854-b393-4423-b903-9be0c42adc15 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:08.447900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:08.455758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40596a98-35e8-4acf-9270-6cdc9f3a439d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:08.456604 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596a98-35e8-4acf-9270-6cdc9f3a439d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:08.456889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596a98-35e8-4acf-9270-6cdc9f3a439d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:08.457161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596a98-35e8-4acf-9270-6cdc9f3a439d None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:08.464276 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40596a98-35e8-4acf-9270-6cdc9f3a439d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:08.467156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40596a98-35e8-4acf-9270-6cdc9f3a439d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-40596a98-35e8-4acf-9270-6cdc9f3a439d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:30:08.480252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:08.481350 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:08.487907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:30:08.487907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:30:08.487907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:30:08.487907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:30:08.487907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:30:08.487907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:30:08.487907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:30:08.487907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:08.487907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 541/1079] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:30:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:08.487907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40596a98-35e8-4acf-9270-6cdc9f3a439d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:08.560513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f2854-b393-4423-b903-9be0c42adc15 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:08.598176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40596a98-35e8-4acf-9270-6cdc9f3a439d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:08.602682 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0193b-13fd-4123-b540-35c62ddf4abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:08.603356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0193b-13fd-4123-b540-35c62ddf4abc None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:08.603643 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0193b-13fd-4123-b540-35c62ddf4abc None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:08.603950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0193b-13fd-4123-b540-35c62ddf4abc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:08.636452 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727fd45-661a-4872-9aef-d70a74c2551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:08.637183 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727fd45-661a-4872-9aef-d70a74c2551c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:08.637494 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727fd45-661a-4872-9aef-d70a74c2551c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:08.637787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727fd45-661a-4872-9aef-d70a74c2551c None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:08.638620 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727fd45-661a-4872-9aef-d70a74c2551c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:08.638929 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727fd45-661a-4872-9aef-d70a74c2551c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:08.640437 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727fd45-661a-4872-9aef-d70a74c2551c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:08.643210 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:08.654926 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 542/1080] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:30:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:08.661734 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727fd45-661a-4872-9aef-d70a74c2551c None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:08.689554 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0193b-13fd-4123-b540-35c62ddf4abc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:08.691840 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0193b-13fd-4123-b540-35c62ddf4abc None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:08.708776 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:08.769702 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727fd45-661a-4872-9aef-d70a74c2551c None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:08.808474 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b5355-3390-4147-b551-ffd13380cd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:08.809371 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b5355-3390-4147-b551-ffd13380cd60 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:08.809701 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b5355-3390-4147-b551-ffd13380cd60 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:08.810075 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b5355-3390-4147-b551-ffd13380cd60 None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/e6a982ae15a6443b9b52529df47aa837/roles/bc67bfe0fbca4ce2bbb378ea4a046931` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:08.811744 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 539/1081] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:30:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:30:08.818061 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0193b-13fd-4123-b540-35c62ddf4abc None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:08.818226 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b5355-3390-4147-b551-ffd13380cd60 None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=e6a982ae15a6443b9b52529df47aa837, role_id=bc67bfe0fbca4ce2bbb378ea4a046931)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:08.819501 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b5355-3390-4147-b551-ffd13380cd60 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:08.909303 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 543/1082] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:30:08 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/e6a982ae15a6443b9b52529df47aa837/roles/bc67bfe0fbca4ce2bbb378ea4a046931 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:30:08.923694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0193b-13fd-4123-b540-35c62ddf4abc None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:08.939037 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:08.963020 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708a12a-7d6b-4b7d-8ab4-b14a1ca72140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:08.963842 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708a12a-7d6b-4b7d-8ab4-b14a1ca72140 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:08.964127 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708a12a-7d6b-4b7d-8ab4-b14a1ca72140 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:08.964433 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708a12a-7d6b-4b7d-8ab4-b14a1ca72140 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:08.977292 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708a12a-7d6b-4b7d-8ab4-b14a1ca72140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:08.978807 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708a12a-7d6b-4b7d-8ab4-b14a1ca72140 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-f708a12a-7d6b-4b7d-8ab4-b14a1ca72140 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:30:08.998393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:08.999497 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:09.005814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:30:09.005814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:30:09.005814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:09.005814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:09.005814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 540/1083] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:30:09.008406 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b5355-3390-4147-b551-ffd13380cd60 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:09.071768 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:09.120911 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b5355-3390-4147-b551-ffd13380cd60 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:09.164941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c49c6e3-a7fd-47bf-825f-deb8d7bdbd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:09.165649 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49c6e3-a7fd-47bf-825f-deb8d7bdbd77 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:09.166209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49c6e3-a7fd-47bf-825f-deb8d7bdbd77 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:09.166486 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49c6e3-a7fd-47bf-825f-deb8d7bdbd77 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:09.167224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49c6e3-a7fd-47bf-825f-deb8d7bdbd77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:09.167529 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49c6e3-a7fd-47bf-825f-deb8d7bdbd77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:09.168926 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c49c6e3-a7fd-47bf-825f-deb8d7bdbd77 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:09.188366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 544/1084] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:10.340542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86136cc0-1cb4-438d-a449-ce79b5b4506b None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:10.341119 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86136cc0-1cb4-438d-a449-ce79b5b4506b None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:10.341119 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86136cc0-1cb4-438d-a449-ce79b5b4506b None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:10.342501 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 541/1085] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:10.348556 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f8dea-f634-4630-ad98-6ecd04fd51ec None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:10.349008 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f8dea-f634-4630-ad98-6ecd04fd51ec None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:10.349184 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f8dea-f634-4630-ad98-6ecd04fd51ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:10.366499 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-8c8f8dea-f634-4630-ad98-6ecd04fd51ec None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:10.367684 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-8c8f8dea-f634-4630-ad98-6ecd04fd51ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:10.435046 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 545/1086] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:10.459344 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86136cc0-1cb4-438d-a449-ce79b5b4506b None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:10.564733 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86136cc0-1cb4-438d-a449-ce79b5b4506b None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:10.599965 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4315d1-1717-4605-afdb-c89f912be6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:10.600722 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4315d1-1717-4605-afdb-c89f912be6f0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:10.601005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4315d1-1717-4605-afdb-c89f912be6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:10.601349 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4315d1-1717-4605-afdb-c89f912be6f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:10.612418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4315d1-1717-4605-afdb-c89f912be6f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-2a4315d1-1717-4605-afdb-c89f912be6f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:30:10.619884 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:10.621025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:10.621955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 542/1087] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:30:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:10.627269 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f8dea-f634-4630-ad98-6ecd04fd51ec None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:10.674978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:10.728900 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f8dea-f634-4630-ad98-6ecd04fd51ec None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:10.763856 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d77d18-6431-40ef-b563-a8fa4e809945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:10.764590 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77d18-6431-40ef-b563-a8fa4e809945 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:10.765414 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77d18-6431-40ef-b563-a8fa4e809945 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:10.765689 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d77d18-6431-40ef-b563-a8fa4e809945 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:10.766437 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d77d18-6431-40ef-b563-a8fa4e809945 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:10.766742 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d77d18-6431-40ef-b563-a8fa4e809945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:10.768109 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d77d18-6431-40ef-b563-a8fa4e809945 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:10.776739 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 546/1088] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:30:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:10.783108 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4315d1-1717-4605-afdb-c89f912be6f0 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:10.812744 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:10.888876 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4315d1-1717-4605-afdb-c89f912be6f0 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:10.924050 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef2453-8a0a-4e80-b98b-0a5d7b34adb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:10.925053 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef2453-8a0a-4e80-b98b-0a5d7b34adb8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:10.925337 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef2453-8a0a-4e80-b98b-0a5d7b34adb8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:10.925612 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef2453-8a0a-4e80-b98b-0a5d7b34adb8 None admin] PATH_INFO: `/v3/users/58d5579030044f1e91dab8efdca5f951` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:10.926826 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef2453-8a0a-4e80-b98b-0a5d7b34adb8 None admin] RBAC: Authorizing `identity:get_user(user_id=58d5579030044f1e91dab8efdca5f951)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:10.928285 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef2453-8a0a-4e80-b98b-0a5d7b34adb8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:10.930004 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 543/1089] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:10 2022] GET /identity/v3/users/58d5579030044f1e91dab8efdca5f951 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:10.935713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d77d18-6431-40ef-b563-a8fa4e809945 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:11.041967 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d77d18-6431-40ef-b563-a8fa4e809945 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:11.050968 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:11.075641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff09599-9fa2-4bb1-8f30-b7bb0367bfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:11.076296 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff09599-9fa2-4bb1-8f30-b7bb0367bfbc None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:11.076604 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff09599-9fa2-4bb1-8f30-b7bb0367bfbc None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:11.076876 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff09599-9fa2-4bb1-8f30-b7bb0367bfbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:11.145182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff09599-9fa2-4bb1-8f30-b7bb0367bfbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:11.146602 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff09599-9fa2-4bb1-8f30-b7bb0367bfbc None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:11.179149 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c5798-0622-4c97-bbc6-ec02b3c642d3 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:11.179435 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c5798-0622-4c97-bbc6-ec02b3c642d3 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:11.179713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c5798-0622-4c97-bbc6-ec02b3c642d3 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:11.181191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 544/1090] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:30:11.186473 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ea831-4173-4230-9796-6fc0a65d9ba9 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:11.187287 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ea831-4173-4230-9796-6fc0a65d9ba9 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:11.187566 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ea831-4173-4230-9796-6fc0a65d9ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:11.204361 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-f43ea831-4173-4230-9796-6fc0a65d9ba9 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:11.205553 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-f43ea831-4173-4230-9796-6fc0a65d9ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:11.250926 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 547/1091] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:30:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:30:11.256746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff09599-9fa2-4bb1-8f30-b7bb0367bfbc None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:11.266842 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:11.277227 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 545/1092] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:30:11.302121 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ea831-4173-4230-9796-6fc0a65d9ba9 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:11.326183 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:11.354987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff09599-9fa2-4bb1-8f30-b7bb0367bfbc None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:11.389526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26278c6b-d289-4ada-85d4-9d471f3622f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:11.390418 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26278c6b-d289-4ada-85d4-9d471f3622f7 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:11.390697 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26278c6b-d289-4ada-85d4-9d471f3622f7 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:11.390973 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26278c6b-d289-4ada-85d4-9d471f3622f7 None admin] PATH_INFO: `/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:11.392158 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26278c6b-d289-4ada-85d4-9d471f3622f7 None admin] RBAC: Authorizing `identity:get_project(project_id=69a68dc9693b41d5bd21ac323cdbe8a6)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:11.393608 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26278c6b-d289-4ada-85d4-9d471f3622f7 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:11.399556 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 548/1093] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:30:11 2022] GET /identity/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:11.401370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26278c6b-d289-4ada-85d4-9d471f3622f7 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:11.407557 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ea831-4173-4230-9796-6fc0a65d9ba9 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:11.443559 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25996-8686-4d9a-ad53-1717013732dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:11.444407 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25996-8686-4d9a-ad53-1717013732dd None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:11.444694 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25996-8686-4d9a-ad53-1717013732dd None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:11.444969 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e25996-8686-4d9a-ad53-1717013732dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:11.457777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e25996-8686-4d9a-ad53-1717013732dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c4e25996-8686-4d9a-ad53-1717013732dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:30:11.465983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:11.467093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:11.468103 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 546/1094] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:30:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:30:11.473270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25996-8686-4d9a-ad53-1717013732dd None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:11.487376 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:11.511162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26278c6b-d289-4ada-85d4-9d471f3622f7 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:11.546996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d4545-143d-4796-8857-f9c8267bfd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:11.547653 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d4545-143d-4796-8857-f9c8267bfd1d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:11.547936 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d4545-143d-4796-8857-f9c8267bfd1d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:11.548211 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d4545-143d-4796-8857-f9c8267bfd1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:11.549210 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d4545-143d-4796-8857-f9c8267bfd1d None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=69a68dc9693b41d5bd21ac323cdbe8a6, user.id=58d5579030044f1e91dab8efdca5f951) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:11.549533 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d4545-143d-4796-8857-f9c8267bfd1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:11.550984 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d4545-143d-4796-8857-f9c8267bfd1d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:11.557157 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 549/1095] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:30:11 2022] GET /identity/v3/role_assignments?user.id=58d5579030044f1e91dab8efdca5f951&scope.project.id=69a68dc9693b41d5bd21ac323cdbe8a6&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:11.557856 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:11.584544 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e25996-8686-4d9a-ad53-1717013732dd None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:11.654744 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fca3d8-7043-4622-bd71-5820dee5110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:11.655920 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fca3d8-7043-4622-bd71-5820dee5110c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:11.656199 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fca3d8-7043-4622-bd71-5820dee5110c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:11.656510 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fca3d8-7043-4622-bd71-5820dee5110c None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:11.657282 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fca3d8-7043-4622-bd71-5820dee5110c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:11.657757 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fca3d8-7043-4622-bd71-5820dee5110c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:11.659871 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fca3d8-7043-4622-bd71-5820dee5110c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:11.670013 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 547/1096] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:30:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:11.676648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d4545-143d-4796-8857-f9c8267bfd1d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:11.785879 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d4545-143d-4796-8857-f9c8267bfd1d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:11.821207 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f9cea-e140-44f1-b77e-4d00f05c9149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:11.822001 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f9cea-e140-44f1-b77e-4d00f05c9149 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:11.822282 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f9cea-e140-44f1-b77e-4d00f05c9149 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:11.822554 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f9cea-e140-44f1-b77e-4d00f05c9149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:11.832405 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:11.834763 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f9cea-e140-44f1-b77e-4d00f05c9149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:11.836300 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f9cea-e140-44f1-b77e-4d00f05c9149 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-9e2f9cea-e140-44f1-b77e-4d00f05c9149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:30:11.853826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:11.854996 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:11.856218 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:30:11.856218 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:30:11.856218 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:11.856218 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:11.856745 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 550/1097] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:30:11.861371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fca3d8-7043-4622-bd71-5820dee5110c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:11.963449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fca3d8-7043-4622-bd71-5820dee5110c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:11.997363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15174d6-5ed9-4182-9339-9eaa3660ffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:11.998026 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15174d6-5ed9-4182-9339-9eaa3660ffaa None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:11.998330 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15174d6-5ed9-4182-9339-9eaa3660ffaa None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:11.998601 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15174d6-5ed9-4182-9339-9eaa3660ffaa None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:11.999422 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15174d6-5ed9-4182-9339-9eaa3660ffaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:11.999726 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15174d6-5ed9-4182-9339-9eaa3660ffaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:12.001124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15174d6-5ed9-4182-9339-9eaa3660ffaa None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:12.014051 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 548/1098] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:12.020242 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f9cea-e140-44f1-b77e-4d00f05c9149 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:12.123160 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f9cea-e140-44f1-b77e-4d00f05c9149 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:12.157734 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759252ca-de0e-4ee7-8c57-6fcb735f25a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:12.158523 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759252ca-de0e-4ee7-8c57-6fcb735f25a2 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:12.158805 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759252ca-de0e-4ee7-8c57-6fcb735f25a2 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:12.159078 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759252ca-de0e-4ee7-8c57-6fcb735f25a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:12.167209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759252ca-de0e-4ee7-8c57-6fcb735f25a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:12.168692 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759252ca-de0e-4ee7-8c57-6fcb735f25a2 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-759252ca-de0e-4ee7-8c57-6fcb735f25a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:30:12.182811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:12.184182 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:12.185511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:12.185511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:12.185511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:30:12.185511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:30:12.185511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:30:12.185511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:30:12.185511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:30:12.185511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:30:12.185511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:30:12.185511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:12.188501 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 551/1099] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:30:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:12.189805 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15174d6-5ed9-4182-9339-9eaa3660ffaa None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:12.193750 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:12.291165 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15174d6-5ed9-4182-9339-9eaa3660ffaa None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:12.325239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf28403-d3d0-446f-8a8a-15c92d227b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:12.325981 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf28403-d3d0-446f-8a8a-15c92d227b46 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:12.326271 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf28403-d3d0-446f-8a8a-15c92d227b46 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:12.326551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf28403-d3d0-446f-8a8a-15c92d227b46 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:12.327925 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf28403-d3d0-446f-8a8a-15c92d227b46 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:12.328237 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf28403-d3d0-446f-8a8a-15c92d227b46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:12.329852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf28403-d3d0-446f-8a8a-15c92d227b46 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:12.348097 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 549/1100] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:30:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:12.353744 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759252ca-de0e-4ee7-8c57-6fcb735f25a2 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:12.408493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:12.460155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759252ca-de0e-4ee7-8c57-6fcb735f25a2 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:12.493332 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa6583-4c22-474b-8aa3-03462aa7bbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:12.493977 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa6583-4c22-474b-8aa3-03462aa7bbdc None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:12.494260 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa6583-4c22-474b-8aa3-03462aa7bbdc None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:12.494537 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa6583-4c22-474b-8aa3-03462aa7bbdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:12.562978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa6583-4c22-474b-8aa3-03462aa7bbdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:12.564394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa6583-4c22-474b-8aa3-03462aa7bbdc None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:12.615756 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:12.664074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 552/1101] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:30:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:30:12.669268 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf28403-d3d0-446f-8a8a-15c92d227b46 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:12.760311 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:12.805424 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf28403-d3d0-446f-8a8a-15c92d227b46 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:12.842042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6b2d0-9361-4188-94e3-8a0115ea4d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:12.843698 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6b2d0-9361-4188-94e3-8a0115ea4d8c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:12.844147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6b2d0-9361-4188-94e3-8a0115ea4d8c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:12.844617 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6b2d0-9361-4188-94e3-8a0115ea4d8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:12.856191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6b2d0-9361-4188-94e3-8a0115ea4d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:12.857944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6b2d0-9361-4188-94e3-8a0115ea4d8c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-08f6b2d0-9361-4188-94e3-8a0115ea4d8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:30:12.875957 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:12.877191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:12.878313 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:30:12.878313 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:30:12.878313 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:30:12.878313 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:12.882138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 550/1102] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:12.883008 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa6583-4c22-474b-8aa3-03462aa7bbdc None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:12.998543 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa6583-4c22-474b-8aa3-03462aa7bbdc None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:13.035378 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bf6f0-7263-4dae-ae08-7f422c015e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:13.036450 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bf6f0-7263-4dae-ae08-7f422c015e24 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:13.036935 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bf6f0-7263-4dae-ae08-7f422c015e24 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:13.037400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bf6f0-7263-4dae-ae08-7f422c015e24 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:13.038373 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bf6f0-7263-4dae-ae08-7f422c015e24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:13.038682 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bf6f0-7263-4dae-ae08-7f422c015e24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:13.040039 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bf6f0-7263-4dae-ae08-7f422c015e24 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:13.052040 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 553/1103] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:13.057856 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6b2d0-9361-4188-94e3-8a0115ea4d8c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:13.157440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6b2d0-9361-4188-94e3-8a0115ea4d8c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:13.171505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:13.192651 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1a60f-ee5f-4aac-a18a-396cde915b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:13.193457 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1a60f-ee5f-4aac-a18a-396cde915b34 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:13.193745 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1a60f-ee5f-4aac-a18a-396cde915b34 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:13.194024 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1a60f-ee5f-4aac-a18a-396cde915b34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:13.200900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1a60f-ee5f-4aac-a18a-396cde915b34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:13.203028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1a60f-ee5f-4aac-a18a-396cde915b34 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-70e1a60f-ee5f-4aac-a18a-396cde915b34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:30:13.217786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:13.219002 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:13.220490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:30:13.220490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:30:13.220490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:30:13.220490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:30:13.220490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:30:13.220490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:30:13.220490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:30:13.220490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:13.220490 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 551/1104] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:30:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:13.224839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bf6f0-7263-4dae-ae08-7f422c015e24 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:13.240415 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:13.329520 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bf6f0-7263-4dae-ae08-7f422c015e24 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:13.363617 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31715b56-ce61-43d6-ba2c-5be385c16459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:13.364276 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31715b56-ce61-43d6-ba2c-5be385c16459 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:13.364590 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31715b56-ce61-43d6-ba2c-5be385c16459 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:13.364863 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31715b56-ce61-43d6-ba2c-5be385c16459 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:13.365653 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31715b56-ce61-43d6-ba2c-5be385c16459 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:13.365964 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31715b56-ce61-43d6-ba2c-5be385c16459 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:13.367391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31715b56-ce61-43d6-ba2c-5be385c16459 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:13.379469 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 554/1105] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:30:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:13.385586 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1a60f-ee5f-4aac-a18a-396cde915b34 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:13.492120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1a60f-ee5f-4aac-a18a-396cde915b34 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:13.527915 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81782e8-6145-48e2-bedc-017bf501c7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:13.528614 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81782e8-6145-48e2-bedc-017bf501c7d1 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:13.528903 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81782e8-6145-48e2-bedc-017bf501c7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:13.529208 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81782e8-6145-48e2-bedc-017bf501c7d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:13.530046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81782e8-6145-48e2-bedc-017bf501c7d1 None admin] RBAC: Adding query filter params (role.id=bc67bfe0fbca4ce2bbb378ea4a046931, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=e6a982ae15a6443b9b52529df47aa837) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:13.530394 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81782e8-6145-48e2-bedc-017bf501c7d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:13.531840 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81782e8-6145-48e2-bedc-017bf501c7d1 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:13.538281 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 552/1106] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:30:13 2022] GET /identity/v3/role_assignments?user.id=e6a982ae15a6443b9b52529df47aa837&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=bc67bfe0fbca4ce2bbb378ea4a046931 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:13.845343 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54624-7a07-4774-8ba9-a05447eac19d None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:13.845891 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54624-7a07-4774-8ba9-a05447eac19d None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:13.845891 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54624-7a07-4774-8ba9-a05447eac19d None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:13.847392 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 555/1107] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:13.853151 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759737be-9171-4241-9fb0-8989c9bca567 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:13.853607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759737be-9171-4241-9fb0-8989c9bca567 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:13.853707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759737be-9171-4241-9fb0-8989c9bca567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:13.871574 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-759737be-9171-4241-9fb0-8989c9bca567 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:13.872806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-759737be-9171-4241-9fb0-8989c9bca567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:13.905646 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:13.947364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 553/1108] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:13.972715 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b54624-7a07-4774-8ba9-a05447eac19d None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:14.047772 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:14.080544 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b54624-7a07-4774-8ba9-a05447eac19d None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:14.116177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70b5de-a39d-49db-b021-3c2b128be40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:14.117018 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70b5de-a39d-49db-b021-3c2b128be40e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:14.117302 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70b5de-a39d-49db-b021-3c2b128be40e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:14.117577 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70b5de-a39d-49db-b021-3c2b128be40e None admin] PATH_INFO: `/v3/domains/default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:14.118790 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70b5de-a39d-49db-b021-3c2b128be40e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:14.120206 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70b5de-a39d-49db-b021-3c2b128be40e None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:14.121875 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 556/1109] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:14.127775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759737be-9171-4241-9fb0-8989c9bca567 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:14.230379 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759737be-9171-4241-9fb0-8989c9bca567 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:14.266789 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f193e3e-a80d-43c1-8fc4-99b22f4d6e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:14.267476 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193e3e-a80d-43c1-8fc4-99b22f4d6e2b None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:14.267762 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193e3e-a80d-43c1-8fc4-99b22f4d6e2b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:14.268039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f193e3e-a80d-43c1-8fc4-99b22f4d6e2b None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:14.268809 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f193e3e-a80d-43c1-8fc4-99b22f4d6e2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:14.270255 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f193e3e-a80d-43c1-8fc4-99b22f4d6e2b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:14.381074 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 554/1110] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:30:14 2022] POST /identity/v3/users => generated 286 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:30:16.706060 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48892df9-0c51-4945-82a7-672cbf2bd319 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:16.706569 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48892df9-0c51-4945-82a7-672cbf2bd319 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:16.706569 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48892df9-0c51-4945-82a7-672cbf2bd319 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:16.707968 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 557/1111] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:16.713339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa4500-5665-46d7-b1c7-7f0a6b5cce3a None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:16.713766 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa4500-5665-46d7-b1c7-7f0a6b5cce3a None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:16.713880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa4500-5665-46d7-b1c7-7f0a6b5cce3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:16.731347 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-befa4500-5665-46d7-b1c7-7f0a6b5cce3a None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:16.734479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-befa4500-5665-46d7-b1c7-7f0a6b5cce3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:16.749752 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:16.799759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 555/1112] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:16.823231 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48892df9-0c51-4945-82a7-672cbf2bd319 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:16.927788 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48892df9-0c51-4945-82a7-672cbf2bd319 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:16.962709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fc82b-ba07-4877-ae88-4b342feb222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:16.963647 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fc82b-ba07-4877-ae88-4b342feb222f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:16.963932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fc82b-ba07-4877-ae88-4b342feb222f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:16.964208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fc82b-ba07-4877-ae88-4b342feb222f None admin] PATH_INFO: `/v3/roles/ff499a8782db4cce8e162a849be212ee` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:16.965718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fc82b-ba07-4877-ae88-4b342feb222f None admin] RBAC: Authorizing `identity:get_role(role_id=ff499a8782db4cce8e162a849be212ee)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:16.967068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fc82b-ba07-4877-ae88-4b342feb222f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:16.968478 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 558/1113] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:16 2022] GET /identity/v3/roles/ff499a8782db4cce8e162a849be212ee => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:16.974471 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa4500-5665-46d7-b1c7-7f0a6b5cce3a None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:17.074727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa4500-5665-46d7-b1c7-7f0a6b5cce3a None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:17.108594 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62ca21-d41f-4759-922c-de01ed430d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:17.109481 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62ca21-d41f-4759-922c-de01ed430d03 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:17.109765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62ca21-d41f-4759-922c-de01ed430d03 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:17.110042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62ca21-d41f-4759-922c-de01ed430d03 None admin] PATH_INFO: `/v3/users/78640c3ac2f04c799ca4999501bc5457` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:17.123147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62ca21-d41f-4759-922c-de01ed430d03 None admin] RBAC: Authorizing `identity:get_user(user_id=78640c3ac2f04c799ca4999501bc5457)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:17.125130 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62ca21-d41f-4759-922c-de01ed430d03 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:17.126791 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 556/1114] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:16 2022] GET /identity/v3/users/78640c3ac2f04c799ca4999501bc5457 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:17.132872 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fc82b-ba07-4877-ae88-4b342feb222f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:17.144278 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:17.233821 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fc82b-ba07-4877-ae88-4b342feb222f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:17.269065 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e398de-5829-4b26-9067-d8b516831eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:17.269716 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e398de-5829-4b26-9067-d8b516831eee None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:17.269997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e398de-5829-4b26-9067-d8b516831eee None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:17.270268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e398de-5829-4b26-9067-d8b516831eee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:17.337899 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e398de-5829-4b26-9067-d8b516831eee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:17.339326 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e398de-5829-4b26-9067-d8b516831eee None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:17.349343 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:17.435109 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 559/1115] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:30:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:30:17.441269 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62ca21-d41f-4759-922c-de01ed430d03 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:17.456640 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:17.541576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62ca21-d41f-4759-922c-de01ed430d03 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:17.574593 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686dbae-5af2-4bb2-b3ad-02d123603ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:17.575443 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686dbae-5af2-4bb2-b3ad-02d123603ef7 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:17.575729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686dbae-5af2-4bb2-b3ad-02d123603ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:17.576006 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686dbae-5af2-4bb2-b3ad-02d123603ef7 None admin] PATH_INFO: `/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:17.577763 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686dbae-5af2-4bb2-b3ad-02d123603ef7 None admin] RBAC: Authorizing `identity:get_project(project_id=69a68dc9693b41d5bd21ac323cdbe8a6)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:17.579196 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686dbae-5af2-4bb2-b3ad-02d123603ef7 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:17.581023 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 557/1116] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:30:17 2022] GET /identity/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:17.587273 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e398de-5829-4b26-9067-d8b516831eee None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:17.690830 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e398de-5829-4b26-9067-d8b516831eee None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:17.700109 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:17.725033 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8ee1f-6f23-4d32-876f-6341bef9b2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:17.725694 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8ee1f-6f23-4d32-876f-6341bef9b2a3 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:17.725955 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8ee1f-6f23-4d32-876f-6341bef9b2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:17.726232 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8ee1f-6f23-4d32-876f-6341bef9b2a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:17.727068 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8ee1f-6f23-4d32-876f-6341bef9b2a3 None admin] RBAC: Adding query filter params (role.id=ff499a8782db4cce8e162a849be212ee, scope.project.id=69a68dc9693b41d5bd21ac323cdbe8a6, user.id=78640c3ac2f04c799ca4999501bc5457) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:17.727374 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8ee1f-6f23-4d32-876f-6341bef9b2a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:17.728839 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8ee1f-6f23-4d32-876f-6341bef9b2a3 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:17.734575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 560/1117] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:30:17 2022] GET /identity/v3/role_assignments?user.id=78640c3ac2f04c799ca4999501bc5457&scope.project.id=69a68dc9693b41d5bd21ac323cdbe8a6&role.id=ff499a8782db4cce8e162a849be212ee => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:19.978504 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c429b-605e-4def-a27b-bcdafa70b441 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:19.979137 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c429b-605e-4def-a27b-bcdafa70b441 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:19.979137 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c429b-605e-4def-a27b-bcdafa70b441 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:19.980482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 558/1118] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:19.986395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb23788-80f6-42f4-ae59-0bf61ed282ba None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:19.986829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb23788-80f6-42f4-ae59-0bf61ed282ba None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:19.987027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb23788-80f6-42f4-ae59-0bf61ed282ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:20.003511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-3cb23788-80f6-42f4-ae59-0bf61ed282ba None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:20.004806 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-3cb23788-80f6-42f4-ae59-0bf61ed282ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:20.070371 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 561/1119] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:20.093198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c429b-605e-4def-a27b-bcdafa70b441 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:20.176151 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:20.195855 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c429b-605e-4def-a27b-bcdafa70b441 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:20.232814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0ed51-67e7-44ee-9960-397859015f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:20.233647 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0ed51-67e7-44ee-9960-397859015f4f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:20.233930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0ed51-67e7-44ee-9960-397859015f4f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:20.234205 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c0ed51-67e7-44ee-9960-397859015f4f None admin] PATH_INFO: `/v3/roles/ff499a8782db4cce8e162a849be212ee` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:20.235707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0ed51-67e7-44ee-9960-397859015f4f None admin] RBAC: Authorizing `identity:get_role(role_id=ff499a8782db4cce8e162a849be212ee)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:20.237133 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c0ed51-67e7-44ee-9960-397859015f4f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:20.238542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 559/1120] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:20 2022] GET /identity/v3/roles/ff499a8782db4cce8e162a849be212ee => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:20.244423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb23788-80f6-42f4-ae59-0bf61ed282ba None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:20.344579 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb23788-80f6-42f4-ae59-0bf61ed282ba None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:20.346113 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:20.379422 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b7377-9ed2-4900-8101-ba9c6e8a2a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:20.380304 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b7377-9ed2-4900-8101-ba9c6e8a2a0f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:20.380628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b7377-9ed2-4900-8101-ba9c6e8a2a0f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:20.380908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b7377-9ed2-4900-8101-ba9c6e8a2a0f None admin] PATH_INFO: `/v3/users/78640c3ac2f04c799ca4999501bc5457` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:20.382102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b7377-9ed2-4900-8101-ba9c6e8a2a0f None admin] RBAC: Authorizing `identity:get_user(user_id=78640c3ac2f04c799ca4999501bc5457)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:20.383525 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b7377-9ed2-4900-8101-ba9c6e8a2a0f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:20.385209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 562/1121] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:20 2022] GET /identity/v3/users/78640c3ac2f04c799ca4999501bc5457 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:20.391001 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0ed51-67e7-44ee-9960-397859015f4f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:20.491486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c0ed51-67e7-44ee-9960-397859015f4f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:20.525102 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38de9c-eecc-4714-bb22-cadabfffcd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:20.525961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38de9c-eecc-4714-bb22-cadabfffcd4a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:20.526245 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38de9c-eecc-4714-bb22-cadabfffcd4a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:20.526522 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38de9c-eecc-4714-bb22-cadabfffcd4a None admin] PATH_INFO: `/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:20.527714 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38de9c-eecc-4714-bb22-cadabfffcd4a None admin] RBAC: Authorizing `identity:get_project(project_id=69a68dc9693b41d5bd21ac323cdbe8a6)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:20.529176 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38de9c-eecc-4714-bb22-cadabfffcd4a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:20.530902 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 560/1122] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:30:20 2022] GET /identity/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:20.536997 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b7377-9ed2-4900-8101-ba9c6e8a2a0f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:20.636590 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b7377-9ed2-4900-8101-ba9c6e8a2a0f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:20.671567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a4652-cf03-4b61-a51e-ff28c97509d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:20.672512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a4652-cf03-4b61-a51e-ff28c97509d1 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:20.672803 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a4652-cf03-4b61-a51e-ff28c97509d1 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:20.673082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a4652-cf03-4b61-a51e-ff28c97509d1 None admin] PATH_INFO: `/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6/users/78640c3ac2f04c799ca4999501bc5457/roles/ff499a8782db4cce8e162a849be212ee` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:20.675399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a4652-cf03-4b61-a51e-ff28c97509d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=69a68dc9693b41d5bd21ac323cdbe8a6, user_id=78640c3ac2f04c799ca4999501bc5457, role_id=ff499a8782db4cce8e162a849be212ee)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:20.676877 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a4652-cf03-4b61-a51e-ff28c97509d1 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:20.784683 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 563/1123] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:30:20 2022] PUT /identity/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6/users/78640c3ac2f04c799ca4999501bc5457/roles/ff499a8782db4cce8e162a849be212ee => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:30:22.985156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116da091-68c0-4f75-9c4a-68f047f62574 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:22.985725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116da091-68c0-4f75-9c4a-68f047f62574 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:22.985725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116da091-68c0-4f75-9c4a-68f047f62574 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:22.987086 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 561/1124] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:22.992396 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cc14c-e4de-4aa5-ad67-b1066d65408f None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:22.992825 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cc14c-e4de-4aa5-ad67-b1066d65408f None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:22.992963 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cc14c-e4de-4aa5-ad67-b1066d65408f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:23.010095 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-374cc14c-e4de-4aa5-ad67-b1066d65408f None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:23.011279 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-374cc14c-e4de-4aa5-ad67-b1066d65408f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:23.035880 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:23.081659 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 564/1125] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:23.105317 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116da091-68c0-4f75-9c4a-68f047f62574 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:23.125071 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:23.212769 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116da091-68c0-4f75-9c4a-68f047f62574 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:23.246790 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8e618-4c69-444e-8fcd-51c46d00de15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:23.247570 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8e618-4c69-444e-8fcd-51c46d00de15 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:23.247852 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8e618-4c69-444e-8fcd-51c46d00de15 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:23.248125 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8e618-4c69-444e-8fcd-51c46d00de15 None admin] PATH_INFO: `/v3/roles/ff499a8782db4cce8e162a849be212ee` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:23.250267 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8e618-4c69-444e-8fcd-51c46d00de15 None admin] RBAC: Authorizing `identity:get_role(role_id=ff499a8782db4cce8e162a849be212ee)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:23.251643 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8e618-4c69-444e-8fcd-51c46d00de15 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:23.253324 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 562/1126] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:23 2022] GET /identity/v3/roles/ff499a8782db4cce8e162a849be212ee => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:23.259191 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cc14c-e4de-4aa5-ad67-b1066d65408f None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:23.368872 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cc14c-e4de-4aa5-ad67-b1066d65408f None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:23.430802 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615b289-8dfe-463c-ad86-ac1a06c0b28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:23.431696 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615b289-8dfe-463c-ad86-ac1a06c0b28f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:23.431977 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615b289-8dfe-463c-ad86-ac1a06c0b28f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:23.432254 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615b289-8dfe-463c-ad86-ac1a06c0b28f None admin] PATH_INFO: `/v3/users/78640c3ac2f04c799ca4999501bc5457` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:23.433512 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615b289-8dfe-463c-ad86-ac1a06c0b28f None admin] RBAC: Authorizing `identity:get_user(user_id=78640c3ac2f04c799ca4999501bc5457)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:23.435214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615b289-8dfe-463c-ad86-ac1a06c0b28f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:23.444499 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 565/1127] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:23 2022] GET /identity/v3/users/78640c3ac2f04c799ca4999501bc5457 => generated 286 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:23.446038 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8e618-4c69-444e-8fcd-51c46d00de15 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:23.541423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:23.557768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8e618-4c69-444e-8fcd-51c46d00de15 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:23.591974 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0b29e-4996-482f-b2c8-891ce8cfb0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:23.593191 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0b29e-4996-482f-b2c8-891ce8cfb0bf None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:23.593479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0b29e-4996-482f-b2c8-891ce8cfb0bf None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:23.593758 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0b29e-4996-482f-b2c8-891ce8cfb0bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:23.664489 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0b29e-4996-482f-b2c8-891ce8cfb0bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:23.666209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f0b29e-4996-482f-b2c8-891ce8cfb0bf None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:23.747816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:23.763611 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 563/1128] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:30:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:30:23.769622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615b289-8dfe-463c-ad86-ac1a06c0b28f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:23.799615 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:23.873814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615b289-8dfe-463c-ad86-ac1a06c0b28f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:23.907079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed50ff-5e31-4544-b03b-e466306b76c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:23.907990 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed50ff-5e31-4544-b03b-e466306b76c5 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:23.908274 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed50ff-5e31-4544-b03b-e466306b76c5 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:23.908576 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed50ff-5e31-4544-b03b-e466306b76c5 None admin] PATH_INFO: `/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:23.909755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed50ff-5e31-4544-b03b-e466306b76c5 None admin] RBAC: Authorizing `identity:get_project(project_id=69a68dc9693b41d5bd21ac323cdbe8a6)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:23.911167 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed50ff-5e31-4544-b03b-e466306b76c5 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:23.912902 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 566/1129] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:30:23 2022] GET /identity/v3/projects/69a68dc9693b41d5bd21ac323cdbe8a6 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:23.919292 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0b29e-4996-482f-b2c8-891ce8cfb0bf None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:24.020698 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0b29e-4996-482f-b2c8-891ce8cfb0bf None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:24.054587 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7b579-ada0-421f-88ed-c2218ba998c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:24.055186 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7b579-ada0-421f-88ed-c2218ba998c2 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:24.056501 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7b579-ada0-421f-88ed-c2218ba998c2 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:24.056501 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7b579-ada0-421f-88ed-c2218ba998c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:24.056713 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7b579-ada0-421f-88ed-c2218ba998c2 None admin] RBAC: Adding query filter params (role.id=ff499a8782db4cce8e162a849be212ee, scope.project.id=69a68dc9693b41d5bd21ac323cdbe8a6, user.id=78640c3ac2f04c799ca4999501bc5457) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:24.056897 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7b579-ada0-421f-88ed-c2218ba998c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:24.058353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7b579-ada0-421f-88ed-c2218ba998c2 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:24.064440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 564/1130] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:30:23 2022] GET /identity/v3/role_assignments?user.id=78640c3ac2f04c799ca4999501bc5457&scope.project.id=69a68dc9693b41d5bd21ac323cdbe8a6&role.id=ff499a8782db4cce8e162a849be212ee => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:26.316407 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3513c0-4886-412a-aac4-fe3bdcd61ba8 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:26.317004 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3513c0-4886-412a-aac4-fe3bdcd61ba8 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:26.317004 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3513c0-4886-412a-aac4-fe3bdcd61ba8 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:26.318363 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 567/1131] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:26.323805 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81607769-2c61-4bc9-ba5b-3b5513cf4314 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:26.324261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81607769-2c61-4bc9-ba5b-3b5513cf4314 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:26.324382 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81607769-2c61-4bc9-ba5b-3b5513cf4314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:26.342321 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-81607769-2c61-4bc9-ba5b-3b5513cf4314 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:26.343768 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-81607769-2c61-4bc9-ba5b-3b5513cf4314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:26.389565 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:26.412978 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 565/1132] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:26.437976 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3513c0-4886-412a-aac4-fe3bdcd61ba8 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:26.532504 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:26.539142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3513c0-4886-412a-aac4-fe3bdcd61ba8 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:26.573637 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fcc2c-075d-4070-b62d-69fbfcb6005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:26.574352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fcc2c-075d-4070-b62d-69fbfcb6005c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:26.574638 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fcc2c-075d-4070-b62d-69fbfcb6005c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:26.574912 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fcc2c-075d-4070-b62d-69fbfcb6005c None admin] PATH_INFO: `/v3/domains/default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:26.576104 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fcc2c-075d-4070-b62d-69fbfcb6005c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:26.577581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fcc2c-075d-4070-b62d-69fbfcb6005c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:26.579208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 568/1133] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:26.585274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81607769-2c61-4bc9-ba5b-3b5513cf4314 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:26.685774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81607769-2c61-4bc9-ba5b-3b5513cf4314 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:26.719377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f237fbc-df0b-4ad7-ac9a-7b99f391ba4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:26.720045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f237fbc-df0b-4ad7-ac9a-7b99f391ba4e None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:26.720351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f237fbc-df0b-4ad7-ac9a-7b99f391ba4e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:26.720634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f237fbc-df0b-4ad7-ac9a-7b99f391ba4e None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:26.721431 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f237fbc-df0b-4ad7-ac9a-7b99f391ba4e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:26.722880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f237fbc-df0b-4ad7-ac9a-7b99f391ba4e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:26.811883 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 566/1134] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:30:26 2022] POST /identity/v3/projects => generated 311 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:30:29.050275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0654c9e-8a4a-47e0-b94d-c5d1a564ba9f None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:29.050828 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0654c9e-8a4a-47e0-b94d-c5d1a564ba9f None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:29.050937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0654c9e-8a4a-47e0-b94d-c5d1a564ba9f None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:29.052437 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 569/1135] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:29.057944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb23f69-d07f-4b21-9e7a-0657ca9c77cb None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:29.058374 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb23f69-d07f-4b21-9e7a-0657ca9c77cb None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:29.058554 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb23f69-d07f-4b21-9e7a-0657ca9c77cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:29.075862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-5cb23f69-d07f-4b21-9e7a-0657ca9c77cb None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:29.077663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-5cb23f69-d07f-4b21-9e7a-0657ca9c77cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:29.101170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:29.144680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 567/1136] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:29.170211 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0654c9e-8a4a-47e0-b94d-c5d1a564ba9f None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:29.273027 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0654c9e-8a4a-47e0-b94d-c5d1a564ba9f None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:29.307954 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547f76e-8a80-493c-86b8-f98fb1a5d4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:29.309292 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547f76e-8a80-493c-86b8-f98fb1a5d4a3 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:29.309573 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547f76e-8a80-493c-86b8-f98fb1a5d4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:29.309872 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547f76e-8a80-493c-86b8-f98fb1a5d4a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:29.311670 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547f76e-8a80-493c-86b8-f98fb1a5d4a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:29.313116 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6547f76e-8a80-493c-86b8-f98fb1a5d4a3 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:29.314742 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 570/1137] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:30:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:29.320829 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb23f69-d07f-4b21-9e7a-0657ca9c77cb None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:29.419507 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb23f69-d07f-4b21-9e7a-0657ca9c77cb None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:29.465064 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090218a5-e9c4-459e-8380-efef289b76e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:29.465746 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090218a5-e9c4-459e-8380-efef289b76e4 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:29.466028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090218a5-e9c4-459e-8380-efef289b76e4 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:29.466303 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090218a5-e9c4-459e-8380-efef289b76e4 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:29.467045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090218a5-e9c4-459e-8380-efef289b76e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:29.468518 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090218a5-e9c4-459e-8380-efef289b76e4 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:29.598353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 568/1138] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:30:29 2022] POST /identity/v3/users => generated 286 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:30:31.882648 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbb2bf-f340-4787-b03d-0dfacd253d5f None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:31.883212 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbb2bf-f340-4787-b03d-0dfacd253d5f None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:31.883212 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbb2bf-f340-4787-b03d-0dfacd253d5f None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:31.884629 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 571/1139] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:31.890611 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ca28c-97cd-497e-bb9f-7af753b2079d None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:31.891056 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ca28c-97cd-497e-bb9f-7af753b2079d None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:31.891161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ca28c-97cd-497e-bb9f-7af753b2079d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:31.908602 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-1e2ca28c-97cd-497e-bb9f-7af753b2079d None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:31.909812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-1e2ca28c-97cd-497e-bb9f-7af753b2079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:31.920900 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:31.976733 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 569/1140] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:30:32.001058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbb2bf-f340-4787-b03d-0dfacd253d5f None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:32.033272 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:32.103070 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbb2bf-f340-4787-b03d-0dfacd253d5f None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:32.136258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f74a4-0efb-4922-bca9-454dc2de4c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:32.137018 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f74a4-0efb-4922-bca9-454dc2de4c34 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:32.137300 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f74a4-0efb-4922-bca9-454dc2de4c34 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:32.137571 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4f74a4-0efb-4922-bca9-454dc2de4c34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:32.148256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4f74a4-0efb-4922-bca9-454dc2de4c34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-7e4f74a4-0efb-4922-bca9-454dc2de4c34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:30:32.155852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:32.157202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:32.158597 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 572/1141] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:30:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:32.163198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ca28c-97cd-497e-bb9f-7af753b2079d None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:32.262272 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ca28c-97cd-497e-bb9f-7af753b2079d None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:32.296664 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74bc85-c8e3-4ee3-965b-02c8d59f6c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:32.297332 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74bc85-c8e3-4ee3-965b-02c8d59f6c53 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:32.297614 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74bc85-c8e3-4ee3-965b-02c8d59f6c53 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:32.297888 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74bc85-c8e3-4ee3-965b-02c8d59f6c53 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:32.298635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74bc85-c8e3-4ee3-965b-02c8d59f6c53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:32.298942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74bc85-c8e3-4ee3-965b-02c8d59f6c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:32.300357 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74bc85-c8e3-4ee3-965b-02c8d59f6c53 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:32.308533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 570/1142] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:30:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:32.314991 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f74a4-0efb-4922-bca9-454dc2de4c34 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:32.405737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:32.414282 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4f74a4-0efb-4922-bca9-454dc2de4c34 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:32.451238 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c9a41-1f07-4eb3-911f-b5b930a34554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:32.452224 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c9a41-1f07-4eb3-911f-b5b930a34554 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:32.452548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c9a41-1f07-4eb3-911f-b5b930a34554 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:32.452831 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c9a41-1f07-4eb3-911f-b5b930a34554 None admin] PATH_INFO: `/v3/users/68113982d4d1483eb975d537d5e2ee44` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:32.466733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c9a41-1f07-4eb3-911f-b5b930a34554 None admin] RBAC: Authorizing `identity:get_user(user_id=68113982d4d1483eb975d537d5e2ee44)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:32.468177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5c9a41-1f07-4eb3-911f-b5b930a34554 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:32.469905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 573/1143] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:32 2022] GET /identity/v3/users/68113982d4d1483eb975d537d5e2ee44 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:32.475727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74bc85-c8e3-4ee3-965b-02c8d59f6c53 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:32.488254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:32.577818 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74bc85-c8e3-4ee3-965b-02c8d59f6c53 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:32.611587 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782d894-4ff5-4f95-bd6f-8e77063c77cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:32.612232 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782d894-4ff5-4f95-bd6f-8e77063c77cf None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:32.612538 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782d894-4ff5-4f95-bd6f-8e77063c77cf None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:32.612813 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782d894-4ff5-4f95-bd6f-8e77063c77cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:32.678937 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782d894-4ff5-4f95-bd6f-8e77063c77cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:32.680424 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782d894-4ff5-4f95-bd6f-8e77063c77cf None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:32.689939 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:32.777396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 571/1144] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:30:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:30:32.784162 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c9a41-1f07-4eb3-911f-b5b930a34554 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:32.883985 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c9a41-1f07-4eb3-911f-b5b930a34554 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:32.918537 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d36be-20e5-48b0-ad36-91550ae3f107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:32.919390 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d36be-20e5-48b0-ad36-91550ae3f107 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:32.919673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d36be-20e5-48b0-ad36-91550ae3f107 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:32.920525 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d36be-20e5-48b0-ad36-91550ae3f107 None admin] PATH_INFO: `/v3/projects/680236500853441c9ab0613d5771ad1c` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:32.921714 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d36be-20e5-48b0-ad36-91550ae3f107 None admin] RBAC: Authorizing `identity:get_project(project_id=680236500853441c9ab0613d5771ad1c)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:32.923133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d36be-20e5-48b0-ad36-91550ae3f107 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:32.924871 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 574/1145] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:30:32 2022] GET /identity/v3/projects/680236500853441c9ab0613d5771ad1c => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:32.931201 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782d894-4ff5-4f95-bd6f-8e77063c77cf None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:33.029486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782d894-4ff5-4f95-bd6f-8e77063c77cf None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:33.039369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:33.063721 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bad333-4403-4313-b1ec-45729c6666a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:33.064398 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad333-4403-4313-b1ec-45729c6666a5 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:33.064741 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad333-4403-4313-b1ec-45729c6666a5 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:33.064963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bad333-4403-4313-b1ec-45729c6666a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:33.065881 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bad333-4403-4313-b1ec-45729c6666a5 None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=680236500853441c9ab0613d5771ad1c, user.id=68113982d4d1483eb975d537d5e2ee44) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:33.066197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bad333-4403-4313-b1ec-45729c6666a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:33.067627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bad333-4403-4313-b1ec-45729c6666a5 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:33.073105 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 572/1146] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:30:32 2022] GET /identity/v3/role_assignments?user.id=68113982d4d1483eb975d537d5e2ee44&scope.project.id=680236500853441c9ab0613d5771ad1c&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:35.293749 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a8b83-ae6c-45bd-a0c6-21a4eec33646 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:35.294346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a8b83-ae6c-45bd-a0c6-21a4eec33646 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:35.294346 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a8b83-ae6c-45bd-a0c6-21a4eec33646 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:35.295668 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 575/1147] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:35.301580 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a840e6a-4583-4b7d-beca-251f5d465688 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:35.301999 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a840e6a-4583-4b7d-beca-251f5d465688 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:35.302154 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a840e6a-4583-4b7d-beca-251f5d465688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:35.319724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-5a840e6a-4583-4b7d-beca-251f5d465688 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:35.321065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-5a840e6a-4583-4b7d-beca-251f5d465688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:35.391347 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 573/1148] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:35.413576 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a8b83-ae6c-45bd-a0c6-21a4eec33646 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:35.435448 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:35.521143 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a8b83-ae6c-45bd-a0c6-21a4eec33646 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:35.556016 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc4faa-4b31-483e-9c68-f6b75c9c2b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:35.556777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc4faa-4b31-483e-9c68-f6b75c9c2b2b None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:35.557118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc4faa-4b31-483e-9c68-f6b75c9c2b2b None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:35.557400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc4faa-4b31-483e-9c68-f6b75c9c2b2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:35.568087 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cc4faa-4b31-483e-9c68-f6b75c9c2b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-66cc4faa-4b31-483e-9c68-f6b75c9c2b2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:30:35.575775 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:35.577074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:35.578199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 576/1149] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:30:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:35.583299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a840e6a-4583-4b7d-beca-251f5d465688 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:35.683123 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a840e6a-4583-4b7d-beca-251f5d465688 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:35.684627 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:35.717189 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76a0cc-d91c-4e7a-9913-53b164b7bd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:35.718381 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76a0cc-d91c-4e7a-9913-53b164b7bd53 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:35.718662 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76a0cc-d91c-4e7a-9913-53b164b7bd53 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:35.718939 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76a0cc-d91c-4e7a-9913-53b164b7bd53 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:35.719677 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76a0cc-d91c-4e7a-9913-53b164b7bd53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:35.719984 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76a0cc-d91c-4e7a-9913-53b164b7bd53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:35.721544 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b76a0cc-d91c-4e7a-9913-53b164b7bd53 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:35.729680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 574/1150] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:30:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:35.736212 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc4faa-4b31-483e-9c68-f6b75c9c2b2b None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:35.817731 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:35.838612 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc4faa-4b31-483e-9c68-f6b75c9c2b2b None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:35.873514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872858a6-7270-4af2-ae18-dfaa8a95c58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:35.874424 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872858a6-7270-4af2-ae18-dfaa8a95c58a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:35.874704 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872858a6-7270-4af2-ae18-dfaa8a95c58a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:35.874974 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872858a6-7270-4af2-ae18-dfaa8a95c58a None admin] PATH_INFO: `/v3/users/68113982d4d1483eb975d537d5e2ee44` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:35.876155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872858a6-7270-4af2-ae18-dfaa8a95c58a None admin] RBAC: Authorizing `identity:get_user(user_id=68113982d4d1483eb975d537d5e2ee44)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:35.877626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872858a6-7270-4af2-ae18-dfaa8a95c58a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:35.879252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 577/1151] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:35 2022] GET /identity/v3/users/68113982d4d1483eb975d537d5e2ee44 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:35.885082 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76a0cc-d91c-4e7a-9913-53b164b7bd53 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:35.987725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b76a0cc-d91c-4e7a-9913-53b164b7bd53 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:36.021581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529746d0-2d0d-489b-acbb-20f6bf953dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:36.022444 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529746d0-2d0d-489b-acbb-20f6bf953dfb None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:36.022730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529746d0-2d0d-489b-acbb-20f6bf953dfb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:36.023009 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529746d0-2d0d-489b-acbb-20f6bf953dfb None admin] PATH_INFO: `/v3/projects/680236500853441c9ab0613d5771ad1c` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:36.024201 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529746d0-2d0d-489b-acbb-20f6bf953dfb None admin] RBAC: Authorizing `identity:get_project(project_id=680236500853441c9ab0613d5771ad1c)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:36.025751 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529746d0-2d0d-489b-acbb-20f6bf953dfb None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:36.027476 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 575/1152] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:30:35 2022] GET /identity/v3/projects/680236500853441c9ab0613d5771ad1c => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:36.033578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872858a6-7270-4af2-ae18-dfaa8a95c58a None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:36.132272 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872858a6-7270-4af2-ae18-dfaa8a95c58a None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:36.165066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77131b8-3dec-425d-af49-128b19bfd6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:36.166294 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77131b8-3dec-425d-af49-128b19bfd6a7 None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:36.166513 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77131b8-3dec-425d-af49-128b19bfd6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:36.166783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77131b8-3dec-425d-af49-128b19bfd6a7 None admin] PATH_INFO: `/v3/projects/680236500853441c9ab0613d5771ad1c/users/68113982d4d1483eb975d537d5e2ee44/roles/8eaff9cd613a49b29548e90d7e3e9215` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:36.168792 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77131b8-3dec-425d-af49-128b19bfd6a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=680236500853441c9ab0613d5771ad1c, user_id=68113982d4d1483eb975d537d5e2ee44, role_id=8eaff9cd613a49b29548e90d7e3e9215)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:36.170208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77131b8-3dec-425d-af49-128b19bfd6a7 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:36.283932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 578/1153] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:30:36 2022] PUT /identity/v3/projects/680236500853441c9ab0613d5771ad1c/users/68113982d4d1483eb975d537d5e2ee44/roles/8eaff9cd613a49b29548e90d7e3e9215 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:30:38.500564 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713aabd-cd88-49fe-8301-4bbcaa9fec72 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:38.501088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713aabd-cd88-49fe-8301-4bbcaa9fec72 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:38.501216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713aabd-cd88-49fe-8301-4bbcaa9fec72 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:38.502975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 576/1154] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:38.508539 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7029a45-4c99-458d-af49-294889e92fda None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:38.508977 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7029a45-4c99-458d-af49-294889e92fda None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:38.509086 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7029a45-4c99-458d-af49-294889e92fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:38.520594 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:38.525941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-f7029a45-4c99-458d-af49-294889e92fda None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:38.527146 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-f7029a45-4c99-458d-af49-294889e92fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:38.594322 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 579/1155] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:38.618287 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713aabd-cd88-49fe-8301-4bbcaa9fec72 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:38.650637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:38.724859 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713aabd-cd88-49fe-8301-4bbcaa9fec72 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:38.759616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6615c3-8fbf-42e0-bb01-35448556b6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:38.760470 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6615c3-8fbf-42e0-bb01-35448556b6ac None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:38.760759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6615c3-8fbf-42e0-bb01-35448556b6ac None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:38.761064 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6615c3-8fbf-42e0-bb01-35448556b6ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:38.771434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6615c3-8fbf-42e0-bb01-35448556b6ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-6c6615c3-8fbf-42e0-bb01-35448556b6ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:30:38.779635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:38.781446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:38.782821 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 577/1156] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:30:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:38.787747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7029a45-4c99-458d-af49-294889e92fda None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:38.888919 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7029a45-4c99-458d-af49-294889e92fda None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:38.923213 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba804f-fd73-4ee9-bbf6-c59a3e923460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:38.923888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba804f-fd73-4ee9-bbf6-c59a3e923460 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:38.924172 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba804f-fd73-4ee9-bbf6-c59a3e923460 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:38.924484 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba804f-fd73-4ee9-bbf6-c59a3e923460 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:38.925240 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba804f-fd73-4ee9-bbf6-c59a3e923460 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:38.925552 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba804f-fd73-4ee9-bbf6-c59a3e923460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:38.926944 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba804f-fd73-4ee9-bbf6-c59a3e923460 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:38.929259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:38.935675 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 580/1157] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:30:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:30:38.943113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6615c3-8fbf-42e0-bb01-35448556b6ac None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:39.037146 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:39.045972 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6615c3-8fbf-42e0-bb01-35448556b6ac None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:39.080253 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bc520-9b70-4f8d-9cbf-fb98eaf35cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:39.081206 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bc520-9b70-4f8d-9cbf-fb98eaf35cfb None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:39.081695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bc520-9b70-4f8d-9cbf-fb98eaf35cfb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:39.082200 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bc520-9b70-4f8d-9cbf-fb98eaf35cfb None admin] PATH_INFO: `/v3/users/68113982d4d1483eb975d537d5e2ee44` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:39.083816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bc520-9b70-4f8d-9cbf-fb98eaf35cfb None admin] RBAC: Authorizing `identity:get_user(user_id=68113982d4d1483eb975d537d5e2ee44)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:39.085302 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bc520-9b70-4f8d-9cbf-fb98eaf35cfb None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:39.087035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 578/1158] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:38 2022] GET /identity/v3/users/68113982d4d1483eb975d537d5e2ee44 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:39.092920 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba804f-fd73-4ee9-bbf6-c59a3e923460 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:39.198807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba804f-fd73-4ee9-bbf6-c59a3e923460 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:39.236268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd082a-185f-42a4-90ce-d2684a84a8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:39.237153 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd082a-185f-42a4-90ce-d2684a84a8da None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:39.237509 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd082a-185f-42a4-90ce-d2684a84a8da None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:39.237709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd082a-185f-42a4-90ce-d2684a84a8da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:39.308047 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:39.313278 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd082a-185f-42a4-90ce-d2684a84a8da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:39.314781 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd082a-185f-42a4-90ce-d2684a84a8da None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:39.420844 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 581/1159] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:30:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:30:39.426637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bc520-9b70-4f8d-9cbf-fb98eaf35cfb None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:39.533075 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bc520-9b70-4f8d-9cbf-fb98eaf35cfb None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:39.568626 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c55db-6cc9-426c-84e6-0a1b8cdae9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:39.569408 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c55db-6cc9-426c-84e6-0a1b8cdae9aa None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:39.569695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c55db-6cc9-426c-84e6-0a1b8cdae9aa None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:39.569975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c55db-6cc9-426c-84e6-0a1b8cdae9aa None admin] PATH_INFO: `/v3/projects/680236500853441c9ab0613d5771ad1c` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:39.571226 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c55db-6cc9-426c-84e6-0a1b8cdae9aa None admin] RBAC: Authorizing `identity:get_project(project_id=680236500853441c9ab0613d5771ad1c)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:39.572721 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637c55db-6cc9-426c-84e6-0a1b8cdae9aa None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:39.574538 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 579/1160] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:30:39 2022] GET /identity/v3/projects/680236500853441c9ab0613d5771ad1c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:39.582116 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd082a-185f-42a4-90ce-d2684a84a8da None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:39.682550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:39.686176 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd082a-185f-42a4-90ce-d2684a84a8da None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:39.721635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7905df-375f-40aa-b559-48b820f0824d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:39.722348 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7905df-375f-40aa-b559-48b820f0824d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:39.722584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7905df-375f-40aa-b559-48b820f0824d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:39.722861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7905df-375f-40aa-b559-48b820f0824d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:39.723695 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7905df-375f-40aa-b559-48b820f0824d None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=680236500853441c9ab0613d5771ad1c, user.id=68113982d4d1483eb975d537d5e2ee44) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:39.724006 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7905df-375f-40aa-b559-48b820f0824d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:39.725737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7905df-375f-40aa-b559-48b820f0824d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:39.732256 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 582/1161] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:30:39 2022] GET /identity/v3/role_assignments?user.id=68113982d4d1483eb975d537d5e2ee44&scope.project.id=680236500853441c9ab0613d5771ad1c&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:42.026220 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e91f75-3637-42b4-a76d-31029eafbdc9 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:42.026751 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e91f75-3637-42b4-a76d-31029eafbdc9 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:42.026751 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e91f75-3637-42b4-a76d-31029eafbdc9 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:42.028153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 580/1162] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:42.033778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb958b3-cef2-4dc7-abbb-eef581c57671 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:42.034214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb958b3-cef2-4dc7-abbb-eef581c57671 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:42.034333 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb958b3-cef2-4dc7-abbb-eef581c57671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:42.052281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-1cb958b3-cef2-4dc7-abbb-eef581c57671 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:42.053659 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-1cb958b3-cef2-4dc7-abbb-eef581c57671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:42.125268 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 583/1163] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:42.147451 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e91f75-3637-42b4-a76d-31029eafbdc9 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:42.178088 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:42.257617 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e91f75-3637-42b4-a76d-31029eafbdc9 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:42.293079 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce400f29-5d04-415c-8bda-f18de1440f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:42.293739 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400f29-5d04-415c-8bda-f18de1440f63 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:42.294025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400f29-5d04-415c-8bda-f18de1440f63 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:42.294302 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce400f29-5d04-415c-8bda-f18de1440f63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:42.366134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce400f29-5d04-415c-8bda-f18de1440f63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:42.367603 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce400f29-5d04-415c-8bda-f18de1440f63 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:42.396093 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:42.471129 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 581/1164] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:30:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:30:42.477175 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb958b3-cef2-4dc7-abbb-eef581c57671 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:42.576026 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:42.585075 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb958b3-cef2-4dc7-abbb-eef581c57671 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:42.620859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c6b5b-b0f7-45ed-b795-19baf52102b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:42.621682 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c6b5b-b0f7-45ed-b795-19baf52102b8 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:42.621972 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c6b5b-b0f7-45ed-b795-19baf52102b8 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:42.622259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c6b5b-b0f7-45ed-b795-19baf52102b8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:42.629431 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c6b5b-b0f7-45ed-b795-19baf52102b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:42.630884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c6b5b-b0f7-45ed-b795-19baf52102b8 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-592c6b5b-b0f7-45ed-b795-19baf52102b8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:42.645855 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:42.647287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:30:42.648584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:30:42.652367 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 584/1165] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:30:42 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:42.653340 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce400f29-5d04-415c-8bda-f18de1440f63 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:42.759309 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce400f29-5d04-415c-8bda-f18de1440f63 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:42.788819 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:42.797712 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b4865-3991-4020-b4b0-0f2aae346f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:42.798438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b4865-3991-4020-b4b0-0f2aae346f82 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:42.798726 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b4865-3991-4020-b4b0-0f2aae346f82 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:42.799057 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b4865-3991-4020-b4b0-0f2aae346f82 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:42.799816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b4865-3991-4020-b4b0-0f2aae346f82 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:42.800125 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b4865-3991-4020-b4b0-0f2aae346f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:42.801656 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b4865-3991-4020-b4b0-0f2aae346f82 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:42.809181 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 582/1166] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:30:42 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:45.076901 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632ef00-5389-47a6-adf9-a57e7988e4d0 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:45.077454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632ef00-5389-47a6-adf9-a57e7988e4d0 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:45.077454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632ef00-5389-47a6-adf9-a57e7988e4d0 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:45.078827 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 585/1167] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:45.084657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b980b42-75a5-4b9d-83d7-b13bd3615604 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:45.085152 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b980b42-75a5-4b9d-83d7-b13bd3615604 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:45.085259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b980b42-75a5-4b9d-83d7-b13bd3615604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:45.102472 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-8b980b42-75a5-4b9d-83d7-b13bd3615604 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:45.103666 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-8b980b42-75a5-4b9d-83d7-b13bd3615604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:45.175141 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 583/1168] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:45.198593 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632ef00-5389-47a6-adf9-a57e7988e4d0 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:45.309821 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632ef00-5389-47a6-adf9-a57e7988e4d0 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:45.345495 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed409460-b9ca-4ef1-9b4d-30730849e255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:45.346179 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed409460-b9ca-4ef1-9b4d-30730849e255 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:45.346471 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed409460-b9ca-4ef1-9b4d-30730849e255 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:45.346755 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed409460-b9ca-4ef1-9b4d-30730849e255 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:45.347515 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed409460-b9ca-4ef1-9b4d-30730849e255 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:45.348932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed409460-b9ca-4ef1-9b4d-30730849e255 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:45.457729 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 586/1169] 10.222.0.88 () {60 vars in 1138 bytes} [Thu May 19 20:30:45 2022] POST /identity/v3/domains => generated 267 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:30:47.735027 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433ba26-8a94-4ad1-b2b5-896954c3f186 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:47.735526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433ba26-8a94-4ad1-b2b5-896954c3f186 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:47.735526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433ba26-8a94-4ad1-b2b5-896954c3f186 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:47.736905 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 584/1170] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:47.742539 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef365b-359a-4f5a-aedf-0ea4a7fc2fc6 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:47.742989 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef365b-359a-4f5a-aedf-0ea4a7fc2fc6 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:47.743087 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef365b-359a-4f5a-aedf-0ea4a7fc2fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:47.760237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-44ef365b-359a-4f5a-aedf-0ea4a7fc2fc6 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:47.761474 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-44ef365b-359a-4f5a-aedf-0ea4a7fc2fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:47.769069 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:47.831221 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 587/1171] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:47.855519 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433ba26-8a94-4ad1-b2b5-896954c3f186 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:47.965347 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433ba26-8a94-4ad1-b2b5-896954c3f186 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:47.988961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:48.000257 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fc492-7720-4b9b-a3f3-116fd5d593c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:48.001212 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fc492-7720-4b9b-a3f3-116fd5d593c0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:48.001496 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fc492-7720-4b9b-a3f3-116fd5d593c0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:48.001773 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fc492-7720-4b9b-a3f3-116fd5d593c0 None admin] PATH_INFO: `/v3/domains/cf1729722ce2403c90f2381a0b682fbf` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:48.012302 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fc492-7720-4b9b-a3f3-116fd5d593c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cf1729722ce2403c90f2381a0b682fbf)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:48.013854 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fc492-7720-4b9b-a3f3-116fd5d593c0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:48.016058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 585/1172] 10.222.0.88 () {56 vars in 1183 bytes} [Thu May 19 20:30:47 2022] GET /identity/v3/domains/cf1729722ce2403c90f2381a0b682fbf => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:48.021946 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef365b-359a-4f5a-aedf-0ea4a7fc2fc6 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:48.121854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef365b-359a-4f5a-aedf-0ea4a7fc2fc6 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:48.156407 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206a3eb-edf9-4e41-9423-baacc5041e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:48.157069 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206a3eb-edf9-4e41-9423-baacc5041e54 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:48.157358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206a3eb-edf9-4e41-9423-baacc5041e54 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:48.157638 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206a3eb-edf9-4e41-9423-baacc5041e54 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:48.158366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206a3eb-edf9-4e41-9423-baacc5041e54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:48.159813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206a3eb-edf9-4e41-9423-baacc5041e54 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:48.172962 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:48.261982 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 588/1173] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:30:48 2022] POST /identity/v3/projects => generated 361 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:30:50.530253 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47270a-0c64-4e37-a4ca-43c08eb13f5c None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:50.530834 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47270a-0c64-4e37-a4ca-43c08eb13f5c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:50.530834 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47270a-0c64-4e37-a4ca-43c08eb13f5c None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:50.532222 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 586/1174] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:50.538879 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed3b35-2506-4736-890f-1048de234f35 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:50.539401 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed3b35-2506-4736-890f-1048de234f35 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:50.539508 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed3b35-2506-4736-890f-1048de234f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:50.557058 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-c5ed3b35-2506-4736-890f-1048de234f35 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:50.558257 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-c5ed3b35-2506-4736-890f-1048de234f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:50.627842 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 589/1175] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:50.653500 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47270a-0c64-4e37-a4ca-43c08eb13f5c None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:50.757263 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47270a-0c64-4e37-a4ca-43c08eb13f5c None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:50.798446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416219fa-5ca8-44db-a143-d702888dad94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:50.799327 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416219fa-5ca8-44db-a143-d702888dad94 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:50.799611 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416219fa-5ca8-44db-a143-d702888dad94 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:50.799885 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416219fa-5ca8-44db-a143-d702888dad94 None admin] PATH_INFO: `/v3/domains/cf1729722ce2403c90f2381a0b682fbf` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:50.801098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416219fa-5ca8-44db-a143-d702888dad94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cf1729722ce2403c90f2381a0b682fbf)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:50.802538 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416219fa-5ca8-44db-a143-d702888dad94 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:50.804182 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 587/1176] 10.222.0.88 () {56 vars in 1183 bytes} [Thu May 19 20:30:50 2022] GET /identity/v3/domains/cf1729722ce2403c90f2381a0b682fbf => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:50.810363 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed3b35-2506-4736-890f-1048de234f35 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:50.922081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed3b35-2506-4736-890f-1048de234f35 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:50.950419 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:50.963854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984a635-26c1-43c4-88f1-66cfe2225764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:50.964559 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984a635-26c1-43c4-88f1-66cfe2225764 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:50.964844 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984a635-26c1-43c4-88f1-66cfe2225764 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:50.965120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984a635-26c1-43c4-88f1-66cfe2225764 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:50.965841 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984a635-26c1-43c4-88f1-66cfe2225764 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:50.967936 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2984a635-26c1-43c4-88f1-66cfe2225764 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:51.066635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 590/1177] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:30:50 2022] POST /identity/v3/users => generated 311 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:30:53.317360 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e2efb-e06f-415b-924c-9212492662cb None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:53.317944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e2efb-e06f-415b-924c-9212492662cb None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:53.317944 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e2efb-e06f-415b-924c-9212492662cb None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:53.319316 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 588/1178] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:53.324709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a73ddc-414a-4366-bb34-9d56ce6bdba5 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:53.325143 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a73ddc-414a-4366-bb34-9d56ce6bdba5 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:53.325258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a73ddc-414a-4366-bb34-9d56ce6bdba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:53.342062 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-e4a73ddc-414a-4366-bb34-9d56ce6bdba5 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:53.343237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-e4a73ddc-414a-4366-bb34-9d56ce6bdba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:53.409644 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 591/1179] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:53.433590 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e2efb-e06f-415b-924c-9212492662cb None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:53.486039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:53.536812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e2efb-e06f-415b-924c-9212492662cb None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:53.586137 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1593e-9a33-4e29-a61b-8c3df4f28aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:53.586137 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1593e-9a33-4e29-a61b-8c3df4f28aa3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:53.586137 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1593e-9a33-4e29-a61b-8c3df4f28aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:53.586137 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1593e-9a33-4e29-a61b-8c3df4f28aa3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:53.586587 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1593e-9a33-4e29-a61b-8c3df4f28aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-f6d1593e-9a33-4e29-a61b-8c3df4f28aa3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:30:53.594188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:53.595455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:53.596780 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 589/1180] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:53.601446 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a73ddc-414a-4366-bb34-9d56ce6bdba5 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:53.701135 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a73ddc-414a-4366-bb34-9d56ce6bdba5 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:53.712289 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:53.736301 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006514-8a52-4b32-bb07-faf668ea4885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:53.737000 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006514-8a52-4b32-bb07-faf668ea4885 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:53.737288 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006514-8a52-4b32-bb07-faf668ea4885 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:53.737565 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20006514-8a52-4b32-bb07-faf668ea4885 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:53.738313 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20006514-8a52-4b32-bb07-faf668ea4885 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:53.738640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20006514-8a52-4b32-bb07-faf668ea4885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:53.740017 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20006514-8a52-4b32-bb07-faf668ea4885 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:53.747977 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 592/1181] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:53.753836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1593e-9a33-4e29-a61b-8c3df4f28aa3 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:53.857632 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1593e-9a33-4e29-a61b-8c3df4f28aa3 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:53.868309 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:53.891419 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1956b4-8659-4200-9c80-7ffa6d6a989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:53.892282 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1956b4-8659-4200-9c80-7ffa6d6a989b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:53.892596 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1956b4-8659-4200-9c80-7ffa6d6a989b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:53.892967 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1956b4-8659-4200-9c80-7ffa6d6a989b None admin] PATH_INFO: `/v3/users/02a55d93d30f4e6c966a7b6bea58f60d` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:53.907141 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1956b4-8659-4200-9c80-7ffa6d6a989b None admin] RBAC: Authorizing `identity:get_user(user_id=02a55d93d30f4e6c966a7b6bea58f60d)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:53.908638 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1956b4-8659-4200-9c80-7ffa6d6a989b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:53.910394 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 590/1182] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:53 2022] GET /identity/v3/users/02a55d93d30f4e6c966a7b6bea58f60d => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:53.916394 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006514-8a52-4b32-bb07-faf668ea4885 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:54.022291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20006514-8a52-4b32-bb07-faf668ea4885 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:54.057583 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515d2b5-d83b-47f7-be3e-251ca3c26967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:54.058228 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515d2b5-d83b-47f7-be3e-251ca3c26967 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:54.058511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515d2b5-d83b-47f7-be3e-251ca3c26967 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:54.058786 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515d2b5-d83b-47f7-be3e-251ca3c26967 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:54.100081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:54.125569 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515d2b5-d83b-47f7-be3e-251ca3c26967 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:54.127219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515d2b5-d83b-47f7-be3e-251ca3c26967 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:54.223900 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 593/1183] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:30:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:30:54.229828 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1956b4-8659-4200-9c80-7ffa6d6a989b None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:54.334731 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1956b4-8659-4200-9c80-7ffa6d6a989b None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:54.368258 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944508e-34a6-4398-bc19-8b32f77d015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:54.369145 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944508e-34a6-4398-bc19-8b32f77d015d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:54.369428 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944508e-34a6-4398-bc19-8b32f77d015d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:54.369704 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944508e-34a6-4398-bc19-8b32f77d015d None admin] PATH_INFO: `/v3/projects/e3e48e89816d44d88da240100ef5c8c7` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:54.370876 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944508e-34a6-4398-bc19-8b32f77d015d None admin] RBAC: Authorizing `identity:get_project(project_id=e3e48e89816d44d88da240100ef5c8c7)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:54.372315 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944508e-34a6-4398-bc19-8b32f77d015d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:54.374563 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 591/1184] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:30:54 2022] GET /identity/v3/projects/e3e48e89816d44d88da240100ef5c8c7 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:54.380803 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515d2b5-d83b-47f7-be3e-251ca3c26967 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:54.450284 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:54.480625 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515d2b5-d83b-47f7-be3e-251ca3c26967 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:54.515353 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a0a38b-53d0-40fb-b6c6-d7ef8c76787c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:54.516009 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0a38b-53d0-40fb-b6c6-d7ef8c76787c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:54.516293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0a38b-53d0-40fb-b6c6-d7ef8c76787c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:54.516690 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a0a38b-53d0-40fb-b6c6-d7ef8c76787c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:54.517526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0a38b-53d0-40fb-b6c6-d7ef8c76787c None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=e3e48e89816d44d88da240100ef5c8c7, user.id=02a55d93d30f4e6c966a7b6bea58f60d) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:54.517844 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0a38b-53d0-40fb-b6c6-d7ef8c76787c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:54.519255 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a0a38b-53d0-40fb-b6c6-d7ef8c76787c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:54.524930 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 594/1185] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:30:54 2022] GET /identity/v3/role_assignments?user.id=02a55d93d30f4e6c966a7b6bea58f60d&scope.project.id=e3e48e89816d44d88da240100ef5c8c7&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:56.761645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453818d-b0c8-438e-8bab-55d51fb56d27 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:56.762220 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453818d-b0c8-438e-8bab-55d51fb56d27 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:56.762220 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453818d-b0c8-438e-8bab-55d51fb56d27 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:56.763466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 592/1186] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:56.768797 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135022e-710a-49e7-9141-1b0799c1f414 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:56.769260 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135022e-710a-49e7-9141-1b0799c1f414 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:56.769366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135022e-710a-49e7-9141-1b0799c1f414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:56.786847 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-6135022e-710a-49e7-9141-1b0799c1f414 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:56.788102 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-6135022e-710a-49e7-9141-1b0799c1f414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:56.855420 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 595/1187] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:30:56.878342 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453818d-b0c8-438e-8bab-55d51fb56d27 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:56.914439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:56.979242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453818d-b0c8-438e-8bab-55d51fb56d27 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:57.012906 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de7d85-e480-4104-8c68-634bd1b5ed8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:57.014154 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de7d85-e480-4104-8c68-634bd1b5ed8e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:57.014437 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de7d85-e480-4104-8c68-634bd1b5ed8e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:57.014711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de7d85-e480-4104-8c68-634bd1b5ed8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:57.024856 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de7d85-e480-4104-8c68-634bd1b5ed8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-36de7d85-e480-4104-8c68-634bd1b5ed8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:30:57.033188 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:30:57.034533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:30:57.035765 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 593/1188] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:30:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:30:57.040779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135022e-710a-49e7-9141-1b0799c1f414 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:57.106988 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:57.141601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135022e-710a-49e7-9141-1b0799c1f414 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:57.175142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a45ae-5134-4542-8b67-2cf3f14a6505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:57.175814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a45ae-5134-4542-8b67-2cf3f14a6505 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:57.176094 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a45ae-5134-4542-8b67-2cf3f14a6505 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:57.176392 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a45ae-5134-4542-8b67-2cf3f14a6505 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:57.177140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a45ae-5134-4542-8b67-2cf3f14a6505 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:30:57.177448 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a45ae-5134-4542-8b67-2cf3f14a6505 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:57.178803 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a45ae-5134-4542-8b67-2cf3f14a6505 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:57.187081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 596/1189] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:30:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:57.193092 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de7d85-e480-4104-8c68-634bd1b5ed8e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:57.287535 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:57.293461 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de7d85-e480-4104-8c68-634bd1b5ed8e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:57.326663 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb5041-0445-437e-8a0d-ad4da552bb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:57.327584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb5041-0445-437e-8a0d-ad4da552bb4b None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:57.328065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb5041-0445-437e-8a0d-ad4da552bb4b None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:57.328589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb5041-0445-437e-8a0d-ad4da552bb4b None admin] PATH_INFO: `/v3/users/02a55d93d30f4e6c966a7b6bea58f60d` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:57.330087 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb5041-0445-437e-8a0d-ad4da552bb4b None admin] RBAC: Authorizing `identity:get_user(user_id=02a55d93d30f4e6c966a7b6bea58f60d)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:57.331537 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb5041-0445-437e-8a0d-ad4da552bb4b None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:57.333302 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 594/1190] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:30:57 2022] GET /identity/v3/users/02a55d93d30f4e6c966a7b6bea58f60d => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:57.339012 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a45ae-5134-4542-8b67-2cf3f14a6505 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:57.438849 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a45ae-5134-4542-8b67-2cf3f14a6505 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:57.464317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:30:57.472422 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ef7cb-67cd-4473-81ca-852a74c78161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:57.473292 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ef7cb-67cd-4473-81ca-852a74c78161 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:57.473576 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ef7cb-67cd-4473-81ca-852a74c78161 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:57.473849 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ef7cb-67cd-4473-81ca-852a74c78161 None admin] PATH_INFO: `/v3/projects/e3e48e89816d44d88da240100ef5c8c7` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:57.475036 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ef7cb-67cd-4473-81ca-852a74c78161 None admin] RBAC: Authorizing `identity:get_project(project_id=e3e48e89816d44d88da240100ef5c8c7)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:57.476499 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ef7cb-67cd-4473-81ca-852a74c78161 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:57.481889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 597/1191] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:30:57 2022] GET /identity/v3/projects/e3e48e89816d44d88da240100ef5c8c7 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:30:57.485039 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb5041-0445-437e-8a0d-ad4da552bb4b None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:30:57.584601 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb5041-0445-437e-8a0d-ad4da552bb4b None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:30:57.620223 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ceed0-a86f-4915-be93-d4d3c87c9f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:30:57.621138 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ceed0-a86f-4915-be93-d4d3c87c9f76 None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:57.621417 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ceed0-a86f-4915-be93-d4d3c87c9f76 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:57.621690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ceed0-a86f-4915-be93-d4d3c87c9f76 None admin] PATH_INFO: `/v3/projects/e3e48e89816d44d88da240100ef5c8c7/users/02a55d93d30f4e6c966a7b6bea58f60d/roles/8eaff9cd613a49b29548e90d7e3e9215` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:57.623655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ceed0-a86f-4915-be93-d4d3c87c9f76 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3e48e89816d44d88da240100ef5c8c7, user_id=02a55d93d30f4e6c966a7b6bea58f60d, role_id=8eaff9cd613a49b29548e90d7e3e9215)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:30:57.625132 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ceed0-a86f-4915-be93-d4d3c87c9f76 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:30:57.729418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 595/1192] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:30:57 2022] PUT /identity/v3/projects/e3e48e89816d44d88da240100ef5c8c7/users/02a55d93d30f4e6c966a7b6bea58f60d/roles/8eaff9cd613a49b29548e90d7e3e9215 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:30:59.937392 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db186f-5bda-40d9-8589-abeed410a1f8 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:59.937984 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db186f-5bda-40d9-8589-abeed410a1f8 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:59.937984 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db186f-5bda-40d9-8589-abeed410a1f8 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:59.939534 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 598/1193] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:30:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:30:59.944890 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b5063-3421-4df2-92da-09cea1db20e8 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:30:59.945319 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b5063-3421-4df2-92da-09cea1db20e8 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:30:59.945433 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b5063-3421-4df2-92da-09cea1db20e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:30:59.961865 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-0f8b5063-3421-4df2-92da-09cea1db20e8 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:30:59.963042 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-0f8b5063-3421-4df2-92da-09cea1db20e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:30:59.980085 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:31:00.028927 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 596/1194] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:30:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:31:00.052897 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db186f-5bda-40d9-8589-abeed410a1f8 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:00.157443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db186f-5bda-40d9-8589-abeed410a1f8 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:00.192081 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bdc84-ac67-4bf2-b018-122f349c2966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:00.192824 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bdc84-ac67-4bf2-b018-122f349c2966 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:00.193106 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bdc84-ac67-4bf2-b018-122f349c2966 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:00.193378 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bdc84-ac67-4bf2-b018-122f349c2966 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:00.203491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bdc84-ac67-4bf2-b018-122f349c2966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-ae2bdc84-ac67-4bf2-b018-122f349c2966 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:31:00.211140 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:31:00.212352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:31:00.213379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 599/1195] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:31:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:31:00.218576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b5063-3421-4df2-92da-09cea1db20e8 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:00.319025 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b5063-3421-4df2-92da-09cea1db20e8 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:00.365519 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bfc5f-9e40-4db2-984f-77e50035af61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:00.366505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bfc5f-9e40-4db2-984f-77e50035af61 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:00.366930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bfc5f-9e40-4db2-984f-77e50035af61 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:00.367351 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bfc5f-9e40-4db2-984f-77e50035af61 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:00.368565 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46bfc5f-9e40-4db2-984f-77e50035af61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:31:00.368881 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46bfc5f-9e40-4db2-984f-77e50035af61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:31:00.370283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46bfc5f-9e40-4db2-984f-77e50035af61 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:31:00.380909 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 597/1196] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:31:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:31:00.386749 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bdc84-ac67-4bf2-b018-122f349c2966 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:00.444307 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:31:00.489495 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bdc84-ac67-4bf2-b018-122f349c2966 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:00.523526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e11a9-891e-4919-bf7e-44ae38594619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:00.524421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e11a9-891e-4919-bf7e-44ae38594619 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:00.524706 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e11a9-891e-4919-bf7e-44ae38594619 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:00.524978 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e11a9-891e-4919-bf7e-44ae38594619 None admin] PATH_INFO: `/v3/users/02a55d93d30f4e6c966a7b6bea58f60d` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:00.526159 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e11a9-891e-4919-bf7e-44ae38594619 None admin] RBAC: Authorizing `identity:get_user(user_id=02a55d93d30f4e6c966a7b6bea58f60d)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:31:00.527566 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e11a9-891e-4919-bf7e-44ae38594619 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:31:00.529761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 600/1197] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:31:00 2022] GET /identity/v3/users/02a55d93d30f4e6c966a7b6bea58f60d => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:31:00.535533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bfc5f-9e40-4db2-984f-77e50035af61 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:00.554856 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:31:00.635782 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bfc5f-9e40-4db2-984f-77e50035af61 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:00.670209 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319efb0-5920-4305-a5c2-81a63cf592f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:00.670869 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319efb0-5920-4305-a5c2-81a63cf592f8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:00.671153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319efb0-5920-4305-a5c2-81a63cf592f8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:00.671428 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319efb0-5920-4305-a5c2-81a63cf592f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:00.738862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319efb0-5920-4305-a5c2-81a63cf592f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:31:00.740234 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319efb0-5920-4305-a5c2-81a63cf592f8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:31:00.756961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:31:00.838340 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 598/1198] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:31:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:31:00.844900 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e11a9-891e-4919-bf7e-44ae38594619 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:00.943949 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e11a9-891e-4919-bf7e-44ae38594619 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:00.967821 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:31:00.978315 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4353cad-4721-46a4-9639-1e1539a0bd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:00.983561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4353cad-4721-46a4-9639-1e1539a0bd88 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:00.983561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4353cad-4721-46a4-9639-1e1539a0bd88 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:00.983561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4353cad-4721-46a4-9639-1e1539a0bd88 None admin] PATH_INFO: `/v3/projects/e3e48e89816d44d88da240100ef5c8c7` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:00.983561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4353cad-4721-46a4-9639-1e1539a0bd88 None admin] RBAC: Authorizing `identity:get_project(project_id=e3e48e89816d44d88da240100ef5c8c7)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:31:00.983561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4353cad-4721-46a4-9639-1e1539a0bd88 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:31:00.984087 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 601/1199] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:31:00 2022] GET /identity/v3/projects/e3e48e89816d44d88da240100ef5c8c7 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:31:00.990435 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319efb0-5920-4305-a5c2-81a63cf592f8 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:31:01.091326 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319efb0-5920-4305-a5c2-81a63cf592f8 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:31:01.107146 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:31:01.124960 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8d6a2-5847-4327-b229-3390977d129f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:31:01.125621 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8d6a2-5847-4327-b229-3390977d129f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:31:01.125906 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8d6a2-5847-4327-b229-3390977d129f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:31:01.126186 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8d6a2-5847-4327-b229-3390977d129f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:31:01.127014 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8d6a2-5847-4327-b229-3390977d129f None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=e3e48e89816d44d88da240100ef5c8c7, user.id=02a55d93d30f4e6c966a7b6bea58f60d) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:31:01.127335 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8d6a2-5847-4327-b229-3390977d129f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:31:01.128822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8d6a2-5847-4327-b229-3390977d129f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:31:01.134882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 599/1200] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:31:00 2022] GET /identity/v3/role_assignments?user.id=02a55d93d30f4e6c966a7b6bea58f60d&scope.project.id=e3e48e89816d44d88da240100ef5c8c7&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:31:11.002998 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=4 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:31:11.136023 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=10 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:23.428568 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76fdac5-d3a9-494c-a4f8-daf4d4c6f492 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:23.429205 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76fdac5-d3a9-494c-a4f8-daf4d4c6f492 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:23.429205 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76fdac5-d3a9-494c-a4f8-daf4d4c6f492 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:23.430562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 602/1201] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:32:23.436100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e02ea-2aa6-45a9-a0e8-d278b3c1d7d6 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:23.436601 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e02ea-2aa6-45a9-a0e8-d278b3c1d7d6 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:23.436711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e02ea-2aa6-45a9-a0e8-d278b3c1d7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:23.453678 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-ab2e02ea-2aa6-45a9-a0e8-d278b3c1d7d6 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:32:23.454874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-ab2e02ea-2aa6-45a9-a0e8-d278b3c1d7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:32:23.523970 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 600/1202] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:32:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:32:23.549867 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76fdac5-d3a9-494c-a4f8-daf4d4c6f492 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:23.664038 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76fdac5-d3a9-494c-a4f8-daf4d4c6f492 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:23.699262 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e00cac-269f-4500-8099-057cf4151a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:23.699987 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e00cac-269f-4500-8099-057cf4151a35 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:23.700277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e00cac-269f-4500-8099-057cf4151a35 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:23.701103 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e00cac-269f-4500-8099-057cf4151a35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:23.711584 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e00cac-269f-4500-8099-057cf4151a35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:23.713062 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e00cac-269f-4500-8099-057cf4151a35 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-26e00cac-269f-4500-8099-057cf4151a35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:23.732066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:23.733370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:23.734712 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:23.734712 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:23.734712 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:23.734712 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:23.734712 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 603/1203] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:23.739577 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e02ea-2aa6-45a9-a0e8-d278b3c1d7d6 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:23.845830 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e02ea-2aa6-45a9-a0e8-d278b3c1d7d6 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:23.874443 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:23.880132 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5e3cb-635a-438e-aae0-802f788f6258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:23.880838 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5e3cb-635a-438e-aae0-802f788f6258 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:23.881105 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5e3cb-635a-438e-aae0-802f788f6258 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:23.881377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5e3cb-635a-438e-aae0-802f788f6258 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:23.882089 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5e3cb-635a-438e-aae0-802f788f6258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:23.882394 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5e3cb-635a-438e-aae0-802f788f6258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:23.883793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5e3cb-635a-438e-aae0-802f788f6258 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:23.894766 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 601/1204] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:23.900815 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e00cac-269f-4500-8099-057cf4151a35 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:23.955082 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:24.001641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e00cac-269f-4500-8099-057cf4151a35 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:24.035636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cd669-fafd-4c5e-9ced-016dae81417a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:24.036305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cd669-fafd-4c5e-9ced-016dae81417a None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:24.036618 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cd669-fafd-4c5e-9ced-016dae81417a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:24.036896 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cd669-fafd-4c5e-9ced-016dae81417a None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:24.037608 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cd669-fafd-4c5e-9ced-016dae81417a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:24.039455 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cd669-fafd-4c5e-9ced-016dae81417a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:24.166627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 604/1205] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:32:23 2022] POST /identity/v3/users => generated 251 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:32:26.403841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cb067-c6ee-4cf8-9e8a-e55f626f6506 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:26.404466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cb067-c6ee-4cf8-9e8a-e55f626f6506 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:26.404466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cb067-c6ee-4cf8-9e8a-e55f626f6506 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:26.405880 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 602/1206] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:32:26.411306 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0e41d-5dfe-483f-bfda-8540711ee826 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:26.411747 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0e41d-5dfe-483f-bfda-8540711ee826 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:26.411851 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0e41d-5dfe-483f-bfda-8540711ee826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:26.428905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-e4d0e41d-5dfe-483f-bfda-8540711ee826 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:32:26.430079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-e4d0e41d-5dfe-483f-bfda-8540711ee826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:32:26.501144 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 605/1207] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:32:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:32:26.524724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cb067-c6ee-4cf8-9e8a-e55f626f6506 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:26.630432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cb067-c6ee-4cf8-9e8a-e55f626f6506 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:26.674424 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f143cc3-0af8-4e15-9f40-2325ceb2cae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:26.675148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f143cc3-0af8-4e15-9f40-2325ceb2cae1 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:26.675434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f143cc3-0af8-4e15-9f40-2325ceb2cae1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:26.675720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f143cc3-0af8-4e15-9f40-2325ceb2cae1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:26.687266 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f143cc3-0af8-4e15-9f40-2325ceb2cae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-3f143cc3-0af8-4e15-9f40-2325ceb2cae1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:32:26.695343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:26.696774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:32:26.697735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:26.701997 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 603/1208] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:32:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:26.702993 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0e41d-5dfe-483f-bfda-8540711ee826 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:26.744631 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:26.808127 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0e41d-5dfe-483f-bfda-8540711ee826 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:26.843865 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960640f-23df-4160-8fd9-0f6ca82d53a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:26.844601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960640f-23df-4160-8fd9-0f6ca82d53a6 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:26.844876 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960640f-23df-4160-8fd9-0f6ca82d53a6 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:26.845209 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960640f-23df-4160-8fd9-0f6ca82d53a6 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:26.845971 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960640f-23df-4160-8fd9-0f6ca82d53a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:26.846291 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960640f-23df-4160-8fd9-0f6ca82d53a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:26.847636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960640f-23df-4160-8fd9-0f6ca82d53a6 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:26.857733 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 606/1209] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:32:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:26.863591 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f143cc3-0af8-4e15-9f40-2325ceb2cae1 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:26.914380 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:26.967164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f143cc3-0af8-4e15-9f40-2325ceb2cae1 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:27.000776 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc85c9c-a8f9-4be0-8c91-12ab211a3c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:27.001568 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc85c9c-a8f9-4be0-8c91-12ab211a3c7e None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:27.001850 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc85c9c-a8f9-4be0-8c91-12ab211a3c7e None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:27.002119 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc85c9c-a8f9-4be0-8c91-12ab211a3c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:27.014833 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc85c9c-a8f9-4be0-8c91-12ab211a3c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:27.016279 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc85c9c-a8f9-4be0-8c91-12ab211a3c7e None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-8bc85c9c-a8f9-4be0-8c91-12ab211a3c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:27.033688 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:27.034857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:27.036259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:27.036259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:27.036259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:27.036259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:27.036259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 604/1210] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:32:27.041076 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960640f-23df-4160-8fd9-0f6ca82d53a6 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:27.139940 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:27.142903 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960640f-23df-4160-8fd9-0f6ca82d53a6 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:27.176317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcf7a9-870b-44b3-9ae9-598a6e8c0feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:27.177001 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcf7a9-870b-44b3-9ae9-598a6e8c0feb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:27.177285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcf7a9-870b-44b3-9ae9-598a6e8c0feb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:27.177562 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcf7a9-870b-44b3-9ae9-598a6e8c0feb None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:27.178288 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bcf7a9-870b-44b3-9ae9-598a6e8c0feb None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:27.178630 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bcf7a9-870b-44b3-9ae9-598a6e8c0feb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:27.180052 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bcf7a9-870b-44b3-9ae9-598a6e8c0feb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:27.191677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 607/1211] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:27.197360 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc85c9c-a8f9-4be0-8c91-12ab211a3c7e None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:27.299545 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc85c9c-a8f9-4be0-8c91-12ab211a3c7e None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:27.301169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:27.332839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48f0fa-1cd7-42d7-a520-c7f56a623de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:27.333642 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48f0fa-1cd7-42d7-a520-c7f56a623de1 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:27.333923 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48f0fa-1cd7-42d7-a520-c7f56a623de1 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:27.334194 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a48f0fa-1cd7-42d7-a520-c7f56a623de1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:27.341553 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48f0fa-1cd7-42d7-a520-c7f56a623de1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:27.342984 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a48f0fa-1cd7-42d7-a520-c7f56a623de1 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-5a48f0fa-1cd7-42d7-a520-c7f56a623de1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:32:27.357938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:27.359067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:27.360148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:27.360148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:27.360148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:32:27.360148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:32:27.360148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:32:27.360148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:32:27.360148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:32:27.360148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:32:27.360148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:27.360148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:27.360148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 605/1212] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:27.370447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcf7a9-870b-44b3-9ae9-598a6e8c0feb None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:27.469134 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bcf7a9-870b-44b3-9ae9-598a6e8c0feb None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:27.502523 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1f72a-f4bb-4d8f-9f10-4db6597bef43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:27.503200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1f72a-f4bb-4d8f-9f10-4db6597bef43 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:27.503485 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1f72a-f4bb-4d8f-9f10-4db6597bef43 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:27.503763 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1f72a-f4bb-4d8f-9f10-4db6597bef43 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:27.504640 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1f72a-f4bb-4d8f-9f10-4db6597bef43 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:27.509636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1f72a-f4bb-4d8f-9f10-4db6597bef43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:27.509636 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1f72a-f4bb-4d8f-9f10-4db6597bef43 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:27.525080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 608/1213] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:32:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:27.531580 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48f0fa-1cd7-42d7-a520-c7f56a623de1 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:27.635942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a48f0fa-1cd7-42d7-a520-c7f56a623de1 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:27.669867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02b1e-ed59-4194-b7e5-b02c85fe9e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:27.670513 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02b1e-ed59-4194-b7e5-b02c85fe9e98 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:27.670800 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02b1e-ed59-4194-b7e5-b02c85fe9e98 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:27.671080 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe02b1e-ed59-4194-b7e5-b02c85fe9e98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:27.701981 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:27.739111 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe02b1e-ed59-4194-b7e5-b02c85fe9e98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:27.740598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe02b1e-ed59-4194-b7e5-b02c85fe9e98 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:27.840195 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 606/1214] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:32:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:32:27.846126 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1f72a-f4bb-4d8f-9f10-4db6597bef43 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:27.865921 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:27.945778 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1f72a-f4bb-4d8f-9f10-4db6597bef43 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:27.980154 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8b017-e031-4340-a21f-2e412da265bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:27.980951 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8b017-e031-4340-a21f-2e412da265bc None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:27.981859 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8b017-e031-4340-a21f-2e412da265bc None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:27.982045 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8b017-e031-4340-a21f-2e412da265bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:27.991981 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8b017-e031-4340-a21f-2e412da265bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:27.993466 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8b017-e031-4340-a21f-2e412da265bc None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-eec8b017-e031-4340-a21f-2e412da265bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:28.011215 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:28.012391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:28.013451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:28.013451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:28.013451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:28.013451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:28.013451 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 609/1215] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:28.018367 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02b1e-ed59-4194-b7e5-b02c85fe9e98 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:28.079411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:28.117724 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe02b1e-ed59-4194-b7e5-b02c85fe9e98 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:28.156168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa2656-c83f-4242-814b-cb7871cbcbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:28.157164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa2656-c83f-4242-814b-cb7871cbcbb3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:28.157744 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa2656-c83f-4242-814b-cb7871cbcbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:28.157998 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa2656-c83f-4242-814b-cb7871cbcbb3 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:28.158767 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa2656-c83f-4242-814b-cb7871cbcbb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:28.159657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa2656-c83f-4242-814b-cb7871cbcbb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:28.161140 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa2656-c83f-4242-814b-cb7871cbcbb3 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:28.175587 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 607/1216] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:28.181832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8b017-e031-4340-a21f-2e412da265bc None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:28.260152 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:28.295196 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8b017-e031-4340-a21f-2e412da265bc None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:28.333702 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eeda11-7f5b-42d0-9082-26560af9a308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:28.335266 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeda11-7f5b-42d0-9082-26560af9a308 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:28.335266 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeda11-7f5b-42d0-9082-26560af9a308 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:28.335266 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eeda11-7f5b-42d0-9082-26560af9a308 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:28.343252 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eeda11-7f5b-42d0-9082-26560af9a308 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:28.344749 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eeda11-7f5b-42d0-9082-26560af9a308 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-97eeda11-7f5b-42d0-9082-26560af9a308 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:32:28.360382 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:28.361589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:28.363044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:32:28.363044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:32:28.363044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:32:28.363044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:32:28.363044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:32:28.363044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:32:28.363044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:28.363044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:28.363044 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 610/1217] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:32:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:28.367699 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa2656-c83f-4242-814b-cb7871cbcbb3 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:28.477207 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa2656-c83f-4242-814b-cb7871cbcbb3 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:28.502296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:28.514501 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32fda5-6909-4613-b77f-f532ce5bffd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:28.515090 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32fda5-6909-4613-b77f-f532ce5bffd8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:28.515460 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32fda5-6909-4613-b77f-f532ce5bffd8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:28.515660 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32fda5-6909-4613-b77f-f532ce5bffd8 None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:28.516519 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32fda5-6909-4613-b77f-f532ce5bffd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:28.523112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32fda5-6909-4613-b77f-f532ce5bffd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:28.523112 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32fda5-6909-4613-b77f-f532ce5bffd8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:28.532282 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 608/1218] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:32:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:28.538513 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eeda11-7f5b-42d0-9082-26560af9a308 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:28.647064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eeda11-7f5b-42d0-9082-26560af9a308 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:28.677340 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:28.683475 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4553cf-1a84-4cdc-9c96-82abf3ba4a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:28.684145 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4553cf-1a84-4cdc-9c96-82abf3ba4a77 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:28.684479 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4553cf-1a84-4cdc-9c96-82abf3ba4a77 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:28.684757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4553cf-1a84-4cdc-9c96-82abf3ba4a77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:28.685606 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4553cf-1a84-4cdc-9c96-82abf3ba4a77 None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=955b371144784fea84ee7a90886c2d0f) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:28.685926 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4553cf-1a84-4cdc-9c96-82abf3ba4a77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:28.687363 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4553cf-1a84-4cdc-9c96-82abf3ba4a77 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:28.693824 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 611/1219] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:32:28 2022] GET /identity/v3/role_assignments?user.id=955b371144784fea84ee7a90886c2d0f&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:30.947310 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca86b-7ee3-4476-bc43-39c68c862c12 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:30.947912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca86b-7ee3-4476-bc43-39c68c862c12 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:30.947912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aca86b-7ee3-4476-bc43-39c68c862c12 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:30.949277 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 609/1220] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:32:30.955230 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfeea70-6fbf-4358-bbb0-7b8b596792cc None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:30.955642 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfeea70-6fbf-4358-bbb0-7b8b596792cc None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:30.955771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfeea70-6fbf-4358-bbb0-7b8b596792cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:30.972935 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-2dfeea70-6fbf-4358-bbb0-7b8b596792cc None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:32:30.974204 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-2dfeea70-6fbf-4358-bbb0-7b8b596792cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:32:31.043287 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 612/1221] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:32:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:32:31.067298 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aca86b-7ee3-4476-bc43-39c68c862c12 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:31.171515 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aca86b-7ee3-4476-bc43-39c68c862c12 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:31.205418 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1ca7c-d4c5-4506-b2eb-715cf3b9f08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:31.206161 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1ca7c-d4c5-4506-b2eb-715cf3b9f08d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:31.206449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1ca7c-d4c5-4506-b2eb-715cf3b9f08d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:31.206727 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1ca7c-d4c5-4506-b2eb-715cf3b9f08d None admin] PATH_INFO: `/v3/roles/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:31.217339 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c1ca7c-d4c5-4506-b2eb-715cf3b9f08d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-a2c1ca7c-d4c5-4506-b2eb-715cf3b9f08d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:32:31.225164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:31.226449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:31.227836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 610/1222] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:32:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:31.233034 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfeea70-6fbf-4358-bbb0-7b8b596792cc None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:31.334852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfeea70-6fbf-4358-bbb0-7b8b596792cc None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:31.358976 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:31.369314 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93fb27-4cca-4b57-a3ba-86b75f75da0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:31.369982 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93fb27-4cca-4b57-a3ba-86b75f75da0d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:31.370278 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93fb27-4cca-4b57-a3ba-86b75f75da0d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:31.370540 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93fb27-4cca-4b57-a3ba-86b75f75da0d None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:31.371293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93fb27-4cca-4b57-a3ba-86b75f75da0d None admin] RBAC: Adding query filter params (name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:31.371599 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93fb27-4cca-4b57-a3ba-86b75f75da0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:31.373006 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93fb27-4cca-4b57-a3ba-86b75f75da0d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:31.380821 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 613/1223] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:32:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:31.386848 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1ca7c-d4c5-4506-b2eb-715cf3b9f08d None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:31.431184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:31.490758 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1ca7c-d4c5-4506-b2eb-715cf3b9f08d None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:31.523992 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3533ee-7fd9-46b9-a45a-850f0ca0e136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:31.524851 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3533ee-7fd9-46b9-a45a-850f0ca0e136 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:31.525134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3533ee-7fd9-46b9-a45a-850f0ca0e136 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:31.525412 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3533ee-7fd9-46b9-a45a-850f0ca0e136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:31.536177 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3533ee-7fd9-46b9-a45a-850f0ca0e136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:31.537679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3533ee-7fd9-46b9-a45a-850f0ca0e136 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-6c3533ee-7fd9-46b9-a45a-850f0ca0e136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:31.555874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:31.557423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:31.558573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:31.558573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:31.558573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:31.558573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:31.558573 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 611/1224] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:31.562944 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93fb27-4cca-4b57-a3ba-86b75f75da0d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:31.664645 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93fb27-4cca-4b57-a3ba-86b75f75da0d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:31.699114 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b04523-1053-44cb-b0a0-6f3b8c0f8ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:31.699833 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b04523-1053-44cb-b0a0-6f3b8c0f8ef6 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:31.700114 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b04523-1053-44cb-b0a0-6f3b8c0f8ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:31.700420 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b04523-1053-44cb-b0a0-6f3b8c0f8ef6 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:31.701357 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b04523-1053-44cb-b0a0-6f3b8c0f8ef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:31.701611 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b04523-1053-44cb-b0a0-6f3b8c0f8ef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:31.702977 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b04523-1053-44cb-b0a0-6f3b8c0f8ef6 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:31.715156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 614/1225] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:31.721654 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3533ee-7fd9-46b9-a45a-850f0ca0e136 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:31.828045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:31.835509 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3533ee-7fd9-46b9-a45a-850f0ca0e136 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:31.876679 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d1655-208c-4f2b-b00d-9eb3ef0fdc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:31.877740 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d1655-208c-4f2b-b00d-9eb3ef0fdc0a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:31.877914 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d1655-208c-4f2b-b00d-9eb3ef0fdc0a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:31.878274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d1655-208c-4f2b-b00d-9eb3ef0fdc0a None admin] PATH_INFO: `/v3/users/placement` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:31.886390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d1655-208c-4f2b-b00d-9eb3ef0fdc0a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:31.888541 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d1655-208c-4f2b-b00d-9eb3ef0fdc0a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-c79d1655-208c-4f2b-b00d-9eb3ef0fdc0a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:32:31.904711 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:31.907657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:31.908771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:31.908771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:31.908771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:32:31.908771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:32:31.908771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:32:31.908771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:32:31.908771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:32:31.908771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:32:31.908771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:31.908771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:31.908771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 612/1226] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:31.912292 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b04523-1053-44cb-b0a0-6f3b8c0f8ef6 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:31.964174 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:32.020682 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b04523-1053-44cb-b0a0-6f3b8c0f8ef6 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:32.056083 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f1683-be9a-4c46-b4bd-099cc9854908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:32.056810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f1683-be9a-4c46-b4bd-099cc9854908 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:32.057092 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f1683-be9a-4c46-b4bd-099cc9854908 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:32.057363 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f1683-be9a-4c46-b4bd-099cc9854908 None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:32.058347 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f1683-be9a-4c46-b4bd-099cc9854908 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:32.058662 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f1683-be9a-4c46-b4bd-099cc9854908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:32.060107 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f1683-be9a-4c46-b4bd-099cc9854908 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:32.078398 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 615/1227] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:32:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:32.083928 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d1655-208c-4f2b-b00d-9eb3ef0fdc0a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:32.189487 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d1655-208c-4f2b-b00d-9eb3ef0fdc0a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:32.224316 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafe623-9f76-4b29-9b38-3b97c63d5b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:32.225143 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafe623-9f76-4b29-9b38-3b97c63d5b63 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:32.225427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafe623-9f76-4b29-9b38-3b97c63d5b63 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:32.225702 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafe623-9f76-4b29-9b38-3b97c63d5b63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:32.236030 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafe623-9f76-4b29-9b38-3b97c63d5b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:32.237522 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafe623-9f76-4b29-9b38-3b97c63d5b63 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-4eafe623-9f76-4b29-9b38-3b97c63d5b63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:32.255486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:32.269341 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:32.270549 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:32.270549 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:32.270549 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:32.270549 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:32.270549 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 613/1228] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:32.270924 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f1683-be9a-4c46-b4bd-099cc9854908 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:32.366982 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f1683-be9a-4c46-b4bd-099cc9854908 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:32.369260 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:32.402235 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be6e04-90c0-4a19-8861-183831696ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:32.402907 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be6e04-90c0-4a19-8861-183831696ecb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:32.403195 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be6e04-90c0-4a19-8861-183831696ecb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:32.403476 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be6e04-90c0-4a19-8861-183831696ecb None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:32.404210 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be6e04-90c0-4a19-8861-183831696ecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:32.404551 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be6e04-90c0-4a19-8861-183831696ecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:32.405923 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be6e04-90c0-4a19-8861-183831696ecb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:32.418062 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 616/1229] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:32:32.423193 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafe623-9f76-4b29-9b38-3b97c63d5b63 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:32.432216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:32.528455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafe623-9f76-4b29-9b38-3b97c63d5b63 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:32.563289 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b477b-da89-48f9-8977-de951853002f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:32.564050 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b477b-da89-48f9-8977-de951853002f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:32.564329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b477b-da89-48f9-8977-de951853002f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:32.564629 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b477b-da89-48f9-8977-de951853002f None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:32.571306 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b477b-da89-48f9-8977-de951853002f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:32.572775 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b477b-da89-48f9-8977-de951853002f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-b59b477b-da89-48f9-8977-de951853002f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:32:32.586432 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:32.587534 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:32.589124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:32:32.589124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:32:32.589124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:32:32.589124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:32:32.589124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:32:32.589124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:32:32.589124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:32.589124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:32.589124 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 614/1230] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:32:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:32.593702 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be6e04-90c0-4a19-8861-183831696ecb None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:32.694492 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be6e04-90c0-4a19-8861-183831696ecb None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:32.727601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c4ad5-4e15-4b34-b9fd-bf27051537b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:32.728259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4ad5-4e15-4b34-b9fd-bf27051537b4 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:32.728575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4ad5-4e15-4b34-b9fd-bf27051537b4 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:32.728854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c4ad5-4e15-4b34-b9fd-bf27051537b4 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:32.729645 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c4ad5-4e15-4b34-b9fd-bf27051537b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:32.729957 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c4ad5-4e15-4b34-b9fd-bf27051537b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:32.731911 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c4ad5-4e15-4b34-b9fd-bf27051537b4 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:32.743985 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 617/1231] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:32:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:32.749976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b477b-da89-48f9-8977-de951853002f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:32.821292 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:32.857637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b477b-da89-48f9-8977-de951853002f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:32.892979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad19804-22f9-429e-ba67-4ee2652b7759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:32.893873 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad19804-22f9-429e-ba67-4ee2652b7759 None admin] REQUEST_METHOD: `PUT` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:32.894156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad19804-22f9-429e-ba67-4ee2652b7759 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:32.894431 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad19804-22f9-429e-ba67-4ee2652b7759 None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/955b371144784fea84ee7a90886c2d0f/roles/ec2a4317dc7343208d439d5f2d44d6c3` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:32.909296 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad19804-22f9-429e-ba67-4ee2652b7759 None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=955b371144784fea84ee7a90886c2d0f, role_id=ec2a4317dc7343208d439d5f2d44d6c3)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:32.910758 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad19804-22f9-429e-ba67-4ee2652b7759 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:33.057364 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 615/1232] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:32:32 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/955b371144784fea84ee7a90886c2d0f/roles/ec2a4317dc7343208d439d5f2d44d6c3 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:32:35.301542 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb097db-7ef3-44d7-b640-509bb03e90a3 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:35.302109 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb097db-7ef3-44d7-b640-509bb03e90a3 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:35.302109 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb097db-7ef3-44d7-b640-509bb03e90a3 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:35.303456 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 618/1233] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:32:35.308955 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf3906-ec37-4e23-b18e-1e624e83b447 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:35.310005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf3906-ec37-4e23-b18e-1e624e83b447 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:35.310005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf3906-ec37-4e23-b18e-1e624e83b447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:35.327527 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-addf3906-ec37-4e23-b18e-1e624e83b447 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:32:35.328822 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-addf3906-ec37-4e23-b18e-1e624e83b447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:32:35.397508 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 616/1234] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:32:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:32:35.421406 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb097db-7ef3-44d7-b640-509bb03e90a3 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:35.452787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:35.529464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb097db-7ef3-44d7-b640-509bb03e90a3 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:35.567984 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc886a3a-304d-4fee-a272-f8c7c5fb5d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:35.568748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc886a3a-304d-4fee-a272-f8c7c5fb5d06 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:35.569031 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc886a3a-304d-4fee-a272-f8c7c5fb5d06 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:35.569305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc886a3a-304d-4fee-a272-f8c7c5fb5d06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:35.579914 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc886a3a-304d-4fee-a272-f8c7c5fb5d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-cc886a3a-304d-4fee-a272-f8c7c5fb5d06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:32:35.587607 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:35.588854 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:32:35.589832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:35.592500 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 619/1235] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:32:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:35.595299 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf3906-ec37-4e23-b18e-1e624e83b447 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:35.644386 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:35.697491 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf3906-ec37-4e23-b18e-1e624e83b447 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:35.733556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96422f3-0a21-4072-8964-664f329ab3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:35.734369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96422f3-0a21-4072-8964-664f329ab3ce None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:35.734657 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96422f3-0a21-4072-8964-664f329ab3ce None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:35.734937 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96422f3-0a21-4072-8964-664f329ab3ce None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:35.735686 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96422f3-0a21-4072-8964-664f329ab3ce None admin] RBAC: Adding query filter params (name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:35.735997 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96422f3-0a21-4072-8964-664f329ab3ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:35.737447 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96422f3-0a21-4072-8964-664f329ab3ce None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:35.745439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 617/1236] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:32:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:35.751212 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc886a3a-304d-4fee-a272-f8c7c5fb5d06 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:35.837000 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:35.853313 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc886a3a-304d-4fee-a272-f8c7c5fb5d06 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:35.889665 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380e1e-4726-4c92-b1a9-a1bd0c52ce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:35.890454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380e1e-4726-4c92-b1a9-a1bd0c52ce64 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:35.890740 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380e1e-4726-4c92-b1a9-a1bd0c52ce64 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:35.891014 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380e1e-4726-4c92-b1a9-a1bd0c52ce64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:35.900979 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24380e1e-4726-4c92-b1a9-a1bd0c52ce64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:35.907049 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24380e1e-4726-4c92-b1a9-a1bd0c52ce64 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-24380e1e-4726-4c92-b1a9-a1bd0c52ce64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:35.921202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:35.922574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:35.923910 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:35.923910 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:35.923910 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:35.923910 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:35.923910 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 620/1237] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:35.928242 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96422f3-0a21-4072-8964-664f329ab3ce None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:36.026862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96422f3-0a21-4072-8964-664f329ab3ce None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:36.036171 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:36.061354 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28dcde-2487-46de-ba53-4d519ea96985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:36.062017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28dcde-2487-46de-ba53-4d519ea96985 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:36.062307 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28dcde-2487-46de-ba53-4d519ea96985 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:36.062629 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa28dcde-2487-46de-ba53-4d519ea96985 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:36.063348 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28dcde-2487-46de-ba53-4d519ea96985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:36.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28dcde-2487-46de-ba53-4d519ea96985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:36.065046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa28dcde-2487-46de-ba53-4d519ea96985 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:36.076602 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 618/1238] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:32:36.082447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380e1e-4726-4c92-b1a9-a1bd0c52ce64 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:36.181511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380e1e-4726-4c92-b1a9-a1bd0c52ce64 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:36.214695 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02397-dd97-4135-a6bc-bbb0bb982ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:36.216033 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02397-dd97-4135-a6bc-bbb0bb982ae0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:36.216358 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02397-dd97-4135-a6bc-bbb0bb982ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:36.216642 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e02397-dd97-4135-a6bc-bbb0bb982ae0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:36.219404 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:36.223968 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e02397-dd97-4135-a6bc-bbb0bb982ae0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:36.225581 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e02397-dd97-4135-a6bc-bbb0bb982ae0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-e9e02397-dd97-4135-a6bc-bbb0bb982ae0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:32:36.240687 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:36.241941 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:36.243259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:36.243259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:36.243259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:32:36.243259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:32:36.243259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:32:36.243259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:32:36.243259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:32:36.243259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:32:36.243259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:36.243259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:36.243259 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 621/1239] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:36.247888 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28dcde-2487-46de-ba53-4d519ea96985 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:36.350505 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa28dcde-2487-46de-ba53-4d519ea96985 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:36.385048 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3241be-a201-47cb-bbcf-26c75124cd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:36.387730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3241be-a201-47cb-bbcf-26c75124cd4f None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:36.387730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3241be-a201-47cb-bbcf-26c75124cd4f None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:36.387730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3241be-a201-47cb-bbcf-26c75124cd4f None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:36.387730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3241be-a201-47cb-bbcf-26c75124cd4f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:36.387730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3241be-a201-47cb-bbcf-26c75124cd4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:36.388977 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3241be-a201-47cb-bbcf-26c75124cd4f None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:36.405861 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 619/1240] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:32:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:36.412031 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02397-dd97-4135-a6bc-bbb0bb982ae0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:36.514333 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e02397-dd97-4135-a6bc-bbb0bb982ae0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:36.548403 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ed453-c082-4610-8f39-ab7a21c9f9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:36.549049 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ed453-c082-4610-8f39-ab7a21c9f9e0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:36.549332 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ed453-c082-4610-8f39-ab7a21c9f9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:36.549601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ed453-c082-4610-8f39-ab7a21c9f9e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:36.610052 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:36.616626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ed453-c082-4610-8f39-ab7a21c9f9e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:36.618066 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ed453-c082-4610-8f39-ab7a21c9f9e0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:36.716208 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 622/1241] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:32:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:32:36.721429 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3241be-a201-47cb-bbcf-26c75124cd4f None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:36.749844 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:36.824422 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3241be-a201-47cb-bbcf-26c75124cd4f None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:36.860023 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453be85-3a06-4589-8cdc-e19271a90432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:36.860882 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453be85-3a06-4589-8cdc-e19271a90432 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:36.861160 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453be85-3a06-4589-8cdc-e19271a90432 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:36.861434 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453be85-3a06-4589-8cdc-e19271a90432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:36.872218 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453be85-3a06-4589-8cdc-e19271a90432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:36.874198 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453be85-3a06-4589-8cdc-e19271a90432 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-5453be85-3a06-4589-8cdc-e19271a90432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:36.892526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:36.893720 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:36.894783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:36.894783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:36.894783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:36.894783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:36.894783 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 620/1242] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:36.899473 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ed453-c082-4610-8f39-ab7a21c9f9e0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:36.993111 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:36.999386 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ed453-c082-4610-8f39-ab7a21c9f9e0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:37.032067 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e768cf-be17-48ed-ade3-a3f1c800cb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:37.032793 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e768cf-be17-48ed-ade3-a3f1c800cb85 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:37.033076 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e768cf-be17-48ed-ade3-a3f1c800cb85 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:37.033352 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e768cf-be17-48ed-ade3-a3f1c800cb85 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:37.034075 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e768cf-be17-48ed-ade3-a3f1c800cb85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:37.034384 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e768cf-be17-48ed-ade3-a3f1c800cb85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:37.035748 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e768cf-be17-48ed-ade3-a3f1c800cb85 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:37.047565 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 623/1243] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:37.053485 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453be85-3a06-4589-8cdc-e19271a90432 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:37.142937 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:37.165274 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453be85-3a06-4589-8cdc-e19271a90432 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:37.197964 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76248d6-5119-45c0-927d-7440150e1f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:37.198798 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76248d6-5119-45c0-927d-7440150e1f0c None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:37.199083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76248d6-5119-45c0-927d-7440150e1f0c None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:37.199356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76248d6-5119-45c0-927d-7440150e1f0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:37.206594 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76248d6-5119-45c0-927d-7440150e1f0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:37.208035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76248d6-5119-45c0-927d-7440150e1f0c None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-d76248d6-5119-45c0-927d-7440150e1f0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:32:37.222184 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:37.223356 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:37.224639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:32:37.224639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:32:37.224639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:32:37.224639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:32:37.224639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:32:37.224639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:32:37.224639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:37.224639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:37.224639 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 621/1244] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:32:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:37.229266 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e768cf-be17-48ed-ade3-a3f1c800cb85 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:37.329040 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e768cf-be17-48ed-ade3-a3f1c800cb85 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:37.362194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf22b70-c6ec-40a2-82a4-75b1354a6963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:37.362852 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22b70-c6ec-40a2-82a4-75b1354a6963 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:37.363136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22b70-c6ec-40a2-82a4-75b1354a6963 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:37.363503 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22b70-c6ec-40a2-82a4-75b1354a6963 None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:37.364232 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf22b70-c6ec-40a2-82a4-75b1354a6963 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:37.364579 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf22b70-c6ec-40a2-82a4-75b1354a6963 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:37.366032 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf22b70-c6ec-40a2-82a4-75b1354a6963 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:37.378639 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 624/1245] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:32:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:37.385026 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76248d6-5119-45c0-927d-7440150e1f0c None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:37.503879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76248d6-5119-45c0-927d-7440150e1f0c None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:37.537652 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff7015-1f5e-4319-a449-df1423a10384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:37.538305 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff7015-1f5e-4319-a449-df1423a10384 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:37.538845 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff7015-1f5e-4319-a449-df1423a10384 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:37.539118 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff7015-1f5e-4319-a449-df1423a10384 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:37.539938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff7015-1f5e-4319-a449-df1423a10384 None admin] RBAC: Adding query filter params (role.id=ec2a4317dc7343208d439d5f2d44d6c3, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=955b371144784fea84ee7a90886c2d0f) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:37.540250 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff7015-1f5e-4319-a449-df1423a10384 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:37.541862 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff7015-1f5e-4319-a449-df1423a10384 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:37.547738 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 622/1246] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:32:37 2022] GET /identity/v3/role_assignments?user.id=955b371144784fea84ee7a90886c2d0f&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=ec2a4317dc7343208d439d5f2d44d6c3 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:39.913338 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07695a7-151a-4b59-b7df-b11c0fce7c75 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:39.914181 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07695a7-151a-4b59-b7df-b11c0fce7c75 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:39.914181 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07695a7-151a-4b59-b7df-b11c0fce7c75 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:39.915524 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 625/1247] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:32:39.921774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228d8f-029c-4524-8b9f-a2a37d3eb022 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:39.925716 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228d8f-029c-4524-8b9f-a2a37d3eb022 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:39.925716 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da228d8f-029c-4524-8b9f-a2a37d3eb022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:39.927460 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:39.939358 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-da228d8f-029c-4524-8b9f-a2a37d3eb022 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:32:39.940582 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-da228d8f-029c-4524-8b9f-a2a37d3eb022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:32:40.007064 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 623/1248] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:32:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:32:40.032119 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07695a7-151a-4b59-b7df-b11c0fce7c75 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:40.050465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:40.131912 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07695a7-151a-4b59-b7df-b11c0fce7c75 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:40.166684 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50e251-b589-47d5-bd96-e37eff913af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:40.167406 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50e251-b589-47d5-bd96-e37eff913af5 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:40.167693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50e251-b589-47d5-bd96-e37eff913af5 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:40.167969 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50e251-b589-47d5-bd96-e37eff913af5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:40.178087 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc50e251-b589-47d5-bd96-e37eff913af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-bc50e251-b589-47d5-bd96-e37eff913af5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:32:40.185826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:40.187053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:40.188385 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 626/1249] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:32:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:40.193147 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da228d8f-029c-4524-8b9f-a2a37d3eb022 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:40.295193 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da228d8f-029c-4524-8b9f-a2a37d3eb022 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:40.324495 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:40.329506 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b422f6c-202d-46f2-aa8b-d65dfa4444f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:40.330173 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b422f6c-202d-46f2-aa8b-d65dfa4444f8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:40.330458 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b422f6c-202d-46f2-aa8b-d65dfa4444f8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:40.330737 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b422f6c-202d-46f2-aa8b-d65dfa4444f8 None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:40.331485 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b422f6c-202d-46f2-aa8b-d65dfa4444f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:40.331797 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b422f6c-202d-46f2-aa8b-d65dfa4444f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:40.333219 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b422f6c-202d-46f2-aa8b-d65dfa4444f8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:40.341837 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 624/1250] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:32:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:40.347617 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50e251-b589-47d5-bd96-e37eff913af5 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:40.427010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:40.449969 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50e251-b589-47d5-bd96-e37eff913af5 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:40.484454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ef1c7-a814-46be-b116-fd853e092a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:40.485282 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ef1c7-a814-46be-b116-fd853e092a93 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:40.485563 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ef1c7-a814-46be-b116-fd853e092a93 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:40.485831 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ef1c7-a814-46be-b116-fd853e092a93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:40.496876 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ef1c7-a814-46be-b116-fd853e092a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:40.504492 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ef1c7-a814-46be-b116-fd853e092a93 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-ac8ef1c7-a814-46be-b116-fd853e092a93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:40.515575 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:40.516770 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:40.518177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:40.518177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:40.518177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:40.518177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:40.518177 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 627/1251] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:40.522796 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b422f6c-202d-46f2-aa8b-d65dfa4444f8 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:40.622819 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b422f6c-202d-46f2-aa8b-d65dfa4444f8 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:40.659373 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a684e7-5b48-465a-a3cb-1c1d30ae01f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:40.664975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a684e7-5b48-465a-a3cb-1c1d30ae01f0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:40.664975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a684e7-5b48-465a-a3cb-1c1d30ae01f0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:40.664975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a684e7-5b48-465a-a3cb-1c1d30ae01f0 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:40.664975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a684e7-5b48-465a-a3cb-1c1d30ae01f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:40.664975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a684e7-5b48-465a-a3cb-1c1d30ae01f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:40.664975 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a684e7-5b48-465a-a3cb-1c1d30ae01f0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:40.676210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 625/1252] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:32:40.682468 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ef1c7-a814-46be-b116-fd853e092a93 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:40.786152 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ef1c7-a814-46be-b116-fd853e092a93 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:40.814175 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:40.822705 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209504e6-377e-487a-a465-4e2cafb2f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:40.823522 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209504e6-377e-487a-a465-4e2cafb2f54d None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:40.823812 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209504e6-377e-487a-a465-4e2cafb2f54d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:40.824092 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209504e6-377e-487a-a465-4e2cafb2f54d None admin] PATH_INFO: `/v3/users/placement` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:40.833053 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209504e6-377e-487a-a465-4e2cafb2f54d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:40.834511 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209504e6-377e-487a-a465-4e2cafb2f54d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-209504e6-377e-487a-a465-4e2cafb2f54d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:32:40.850531 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:40.851908 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:40.853035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:40.853035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:40.853035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:32:40.853035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:32:40.853035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:32:40.853035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:32:40.853035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:32:40.853035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:32:40.853035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:40.853035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:40.853035 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 628/1253] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:40.857798 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a684e7-5b48-465a-a3cb-1c1d30ae01f0 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:40.914904 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:40.959518 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a684e7-5b48-465a-a3cb-1c1d30ae01f0 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:40.993850 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ba299-5b41-4b8d-bc57-dc8f86d8c696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:40.994521 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ba299-5b41-4b8d-bc57-dc8f86d8c696 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:40.994806 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ba299-5b41-4b8d-bc57-dc8f86d8c696 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:40.995082 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ba299-5b41-4b8d-bc57-dc8f86d8c696 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:40.995916 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ba299-5b41-4b8d-bc57-dc8f86d8c696 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:40.996247 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ba299-5b41-4b8d-bc57-dc8f86d8c696 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:40.997726 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ba299-5b41-4b8d-bc57-dc8f86d8c696 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:41.015474 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 626/1254] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:32:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:41.021202 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209504e6-377e-487a-a465-4e2cafb2f54d None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:41.122194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209504e6-377e-487a-a465-4e2cafb2f54d None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:41.157587 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61099e-407e-4794-945f-3214f5042b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:41.158359 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61099e-407e-4794-945f-3214f5042b0f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:41.158641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61099e-407e-4794-945f-3214f5042b0f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:41.158911 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61099e-407e-4794-945f-3214f5042b0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:41.215325 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:41.227548 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61099e-407e-4794-945f-3214f5042b0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:41.229023 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61099e-407e-4794-945f-3214f5042b0f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:41.328811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 629/1255] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:32:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:32:41.334816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ba299-5b41-4b8d-bc57-dc8f86d8c696 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:41.437683 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ba299-5b41-4b8d-bc57-dc8f86d8c696 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:41.454115 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:41.476065 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72271c-61c6-4873-8a50-6663a1103884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:41.476832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72271c-61c6-4873-8a50-6663a1103884 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:41.477117 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72271c-61c6-4873-8a50-6663a1103884 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:41.477396 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72271c-61c6-4873-8a50-6663a1103884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:41.488390 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72271c-61c6-4873-8a50-6663a1103884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:41.492489 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72271c-61c6-4873-8a50-6663a1103884 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-4b72271c-61c6-4873-8a50-6663a1103884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:41.507561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:41.508730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:41.510210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:41.510210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:41.510210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:41.510210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:41.510210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 627/1256] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:41.515165 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61099e-407e-4794-945f-3214f5042b0f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:41.604761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:41.618506 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61099e-407e-4794-945f-3214f5042b0f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:41.656905 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c111adf-247c-43e6-b182-3b1583c965e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:41.657574 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c111adf-247c-43e6-b182-3b1583c965e7 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:41.657861 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c111adf-247c-43e6-b182-3b1583c965e7 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:41.658134 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c111adf-247c-43e6-b182-3b1583c965e7 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:41.658863 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c111adf-247c-43e6-b182-3b1583c965e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:41.659170 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c111adf-247c-43e6-b182-3b1583c965e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:41.660590 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c111adf-247c-43e6-b182-3b1583c965e7 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:41.672366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 630/1257] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:41.678479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72271c-61c6-4873-8a50-6663a1103884 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:41.782150 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72271c-61c6-4873-8a50-6663a1103884 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:41.820006 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b945-7b0e-4fbb-ad39-0bcdaed30b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:41.820863 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b945-7b0e-4fbb-ad39-0bcdaed30b95 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:41.821148 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b945-7b0e-4fbb-ad39-0bcdaed30b95 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:41.821422 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b945-7b0e-4fbb-ad39-0bcdaed30b95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:41.828671 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635b945-7b0e-4fbb-ad39-0bcdaed30b95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:41.830120 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635b945-7b0e-4fbb-ad39-0bcdaed30b95 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-a635b945-7b0e-4fbb-ad39-0bcdaed30b95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:32:41.844377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:41.846070 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:41.847839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:32:41.847839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:32:41.847839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:32:41.847839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:32:41.847839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:32:41.847839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:32:41.847839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:41.847839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:41.847839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 628/1258] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:32:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:41.851811 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c111adf-247c-43e6-b182-3b1583c965e7 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:41.953743 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c111adf-247c-43e6-b182-3b1583c965e7 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:41.986909 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76fca0a-7b87-4c42-a794-d70955fbed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:41.987578 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fca0a-7b87-4c42-a794-d70955fbed3c None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:41.987865 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fca0a-7b87-4c42-a794-d70955fbed3c None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:41.988143 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76fca0a-7b87-4c42-a794-d70955fbed3c None admin] PATH_INFO: `/v3/projects` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:41.988971 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76fca0a-7b87-4c42-a794-d70955fbed3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:41.989281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76fca0a-7b87-4c42-a794-d70955fbed3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:41.990704 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76fca0a-7b87-4c42-a794-d70955fbed3c None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:42.003333 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 631/1259] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:32:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:42.009547 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b945-7b0e-4fbb-ad39-0bcdaed30b95 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:42.036078 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:42.114118 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b945-7b0e-4fbb-ad39-0bcdaed30b95 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:42.148818 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14865d4f-e6bb-4dd0-af2b-13186fe5d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:42.149482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14865d4f-e6bb-4dd0-af2b-13186fe5d055 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:42.149761 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14865d4f-e6bb-4dd0-af2b-13186fe5d055 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:42.150035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14865d4f-e6bb-4dd0-af2b-13186fe5d055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:42.150889 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14865d4f-e6bb-4dd0-af2b-13186fe5d055 None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=955b371144784fea84ee7a90886c2d0f) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:42.151180 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14865d4f-e6bb-4dd0-af2b-13186fe5d055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:42.153224 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14865d4f-e6bb-4dd0-af2b-13186fe5d055 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:42.158967 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 629/1260] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:32:42 2022] GET /identity/v3/role_assignments?user.id=955b371144784fea84ee7a90886c2d0f&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:44.387735 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2422c-b1de-4f51-b560-f94ccc41b39b None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:44.388305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2422c-b1de-4f51-b560-f94ccc41b39b None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:44.388429 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2422c-b1de-4f51-b560-f94ccc41b39b None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:44.389864 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 632/1261] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:32:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:32:44.396024 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42477cdc-6b17-4189-bf76-55a0cff33633 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:44.396850 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42477cdc-6b17-4189-bf76-55a0cff33633 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:44.396850 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42477cdc-6b17-4189-bf76-55a0cff33633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:44.414436 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-42477cdc-6b17-4189-bf76-55a0cff33633 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:32:44.415746 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-42477cdc-6b17-4189-bf76-55a0cff33633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:32:44.485460 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 630/1262] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:32:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:32:44.508913 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2422c-b1de-4f51-b560-f94ccc41b39b None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:44.520199 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:44.614275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2422c-b1de-4f51-b560-f94ccc41b39b None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:44.655098 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b0f4e-1675-436f-9ae2-d821dddbbf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:44.656122 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b0f4e-1675-436f-9ae2-d821dddbbf63 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:44.656300 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b0f4e-1675-436f-9ae2-d821dddbbf63 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:44.656871 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b0f4e-1675-436f-9ae2-d821dddbbf63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:44.668536 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b0f4e-1675-436f-9ae2-d821dddbbf63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-0c7b0f4e-1675-436f-9ae2-d821dddbbf63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:32:44.676200 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:44.677444 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:44.678829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 633/1263] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:32:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:44.684355 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42477cdc-6b17-4189-bf76-55a0cff33633 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:44.702947 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:44.791870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42477cdc-6b17-4189-bf76-55a0cff33633 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:44.828651 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b74dd8-35f0-4490-b075-7597c2b6770a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:44.829331 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74dd8-35f0-4490-b075-7597c2b6770a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:44.829613 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74dd8-35f0-4490-b075-7597c2b6770a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:44.829890 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74dd8-35f0-4490-b075-7597c2b6770a None admin] PATH_INFO: `/v3/roles` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:44.830642 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b74dd8-35f0-4490-b075-7597c2b6770a None admin] RBAC: Adding query filter params (name=admin) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:44.830950 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b74dd8-35f0-4490-b075-7597c2b6770a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:44.832466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b74dd8-35f0-4490-b075-7597c2b6770a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:44.841841 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 631/1264] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:32:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:44.848049 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b0f4e-1675-436f-9ae2-d821dddbbf63 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:44.915787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:44.956143 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b0f4e-1675-436f-9ae2-d821dddbbf63 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:44.990440 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18498661-8056-47ee-baa6-3a36321797c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:44.991286 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18498661-8056-47ee-baa6-3a36321797c0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:44.991602 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18498661-8056-47ee-baa6-3a36321797c0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:44.991885 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18498661-8056-47ee-baa6-3a36321797c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:45.002883 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18498661-8056-47ee-baa6-3a36321797c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:45.004365 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18498661-8056-47ee-baa6-3a36321797c0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-18498661-8056-47ee-baa6-3a36321797c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:45.021868 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:45.023118 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:45.024112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:45.024112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:45.024112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:45.024112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:45.025298 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 634/1265] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:45.029600 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b74dd8-35f0-4490-b075-7597c2b6770a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:45.113384 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:45.131992 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b74dd8-35f0-4490-b075-7597c2b6770a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:45.166666 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fc75b-4302-4ad0-a7e9-627a360fb3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:45.167270 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fc75b-4302-4ad0-a7e9-627a360fb3ef None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:45.167551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fc75b-4302-4ad0-a7e9-627a360fb3ef None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:45.167827 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fc75b-4302-4ad0-a7e9-627a360fb3ef None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:45.169159 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fc75b-4302-4ad0-a7e9-627a360fb3ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:45.169474 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fc75b-4302-4ad0-a7e9-627a360fb3ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:45.170846 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7fc75b-4302-4ad0-a7e9-627a360fb3ef None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:45.194374 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 632/1266] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:45.201013 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18498661-8056-47ee-baa6-3a36321797c0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:45.301038 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18498661-8056-47ee-baa6-3a36321797c0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:45.320832 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:45.335142 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a180a17-cac9-4099-bc89-9c1bd5d4ca63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:45.335934 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a180a17-cac9-4099-bc89-9c1bd5d4ca63 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:45.336214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a180a17-cac9-4099-bc89-9c1bd5d4ca63 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:45.336518 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a180a17-cac9-4099-bc89-9c1bd5d4ca63 None admin] PATH_INFO: `/v3/users/placement` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:45.344123 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a180a17-cac9-4099-bc89-9c1bd5d4ca63 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:45.345975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a180a17-cac9-4099-bc89-9c1bd5d4ca63 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-6a180a17-cac9-4099-bc89-9c1bd5d4ca63 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:32:45.360156 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:45.361397 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:45.362713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:45.362713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:45.362713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:32:45.362713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:32:45.362713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:32:45.362713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:32:45.362713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:32:45.362713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:32:45.362713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:45.362713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:45.362713 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 635/1267] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:45.367426 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fc75b-4302-4ad0-a7e9-627a360fb3ef None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:45.468744 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fc75b-4302-4ad0-a7e9-627a360fb3ef None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:45.506440 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcaec9a-4b47-43ef-84fe-a2e4cd2dc057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:45.507142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcaec9a-4b47-43ef-84fe-a2e4cd2dc057 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:45.507424 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcaec9a-4b47-43ef-84fe-a2e4cd2dc057 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:45.507698 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcaec9a-4b47-43ef-84fe-a2e4cd2dc057 None admin] PATH_INFO: `/v3/users` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:45.508563 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcaec9a-4b47-43ef-84fe-a2e4cd2dc057 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:45.508874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcaec9a-4b47-43ef-84fe-a2e4cd2dc057 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:45.510359 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcaec9a-4b47-43ef-84fe-a2e4cd2dc057 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:45.520459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:45.528740 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 633/1268] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:32:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:45.534666 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a180a17-cac9-4099-bc89-9c1bd5d4ca63 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:45.635423 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a180a17-cac9-4099-bc89-9c1bd5d4ca63 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:45.671055 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa9de4-8961-49d4-b1f2-47b38832f4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:45.671844 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa9de4-8961-49d4-b1f2-47b38832f4ce None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:45.672124 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa9de4-8961-49d4-b1f2-47b38832f4ce None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:45.672425 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa9de4-8961-49d4-b1f2-47b38832f4ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:45.682758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa9de4-8961-49d4-b1f2-47b38832f4ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:45.684217 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa9de4-8961-49d4-b1f2-47b38832f4ce None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-32fa9de4-8961-49d4-b1f2-47b38832f4ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:45.700777 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:45.701779 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:45.703085 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:45.703085 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:45.703085 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:45.703085 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:45.703085 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 636/1269] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:45.707614 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcaec9a-4b47-43ef-84fe-a2e4cd2dc057 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:45.809471 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcaec9a-4b47-43ef-84fe-a2e4cd2dc057 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:45.845770 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e27ed1-fe49-4349-ae8e-7570b896e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:45.846429 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e27ed1-fe49-4349-ae8e-7570b896e4bb None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:45.846708 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e27ed1-fe49-4349-ae8e-7570b896e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:45.846979 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e27ed1-fe49-4349-ae8e-7570b896e4bb None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:45.847696 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e27ed1-fe49-4349-ae8e-7570b896e4bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:45.848000 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e27ed1-fe49-4349-ae8e-7570b896e4bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:45.849488 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e27ed1-fe49-4349-ae8e-7570b896e4bb None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:45.861538 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 634/1270] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:45.867374 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa9de4-8961-49d4-b1f2-47b38832f4ce None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:45.900784 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:45.970803 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa9de4-8961-49d4-b1f2-47b38832f4ce None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:46.003942 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c97d53-b1e3-400a-a281-691f500e7f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:46.004761 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c97d53-b1e3-400a-a281-691f500e7f16 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:46.005049 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c97d53-b1e3-400a-a281-691f500e7f16 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:46.005324 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c97d53-b1e3-400a-a281-691f500e7f16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:46.012368 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c97d53-b1e3-400a-a281-691f500e7f16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:46.013814 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c97d53-b1e3-400a-a281-691f500e7f16 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-10c97d53-b1e3-400a-a281-691f500e7f16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:32:46.027838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:46.034421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:46.035736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:32:46.035736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:32:46.035736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:32:46.035736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:32:46.035736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:32:46.035736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:32:46.035736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:46.035736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:46.035736 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 637/1271] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:32:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:46.036438 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e27ed1-fe49-4349-ae8e-7570b896e4bb None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:46.094425 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:46.136144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e27ed1-fe49-4349-ae8e-7570b896e4bb None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:46.170140 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ccb31-2841-44ef-b3d1-f4f980da0f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:46.170781 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ccb31-2841-44ef-b3d1-f4f980da0f5a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:46.171069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ccb31-2841-44ef-b3d1-f4f980da0f5a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:46.171349 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ccb31-2841-44ef-b3d1-f4f980da0f5a None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:46.172136 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ccb31-2841-44ef-b3d1-f4f980da0f5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:46.172526 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ccb31-2841-44ef-b3d1-f4f980da0f5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:46.173980 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ccb31-2841-44ef-b3d1-f4f980da0f5a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:46.187205 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 635/1272] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:32:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:46.193279 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c97d53-b1e3-400a-a281-691f500e7f16 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:46.286037 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:46.294709 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c97d53-b1e3-400a-a281-691f500e7f16 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:46.327791 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd401a8-462a-4d81-9abe-220c12a53f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:46.328706 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd401a8-462a-4d81-9abe-220c12a53f3d None admin] REQUEST_METHOD: `PUT` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:46.329022 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd401a8-462a-4d81-9abe-220c12a53f3d None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:46.329296 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd401a8-462a-4d81-9abe-220c12a53f3d None admin] PATH_INFO: `/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/955b371144784fea84ee7a90886c2d0f/roles/8eaff9cd613a49b29548e90d7e3e9215` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:46.331389 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd401a8-462a-4d81-9abe-220c12a53f3d None admin] RBAC: Authorizing `identity:create_grant(project_id=895c72b9c9614f5dbf18b4255821b478, user_id=955b371144784fea84ee7a90886c2d0f, role_id=8eaff9cd613a49b29548e90d7e3e9215)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:46.332851 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd401a8-462a-4d81-9abe-220c12a53f3d None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:46.482161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 638/1273] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:32:46 2022] PUT /identity/v3/projects/895c72b9c9614f5dbf18b4255821b478/users/955b371144784fea84ee7a90886c2d0f/roles/8eaff9cd613a49b29548e90d7e3e9215 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:32:48.679439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff2535-500b-4d29-86bc-8cad5c0ed2f1 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:48.680014 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff2535-500b-4d29-86bc-8cad5c0ed2f1 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:48.680014 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff2535-500b-4d29-86bc-8cad5c0ed2f1 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:48.681399 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 636/1274] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:32:48.686816 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcd1c7-c504-4e94-836b-766d951821ab None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:48.687242 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcd1c7-c504-4e94-836b-766d951821ab None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:48.687381 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcd1c7-c504-4e94-836b-766d951821ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:48.704533 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-e3bcd1c7-c504-4e94-836b-766d951821ab None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:32:48.705649 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-e3bcd1c7-c504-4e94-836b-766d951821ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:32:48.771720 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 639/1275] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:32:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:32:48.797117 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff2535-500b-4d29-86bc-8cad5c0ed2f1 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:48.898798 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff2535-500b-4d29-86bc-8cad5c0ed2f1 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:48.916800 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:48.933745 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb5262-8a4a-4176-8925-879376053e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:48.934533 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5262-8a4a-4176-8925-879376053e3a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:48.934814 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5262-8a4a-4176-8925-879376053e3a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:48.935081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5262-8a4a-4176-8925-879376053e3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:48.944976 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb5262-8a4a-4176-8925-879376053e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-aadb5262-8a4a-4176-8925-879376053e3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:32:48.952576 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:48.953836 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:48.954874 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 637/1276] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:32:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:48.960011 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcd1c7-c504-4e94-836b-766d951821ab None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:49.043293 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:49.061275 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcd1c7-c504-4e94-836b-766d951821ab None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:49.095768 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b595c7-2b35-4f3d-bb96-9f7d8829c767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:49.096491 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b595c7-2b35-4f3d-bb96-9f7d8829c767 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:49.096995 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b595c7-2b35-4f3d-bb96-9f7d8829c767 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:49.097654 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b595c7-2b35-4f3d-bb96-9f7d8829c767 None admin] PATH_INFO: `/v3/roles` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:49.098516 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b595c7-2b35-4f3d-bb96-9f7d8829c767 None admin] RBAC: Adding query filter params (name=admin) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:49.098833 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b595c7-2b35-4f3d-bb96-9f7d8829c767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:49.100201 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b595c7-2b35-4f3d-bb96-9f7d8829c767 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:49.108656 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 640/1277] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:32:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:49.114558 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb5262-8a4a-4176-8925-879376053e3a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:49.216598 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb5262-8a4a-4176-8925-879376053e3a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:49.259081 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77d294-eab0-442a-b849-5e27e99b671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:49.259867 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77d294-eab0-442a-b849-5e27e99b671a None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:49.260156 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77d294-eab0-442a-b849-5e27e99b671a None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:49.260467 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77d294-eab0-442a-b849-5e27e99b671a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:49.271254 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77d294-eab0-442a-b849-5e27e99b671a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:49.272716 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77d294-eab0-442a-b849-5e27e99b671a None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-ec77d294-eab0-442a-b849-5e27e99b671a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:49.289921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:49.290930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:49.292067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:49.292067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:49.292067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:49.292067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:49.292067 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 638/1278] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:49.297546 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b595c7-2b35-4f3d-bb96-9f7d8829c767 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:49.397011 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b595c7-2b35-4f3d-bb96-9f7d8829c767 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:49.431253 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88e039-23f1-480d-afe3-805557342ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:49.431939 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88e039-23f1-480d-afe3-805557342ed0 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:49.432189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88e039-23f1-480d-afe3-805557342ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:49.432492 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88e039-23f1-480d-afe3-805557342ed0 None admin] PATH_INFO: `/v3/domains` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:49.433214 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88e039-23f1-480d-afe3-805557342ed0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:49.433582 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88e039-23f1-480d-afe3-805557342ed0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:49.435130 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88e039-23f1-480d-afe3-805557342ed0 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:49.447354 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 641/1279] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:49.452952 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77d294-eab0-442a-b849-5e27e99b671a None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:49.485032 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:49.558142 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77d294-eab0-442a-b849-5e27e99b671a None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:49.591812 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ecbb3-d4c1-48c0-9372-c1af79f7e327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:49.592637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ecbb3-d4c1-48c0-9372-c1af79f7e327 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:49.592922 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ecbb3-d4c1-48c0-9372-c1af79f7e327 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:49.593197 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ecbb3-d4c1-48c0-9372-c1af79f7e327 None admin] PATH_INFO: `/v3/users/placement` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:49.600202 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ecbb3-d4c1-48c0-9372-c1af79f7e327 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:49.601670 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ecbb3-d4c1-48c0-9372-c1af79f7e327 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-764ecbb3-d4c1-48c0-9372-c1af79f7e327 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:32:49.616793 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:49.618043 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:49.619343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:49.619343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:49.619343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:32:49.619343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:32:49.619343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:32:49.619343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:32:49.619343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:32:49.619343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:32:49.619343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:32:49.619343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:49.619343 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 639/1280] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:32:49.624028 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88e039-23f1-480d-afe3-805557342ed0 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:49.627838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:49.724022 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88e039-23f1-480d-afe3-805557342ed0 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:49.759000 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5101a9-e297-4ce0-9279-f788c219fe5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:49.759552 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5101a9-e297-4ce0-9279-f788c219fe5f None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:49.759831 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5101a9-e297-4ce0-9279-f788c219fe5f None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:49.760100 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5101a9-e297-4ce0-9279-f788c219fe5f None admin] PATH_INFO: `/v3/users` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:49.760966 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5101a9-e297-4ce0-9279-f788c219fe5f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:49.761274 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5101a9-e297-4ce0-9279-f788c219fe5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:49.762693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5101a9-e297-4ce0-9279-f788c219fe5f None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:49.779681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 642/1281] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:32:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:32:49.785766 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ecbb3-d4c1-48c0-9372-c1af79f7e327 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:49.878377 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:49.887954 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ecbb3-d4c1-48c0-9372-c1af79f7e327 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:49.925929 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025ede7-4847-4af7-8ad9-e14c7e9da0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:49.926583 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ede7-4847-4af7-8ad9-e14c7e9da0d0 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:49.926866 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ede7-4847-4af7-8ad9-e14c7e9da0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:49.927140 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025ede7-4847-4af7-8ad9-e14c7e9da0d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:49.996921 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025ede7-4847-4af7-8ad9-e14c7e9da0d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:49.998381 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025ede7-4847-4af7-8ad9-e14c7e9da0d0 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:50.086150 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:50.097169 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 640/1282] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:32:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:32:50.103133 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5101a9-e297-4ce0-9279-f788c219fe5f None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:50.164601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:50.202400 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5101a9-e297-4ce0-9279-f788c219fe5f None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:50.236288 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6d12e-43a5-4227-9513-8d53d5ab52eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:50.237113 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6d12e-43a5-4227-9513-8d53d5ab52eb None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:50.237395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6d12e-43a5-4227-9513-8d53d5ab52eb None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:50.237718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6d12e-43a5-4227-9513-8d53d5ab52eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:50.247602 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6d12e-43a5-4227-9513-8d53d5ab52eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:50.249069 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6d12e-43a5-4227-9513-8d53d5ab52eb None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-abe6d12e-43a5-4227-9513-8d53d5ab52eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:32:50.266105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:50.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:50.273253 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:32:50.273253 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:32:50.273253 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:32:50.273253 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:50.273253 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 643/1283] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:32:50.273638 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025ede7-4847-4af7-8ad9-e14c7e9da0d0 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:50.373870 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025ede7-4847-4af7-8ad9-e14c7e9da0d0 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:50.407834 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2ef6b-b482-46e3-a87e-93ca8255e9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:50.411028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2ef6b-b482-46e3-a87e-93ca8255e9b8 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:50.411028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2ef6b-b482-46e3-a87e-93ca8255e9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:50.411028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2ef6b-b482-46e3-a87e-93ca8255e9b8 None admin] PATH_INFO: `/v3/domains` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:50.411028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2ef6b-b482-46e3-a87e-93ca8255e9b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:50.411028 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2ef6b-b482-46e3-a87e-93ca8255e9b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:50.411501 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2ef6b-b482-46e3-a87e-93ca8255e9b8 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:50.423393 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 641/1284] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:32:50.429447 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6d12e-43a5-4227-9513-8d53d5ab52eb None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:50.534457 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6d12e-43a5-4227-9513-8d53d5ab52eb None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:50.548409 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:50.568189 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e4072-6413-426a-ad6f-afd0ef567c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:50.569019 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e4072-6413-426a-ad6f-afd0ef567c72 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:50.569316 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e4072-6413-426a-ad6f-afd0ef567c72 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:50.569589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e4072-6413-426a-ad6f-afd0ef567c72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:50.576849 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e4072-6413-426a-ad6f-afd0ef567c72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:50.578274 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e4072-6413-426a-ad6f-afd0ef567c72 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application [None req-ab1e4072-6413-426a-ad6f-afd0ef567c72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:32:50.592888 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application with Lock( May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:50.594064 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:50.595320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:32:50.595320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:32:50.595320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:32:50.595320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:32:50.595320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:32:50.595320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:32:50.595320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:32:50.595320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: ERROR keystone.server.flask.application May 19 20:32:50.595320 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 644/1285] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:32:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:32:50.600015 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2ef6b-b482-46e3-a87e-93ca8255e9b8 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:50.650872 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:50.701704 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2ef6b-b482-46e3-a87e-93ca8255e9b8 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:50.742551 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b504a-3b4a-4513-99b5-f0b47ba2672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:50.743529 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b504a-3b4a-4513-99b5-f0b47ba2672d None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:50.743963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b504a-3b4a-4513-99b5-f0b47ba2672d None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:50.744389 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b504a-3b4a-4513-99b5-f0b47ba2672d None admin] PATH_INFO: `/v3/projects` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:50.745726 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b504a-3b4a-4513-99b5-f0b47ba2672d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:50.746098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b504a-3b4a-4513-99b5-f0b47ba2672d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:50.748158 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4b504a-3b4a-4513-99b5-f0b47ba2672d None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:50.767221 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 642/1286] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:32:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:50.773334 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e4072-6413-426a-ad6f-afd0ef567c72 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:50.876416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e4072-6413-426a-ad6f-afd0ef567c72 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:50.910500 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf34d6-b2a9-452d-a7dc-8fed385a7f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:50.911137 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf34d6-b2a9-452d-a7dc-8fed385a7f78 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:50.911421 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf34d6-b2a9-452d-a7dc-8fed385a7f78 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:50.911746 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf34d6-b2a9-452d-a7dc-8fed385a7f78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:50.912626 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf34d6-b2a9-452d-a7dc-8fed385a7f78 None admin] RBAC: Adding query filter params (role.id=8eaff9cd613a49b29548e90d7e3e9215, scope.project.id=895c72b9c9614f5dbf18b4255821b478, user.id=955b371144784fea84ee7a90886c2d0f) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:50.912942 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf34d6-b2a9-452d-a7dc-8fed385a7f78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:50.914627 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf34d6-b2a9-452d-a7dc-8fed385a7f78 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:50.920539 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 645/1287] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:32:50 2022] GET /identity/v3/role_assignments?user.id=955b371144784fea84ee7a90886c2d0f&scope.project.id=895c72b9c9614f5dbf18b4255821b478&role.id=8eaff9cd613a49b29548e90d7e3e9215 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:53.287805 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36732cc2-64b2-47b1-b740-24ff20e8f3d5 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:53.288674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36732cc2-64b2-47b1-b740-24ff20e8f3d5 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:53.288674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36732cc2-64b2-47b1-b740-24ff20e8f3d5 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:53.290073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 643/1288] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:32:53.296643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c277a-0bac-4622-8c3e-8ba84aa676e0 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:53.297141 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c277a-0bac-4622-8c3e-8ba84aa676e0 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:53.297242 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c277a-0bac-4622-8c3e-8ba84aa676e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:53.315010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-ce6c277a-0bac-4622-8c3e-8ba84aa676e0 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:32:53.316194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-ce6c277a-0bac-4622-8c3e-8ba84aa676e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:32:53.346370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:53.384001 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 646/1289] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:32:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:32:53.406332 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36732cc2-64b2-47b1-b740-24ff20e8f3d5 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:53.510140 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36732cc2-64b2-47b1-b740-24ff20e8f3d5 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:53.521636 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:53.548019 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07051c52-269c-4ec6-ae25-ca392d2ce403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:53.548777 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07051c52-269c-4ec6-ae25-ca392d2ce403 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:53.549058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07051c52-269c-4ec6-ae25-ca392d2ce403 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:53.549329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07051c52-269c-4ec6-ae25-ca392d2ce403 None admin] PATH_INFO: `/v3/services/placement` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:53.549974 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07051c52-269c-4ec6-ae25-ca392d2ce403 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:53.551386 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07051c52-269c-4ec6-ae25-ca392d2ce403 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-07051c52-269c-4ec6-ae25-ca392d2ce403 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:53.563153 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:53.564353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:53.565329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 644/1290] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:32:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:53.570678 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c277a-0bac-4622-8c3e-8ba84aa676e0 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:53.672495 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c277a-0bac-4622-8c3e-8ba84aa676e0 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:53.706472 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11742489-3ea3-40b2-aa33-39c7207ab982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:53.707131 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11742489-3ea3-40b2-aa33-39c7207ab982 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:53.707416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11742489-3ea3-40b2-aa33-39c7207ab982 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:53.707693 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11742489-3ea3-40b2-aa33-39c7207ab982 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:53.708427 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11742489-3ea3-40b2-aa33-39c7207ab982 None admin] RBAC: Adding query filter params (name=placement) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:53.708737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11742489-3ea3-40b2-aa33-39c7207ab982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:53.710105 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11742489-3ea3-40b2-aa33-39c7207ab982 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:53.716416 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 647/1291] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:32:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:53.722309 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07051c52-269c-4ec6-ae25-ca392d2ce403 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:53.824642 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07051c52-269c-4ec6-ae25-ca392d2ce403 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:53.858400 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b812d9b-69de-4025-8551-4ee81cc4b235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:53.859069 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b812d9b-69de-4025-8551-4ee81cc4b235 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:53.859350 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b812d9b-69de-4025-8551-4ee81cc4b235 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:53.859632 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b812d9b-69de-4025-8551-4ee81cc4b235 None admin] PATH_INFO: `/v3/services` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:53.860365 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b812d9b-69de-4025-8551-4ee81cc4b235 None admin] RBAC: Adding query filter params (type=placement) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:53.860673 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b812d9b-69de-4025-8551-4ee81cc4b235 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:53.862045 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b812d9b-69de-4025-8551-4ee81cc4b235 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:53.866522 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 645/1292] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:32:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:56.081906 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8922248-2ab8-4a09-9d8a-e3b9abcbbc16 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:56.082452 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8922248-2ab8-4a09-9d8a-e3b9abcbbc16 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:56.082452 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8922248-2ab8-4a09-9d8a-e3b9abcbbc16 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:56.084120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 648/1293] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:32:56.089421 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ca839-111a-482e-8da6-1a5a02e3d104 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:56.089938 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ca839-111a-482e-8da6-1a5a02e3d104 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:56.090046 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ca839-111a-482e-8da6-1a5a02e3d104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:56.107306 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-8c9ca839-111a-482e-8da6-1a5a02e3d104 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:32:56.109100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-8c9ca839-111a-482e-8da6-1a5a02e3d104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:32:56.140119 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:56.176636 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 646/1294] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:32:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:32:56.199957 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8922248-2ab8-4a09-9d8a-e3b9abcbbc16 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:56.231489 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:56.303533 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8922248-2ab8-4a09-9d8a-e3b9abcbbc16 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:56.337723 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a80ee7-0346-4b72-b283-e6a498287247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:56.338317 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a80ee7-0346-4b72-b283-e6a498287247 None admin] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:56.338602 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a80ee7-0346-4b72-b283-e6a498287247 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:56.338882 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a80ee7-0346-4b72-b283-e6a498287247 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:56.339605 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a80ee7-0346-4b72-b283-e6a498287247 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:56.341030 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a80ee7-0346-4b72-b283-e6a498287247 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:56.416838 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 649/1295] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:32:56 2022] POST /identity/v3/services => generated 246 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:32:58.656569 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7946c-ef87-4d91-b711-c3dcce77af71 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:58.657212 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7946c-ef87-4d91-b711-c3dcce77af71 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:58.657212 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7946c-ef87-4d91-b711-c3dcce77af71 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:58.658512 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 647/1296] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:32:58.664237 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258f033-6a61-434b-84ce-9ed1d889d3b1 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:58.664703 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258f033-6a61-434b-84ce-9ed1d889d3b1 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:58.664818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258f033-6a61-434b-84ce-9ed1d889d3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:58.682572 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-9258f033-6a61-434b-84ce-9ed1d889d3b1 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:32:58.683771 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-9258f033-6a61-434b-84ce-9ed1d889d3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:32:58.757432 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 650/1297] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:32:58 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:32:58.780759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7946c-ef87-4d91-b711-c3dcce77af71 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:58.889424 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7946c-ef87-4d91-b711-c3dcce77af71 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:58.923666 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89a88f-1097-4881-831a-686dfdf11438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:58.924555 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89a88f-1097-4881-831a-686dfdf11438 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:58.924839 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89a88f-1097-4881-831a-686dfdf11438 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:58.925111 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89a88f-1097-4881-831a-686dfdf11438 None admin] PATH_INFO: `/v3/services/placement` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:58.925758 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89a88f-1097-4881-831a-686dfdf11438 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:58.927126 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89a88f-1097-4881-831a-686dfdf11438 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-0b89a88f-1097-4881-831a-686dfdf11438 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:32:58.939427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:32:58.941099 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:32:58.942459 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 648/1298] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:32:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:32:58.946972 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258f033-6a61-434b-84ce-9ed1d889d3b1 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:58.965520 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:59.054749 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258f033-6a61-434b-84ce-9ed1d889d3b1 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:59.089750 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f8adf-301f-4470-8154-79593dae4695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:59.090464 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f8adf-301f-4470-8154-79593dae4695 None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:59.090697 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f8adf-301f-4470-8154-79593dae4695 None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:59.090968 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f8adf-301f-4470-8154-79593dae4695 None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:59.091721 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f8adf-301f-4470-8154-79593dae4695 None admin] RBAC: Adding query filter params (name=placement) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:59.092596 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f8adf-301f-4470-8154-79593dae4695 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:59.094015 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f8adf-301f-4470-8154-79593dae4695 None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:59.101677 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 651/1299] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:32:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:59.108019 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89a88f-1097-4881-831a-686dfdf11438 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:59.141518 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:59.211544 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89a88f-1097-4881-831a-686dfdf11438 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:59.245792 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9841f-55df-4728-bead-846295c67dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:59.246449 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9841f-55df-4728-bead-846295c67dc2 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:59.246730 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9841f-55df-4728-bead-846295c67dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:59.247005 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9841f-55df-4728-bead-846295c67dc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:59.247786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9841f-55df-4728-bead-846295c67dc2 None admin] RBAC: Adding query filter params (interface=public, service_id=eea726e4bc7b4b7eaf2129d10df85e8a, region_id=RegionOne) {{(pid=208582) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:32:59.248096 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9841f-55df-4728-bead-846295c67dc2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:59.249520 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9841f-55df-4728-bead-846295c67dc2 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:59.254704 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 649/1300] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:32:59 2022] GET /identity/v3/endpoints?service_id=eea726e4bc7b4b7eaf2129d10df85e8a&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:32:59.260449 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f8adf-301f-4470-8154-79593dae4695 None admin] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:32:59.347285 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:32:59.365355 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f8adf-301f-4470-8154-79593dae4695 None admin] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:32:59.409619 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d832ec-0b4f-4043-98d2-2db62ef17d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:32:59.410288 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d832ec-0b4f-4043-98d2-2db62ef17d1a None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:32:59.410567 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d832ec-0b4f-4043-98d2-2db62ef17d1a None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:32:59.410840 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d832ec-0b4f-4043-98d2-2db62ef17d1a None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:32:59.411518 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d832ec-0b4f-4043-98d2-2db62ef17d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:32:59.412912 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d832ec-0b4f-4043-98d2-2db62ef17d1a None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:32:59.419917 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 652/1301] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:32:59 2022] GET /identity/v3/services => generated 2191 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:33:01.632430 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d53231-f9b8-4867-8e2b-c4f16b2866ef None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:33:01.633011 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d53231-f9b8-4867-8e2b-c4f16b2866ef None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:33:01.633011 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d53231-f9b8-4867-8e2b-c4f16b2866ef None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:33:01.634450 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 650/1302] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:33:01.640250 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c565e-6ec0-4989-8897-f12151be2c5f None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:33:01.641112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c565e-6ec0-4989-8897-f12151be2c5f None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:33:01.641112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c565e-6ec0-4989-8897-f12151be2c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:33:01.658312 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-881c565e-6ec0-4989-8897-f12151be2c5f None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:33:01.659492 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-881c565e-6ec0-4989-8897-f12151be2c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:33:01.728558 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 653/1303] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:33:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:33:01.751011 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d53231-f9b8-4867-8e2b-c4f16b2866ef None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:33:01.854678 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:33:01.861985 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d53231-f9b8-4867-8e2b-c4f16b2866ef None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:33:01.895556 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cd032-a1f3-4c50-9162-6b62213b97b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:33:01.896368 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cd032-a1f3-4c50-9162-6b62213b97b3 None admin] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:33:01.896707 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cd032-a1f3-4c50-9162-6b62213b97b3 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:33:01.896981 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cd032-a1f3-4c50-9162-6b62213b97b3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:33:01.897632 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cd032-a1f3-4c50-9162-6b62213b97b3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:33:01.899003 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cd032-a1f3-4c50-9162-6b62213b97b3 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application [None req-bb3cd032-a1f3-4c50-9162-6b62213b97b3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:33:01.911098 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:33:01.912764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application with Lock( May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self._enter() May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application return self.creator() May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application created_value = creator( May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: ERROR keystone.server.flask.application May 19 20:33:01.914455 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 651/1304] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:33:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:33:01.918902 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c565e-6ec0-4989-8897-f12151be2c5f None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:33:01.997671 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:33:02.020588 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c565e-6ec0-4989-8897-f12151be2c5f None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:33:02.056796 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c37be-db52-4259-9c93-10e79387836e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:33:02.057472 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c37be-db52-4259-9c93-10e79387836e None admin] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:33:02.057758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c37be-db52-4259-9c93-10e79387836e None admin] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:33:02.058037 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c37be-db52-4259-9c93-10e79387836e None admin] PATH_INFO: `/v3/services` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:33:02.058752 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c37be-db52-4259-9c93-10e79387836e None admin] RBAC: Adding query filter params (name=placement) {{(pid=208581) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:33:02.059062 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c37be-db52-4259-9c93-10e79387836e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:33:02.060541 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c37be-db52-4259-9c93-10e79387836e None admin] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:33:02.065737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 654/1305] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:33:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:33:02.071589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cd032-a1f3-4c50-9162-6b62213b97b3 None admin] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:33:02.173050 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cd032-a1f3-4c50-9162-6b62213b97b3 None admin] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:33:02.207122 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c82741-b350-4813-8b73-f8c0b13f7c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc336177e5d04523af605ab7e66532eb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:33:02.207773 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c82741-b350-4813-8b73-f8c0b13f7c42 None admin] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:33:02.208059 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c82741-b350-4813-8b73-f8c0b13f7c42 None admin] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:33:02.208404 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c82741-b350-4813-8b73-f8c0b13f7c42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:33:02.209144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c82741-b350-4813-8b73-f8c0b13f7c42 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:33:02.210506 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c82741-b350-4813-8b73-f8c0b13f7c42 None admin] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:33:02.296795 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 652/1306] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:33:02 2022] POST /identity/v3/endpoints => generated 331 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:33:12.113073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=30 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:33:12.291363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:00.695145 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827151a-22a2-4f88-b2aa-15a1c3b77dd0 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:00.696194 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827151a-22a2-4f88-b2aa-15a1c3b77dd0 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:00.696723 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827151a-22a2-4f88-b2aa-15a1c3b77dd0 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:00.704466 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc692ee-f397-4e24-b447-e10500628188 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:00.705512 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc692ee-f397-4e24-b447-e10500628188 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:00.706032 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc692ee-f397-4e24-b447-e10500628188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:00.708561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 655/1307] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:00 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:00.792849 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-9dc692ee-f397-4e24-b447-e10500628188 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:00.794472 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-9dc692ee-f397-4e24-b447-e10500628188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:00.928853 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 653/1308] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:40:00.963934 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c7cc-b085-4609-9b42-035ccbe16924 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:00.964641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c7cc-b085-4609-9b42-035ccbe16924 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:00.965134 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c7cc-b085-4609-9b42-035ccbe16924 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:00.966808 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 656/1309] 10.222.0.88 () {54 vars in 964 bytes} [Thu May 19 20:40:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:00.976312 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90483e51-ba41-4bac-8689-28b17d89c46f None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:00.976995 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90483e51-ba41-4bac-8689-28b17d89c46f None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:00.977817 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90483e51-ba41-4bac-8689-28b17d89c46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:01.148794 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-90483e51-ba41-4bac-8689-28b17d89c46f None None] MFA Rules not processed for user `b272b57eafcb4de9b85ce8a8a893a2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:01.150589 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-90483e51-ba41-4bac-8689-28b17d89c46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:01.269632 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:01.436144 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 654/1310] 10.222.0.88 () {58 vars in 1061 bytes} [Thu May 19 20:40:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:40:01.446052 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678c7cc-b085-4609-9b42-035ccbe16924 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:01.448550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-2678c7cc-b085-4609-9b42-035ccbe16924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:01.687244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678c7cc-b085-4609-9b42-035ccbe16924 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:01.852410 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2307a4-710f-45de-a249-fd9abb83afef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:01.853112 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2307a4-710f-45de-a249-fd9abb83afef service neutron] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:01.853351 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2307a4-710f-45de-a249-fd9abb83afef service neutron] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:01.853795 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2307a4-710f-45de-a249-fd9abb83afef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:01.855641 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-fe2307a4-710f-45de-a249-fd9abb83afef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:01.969467 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:02.089813 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2307a4-710f-45de-a249-fd9abb83afef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:02.091283 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2307a4-710f-45de-a249-fd9abb83afef service neutron] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:02.111366 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 657/1311] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:40:05.041539 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ea642-5aac-4597-9cfa-6314bcbc6e4c None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:05.042015 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ea642-5aac-4597-9cfa-6314bcbc6e4c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:05.042127 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ea642-5aac-4597-9cfa-6314bcbc6e4c None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:05.044499 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 655/1312] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:05.049500 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c502a03-ee65-4f01-98fc-19794e2b66d5 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:05.050074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c502a03-ee65-4f01-98fc-19794e2b66d5 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:05.050074 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c502a03-ee65-4f01-98fc-19794e2b66d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:05.224947 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-7c502a03-ee65-4f01-98fc-19794e2b66d5 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:05.226521 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-7c502a03-ee65-4f01-98fc-19794e2b66d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:05.432388 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 658/1313] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:05.469350 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ea642-5aac-4597-9cfa-6314bcbc6e4c None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:05.471149 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-d11ea642-5aac-4597-9cfa-6314bcbc6e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:05.599680 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ea642-5aac-4597-9cfa-6314bcbc6e4c None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:05.678614 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:05.681369 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93834c-8232-414e-ad5e-b3e0cd88322e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:05.682044 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93834c-8232-414e-ad5e-b3e0cd88322e service neutron] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:05.682334 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93834c-8232-414e-ad5e-b3e0cd88322e service neutron] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:05.682607 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93834c-8232-414e-ad5e-b3e0cd88322e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:05.684371 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-da93834c-8232-414e-ad5e-b3e0cd88322e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:05.809486 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93834c-8232-414e-ad5e-b3e0cd88322e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:05.810960 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93834c-8232-414e-ad5e-b3e0cd88322e service neutron] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:05.830446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 656/1314] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:40:08.587583 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4c1e0-bbfe-4388-9f12-3379ab5c5053 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:08.588120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4c1e0-bbfe-4388-9f12-3379ab5c5053 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:08.588120 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4c1e0-bbfe-4388-9f12-3379ab5c5053 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:08.589554 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 659/1315] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd766a7-7c8a-4949-b1c2-dc70639289f8 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:08.595532 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd766a7-7c8a-4949-b1c2-dc70639289f8 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:08.595631 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd766a7-7c8a-4949-b1c2-dc70639289f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:08.660970 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-bbd766a7-7c8a-4949-b1c2-dc70639289f8 None None] MFA Rules not processed for user `ff96da569f6448c9b0a3b1aebf6f10e0`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:08.662454 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-bbd766a7-7c8a-4949-b1c2-dc70639289f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:08.714690 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:08.760759 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 657/1316] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:08 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:40:08.816810 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8f03d-3a2e-4d8b-b3fb-37b4f85da8a8 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:08.817097 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8f03d-3a2e-4d8b-b3fb-37b4f85da8a8 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:08.817372 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8f03d-3a2e-4d8b-b3fb-37b4f85da8a8 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:08.818758 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 660/1317] 10.222.0.88 () {54 vars in 964 bytes} [Thu May 19 20:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:08.824963 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6717a0-4363-49db-a8c0-fcdf83a1fc90 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:08.825259 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6717a0-4363-49db-a8c0-fcdf83a1fc90 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:08.825542 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6717a0-4363-49db-a8c0-fcdf83a1fc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:08.900959 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-5a6717a0-4363-49db-a8c0-fcdf83a1fc90 None None] MFA Rules not processed for user `b272b57eafcb4de9b85ce8a8a893a2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:08.902167 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-5a6717a0-4363-49db-a8c0-fcdf83a1fc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:08.979073 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 658/1318] 10.222.0.88 () {58 vars in 1061 bytes} [Thu May 19 20:40:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:08.988845 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8f03d-3a2e-4d8b-b3fb-37b4f85da8a8 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:08.990395 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-80a8f03d-3a2e-4d8b-b3fb-37b4f85da8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:09.000240 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:09.096090 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8f03d-3a2e-4d8b-b3fb-37b4f85da8a8 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:09.168649 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409228c-5816-42e8-9b4b-a4ae381d2a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:09.169314 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409228c-5816-42e8-9b4b-a4ae381d2a2e service neutron] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:09.169601 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409228c-5816-42e8-9b4b-a4ae381d2a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:09.169877 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409228c-5816-42e8-9b4b-a4ae381d2a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:09.171550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-b409228c-5816-42e8-9b4b-a4ae381d2a2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:09.234073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:09.280487 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409228c-5816-42e8-9b4b-a4ae381d2a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:09.281916 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409228c-5816-42e8-9b4b-a4ae381d2a2e service neutron] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:09.301288 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 661/1319] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:08 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:40:12.652637 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c092bc-4184-47de-8192-ccd3b680afca None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:12.653255 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c092bc-4184-47de-8192-ccd3b680afca None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:12.653255 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c092bc-4184-47de-8192-ccd3b680afca None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:12.654569 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 659/1320] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:12.659726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb77f1-3779-4913-ae7a-820c070e1682 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:12.660188 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb77f1-3779-4913-ae7a-820c070e1682 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:12.660295 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb77f1-3779-4913-ae7a-820c070e1682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:12.726953 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-2adb77f1-3779-4913-ae7a-820c070e1682 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:12.728093 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-2adb77f1-3779-4913-ae7a-820c070e1682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:12.825877 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 662/1321] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:12.864275 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c092bc-4184-47de-8192-ccd3b680afca None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:12.865879 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-c8c092bc-4184-47de-8192-ccd3b680afca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:12.916058 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:12.971977 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c092bc-4184-47de-8192-ccd3b680afca None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:13.050239 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1185b20-80de-4385-9ca7-97c87bbfd65c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:13.051268 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185b20-80de-4385-9ca7-97c87bbfd65c service neutron] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:13.051709 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185b20-80de-4385-9ca7-97c87bbfd65c service neutron] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:13.052164 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1185b20-80de-4385-9ca7-97c87bbfd65c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:13.053936 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-d1185b20-80de-4385-9ca7-97c87bbfd65c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:13.154439 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:13.169078 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1185b20-80de-4385-9ca7-97c87bbfd65c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:13.170561 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1185b20-80de-4385-9ca7-97c87bbfd65c service neutron] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:13.191523 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 660/1322] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:40:16.633589 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0801a3-19e8-4cb1-ac58-940542ab6af2 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:16.634175 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0801a3-19e8-4cb1-ac58-940542ab6af2 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:16.634175 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0801a3-19e8-4cb1-ac58-940542ab6af2 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:16.635719 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 663/1323] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:16.640857 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85021db2-d4ef-4c88-95ec-e02efcb4220c None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:16.641281 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85021db2-d4ef-4c88-95ec-e02efcb4220c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:16.641431 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85021db2-d4ef-4c88-95ec-e02efcb4220c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:16.709020 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-85021db2-d4ef-4c88-95ec-e02efcb4220c None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:16.710253 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-85021db2-d4ef-4c88-95ec-e02efcb4220c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:16.810634 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 661/1324] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:16.846522 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0801a3-19e8-4cb1-ac58-940542ab6af2 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:16.848052 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-af0801a3-19e8-4cb1-ac58-940542ab6af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:16.867760 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:16.954336 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0801a3-19e8-4cb1-ac58-940542ab6af2 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:17.028277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93e556-5d7a-438d-b47b-c2d557902679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:17.028971 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93e556-5d7a-438d-b47b-c2d557902679 service neutron] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:17.029261 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93e556-5d7a-438d-b47b-c2d557902679 service neutron] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:17.029546 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93e556-5d7a-438d-b47b-c2d557902679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:17.031219 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-9b93e556-5d7a-438d-b47b-c2d557902679 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:17.120007 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:17.161018 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93e556-5d7a-438d-b47b-c2d557902679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:17.162718 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93e556-5d7a-438d-b47b-c2d557902679 service neutron] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:17.181885 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 664/1325] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 20:40:19.500363 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcedb4-51dd-408e-b386-2b8bd7510d8b None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:19.500774 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcedb4-51dd-408e-b386-2b8bd7510d8b None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:19.500933 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcedb4-51dd-408e-b386-2b8bd7510d8b None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:19.502446 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 662/1326] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:19.508180 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94ed6e-79d5-4227-b1b8-cdfc38c9c7f7 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:19.516483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94ed6e-79d5-4227-b1b8-cdfc38c9c7f7 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:19.516483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94ed6e-79d5-4227-b1b8-cdfc38c9c7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:19.578465 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-fb94ed6e-79d5-4227-b1b8-cdfc38c9c7f7 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:19.579639 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-fb94ed6e-79d5-4227-b1b8-cdfc38c9c7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:19.686370 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 665/1327] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:19.723802 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcedb4-51dd-408e-b386-2b8bd7510d8b None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:19.725413 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-dddcedb4-51dd-408e-b386-2b8bd7510d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:19.787117 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:19.829752 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcedb4-51dd-408e-b386-2b8bd7510d8b None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:19.909953 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48bf80-6557-4508-9d9a-4096dc0f943e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:19.910616 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48bf80-6557-4508-9d9a-4096dc0f943e service neutron] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:19.910898 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48bf80-6557-4508-9d9a-4096dc0f943e service neutron] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:19.911170 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48bf80-6557-4508-9d9a-4096dc0f943e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:19.913109 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-1d48bf80-6557-4508-9d9a-4096dc0f943e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:20.036376 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:20.039384 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48bf80-6557-4508-9d9a-4096dc0f943e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:20.041450 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48bf80-6557-4508-9d9a-4096dc0f943e service neutron] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:20.063674 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 663/1328] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:40:22.380290 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d86ab4-a8fe-45db-b463-2f2423f39d20 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:22.380896 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d86ab4-a8fe-45db-b463-2f2423f39d20 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:22.380896 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d86ab4-a8fe-45db-b463-2f2423f39d20 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:22.382281 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 666/1329] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:22.387729 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eeddd-9fc5-47f8-adc7-98271bb5746c None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:22.388157 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eeddd-9fc5-47f8-adc7-98271bb5746c None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:22.388275 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40eeddd-9fc5-47f8-adc7-98271bb5746c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:22.456832 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-b40eeddd-9fc5-47f8-adc7-98271bb5746c None None] MFA Rules not processed for user `ff96da569f6448c9b0a3b1aebf6f10e0`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:22.458083 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-b40eeddd-9fc5-47f8-adc7-98271bb5746c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:22.556395 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 664/1330] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:22.593483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d86ab4-a8fe-45db-b463-2f2423f39d20 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:22.595043 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-52d86ab4-a8fe-45db-b463-2f2423f39d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:22.626550 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:22.700622 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d86ab4-a8fe-45db-b463-2f2423f39d20 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:22.772783 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe58a3-b798-4b8e-ad19-6b7a5ac2a633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:22.773442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe58a3-b798-4b8e-ad19-6b7a5ac2a633 service neutron] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:22.773726 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe58a3-b798-4b8e-ad19-6b7a5ac2a633 service neutron] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:22.774005 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe58a3-b798-4b8e-ad19-6b7a5ac2a633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:22.775669 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-f2fe58a3-b798-4b8e-ad19-6b7a5ac2a633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:22.851988 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:22.884788 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe58a3-b798-4b8e-ad19-6b7a5ac2a633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:22.886244 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe58a3-b798-4b8e-ad19-6b7a5ac2a633 service neutron] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:22.905274 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 667/1331] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:22 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:40:25.554268 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16abf4-229a-435f-ae89-64df1c96c04a None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:25.554838 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16abf4-229a-435f-ae89-64df1c96c04a None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:25.554838 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce16abf4-229a-435f-ae89-64df1c96c04a None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:25.556210 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 665/1332] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:25.562090 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4d731-d416-449a-929f-2371c7859486 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:25.562586 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4d731-d416-449a-929f-2371c7859486 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:25.562695 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4d731-d416-449a-929f-2371c7859486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:25.632198 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-dcb4d731-d416-449a-929f-2371c7859486 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:25.633483 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-dcb4d731-d416-449a-929f-2371c7859486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:25.734025 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 668/1333] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:25.771410 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16abf4-229a-435f-ae89-64df1c96c04a None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:25.773672 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-ce16abf4-229a-435f-ae89-64df1c96c04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:25.848626 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:25.887216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce16abf4-229a-435f-ae89-64df1c96c04a None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:25.962786 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa06be8-08f0-47dc-a06a-c7f22f977f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:25.963448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa06be8-08f0-47dc-a06a-c7f22f977f96 service neutron] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:25.963732 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa06be8-08f0-47dc-a06a-c7f22f977f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:25.964017 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa06be8-08f0-47dc-a06a-c7f22f977f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:25.965816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-1fa06be8-08f0-47dc-a06a-c7f22f977f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:26.079989 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa06be8-08f0-47dc-a06a-c7f22f977f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:26.081482 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa06be8-08f0-47dc-a06a-c7f22f977f96 service neutron] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:26.091901 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:26.100997 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 666/1334] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:40:28.417757 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed2089-89df-4d0f-ab38-20d3aafdeca3 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:28.418329 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed2089-89df-4d0f-ab38-20d3aafdeca3 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:28.418329 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed2089-89df-4d0f-ab38-20d3aafdeca3 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:28.419683 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 669/1335] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:28.424962 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ede55-fa7e-46f8-90b0-e480468f0c49 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:28.425401 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ede55-fa7e-46f8-90b0-e480468f0c49 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:28.425532 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ede55-fa7e-46f8-90b0-e480468f0c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:28.490942 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-dd9ede55-fa7e-46f8-90b0-e480468f0c49 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:28.492194 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-dd9ede55-fa7e-46f8-90b0-e480468f0c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:28.591370 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 667/1336] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:28.633673 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed2089-89df-4d0f-ab38-20d3aafdeca3 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:28.635299 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-50ed2089-89df-4d0f-ab38-20d3aafdeca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:28.683155 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:28.754525 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed2089-89df-4d0f-ab38-20d3aafdeca3 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:28.832079 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00022a79-f24a-48ea-a10b-8d0235d960dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:28.832772 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00022a79-f24a-48ea-a10b-8d0235d960dd service neutron] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:28.833055 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00022a79-f24a-48ea-a10b-8d0235d960dd service neutron] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:28.833332 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00022a79-f24a-48ea-a10b-8d0235d960dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:28.834999 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-00022a79-f24a-48ea-a10b-8d0235d960dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:28.917697 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:28.943488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00022a79-f24a-48ea-a10b-8d0235d960dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:28.945122 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00022a79-f24a-48ea-a10b-8d0235d960dd service neutron] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:28.964467 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 670/1337] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:40:31.975424 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aff499-fc0b-4e36-a798-aedb79e67c33 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:31.975983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aff499-fc0b-4e36-a798-aedb79e67c33 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:31.975983 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aff499-fc0b-4e36-a798-aedb79e67c33 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:31.977411 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 668/1338] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:31.982725 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98956a51-95dd-412c-96ac-8d31b8660bdd None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:31.983190 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98956a51-95dd-412c-96ac-8d31b8660bdd None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:31.983305 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98956a51-95dd-412c-96ac-8d31b8660bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:32.051639 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-98956a51-95dd-412c-96ac-8d31b8660bdd None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:32.052865 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-98956a51-95dd-412c-96ac-8d31b8660bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:32.156229 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 671/1339] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:32.193113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aff499-fc0b-4e36-a798-aedb79e67c33 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:32.195273 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-d6aff499-fc0b-4e36-a798-aedb79e67c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:32.275635 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=100 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:32.291051 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aff499-fc0b-4e36-a798-aedb79e67c33 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:32.298597 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68667374-cae2-41b5-be47-1b70ab20ceed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:32.298597 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68667374-cae2-41b5-be47-1b70ab20ceed service neutron] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:32.298597 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68667374-cae2-41b5-be47-1b70ab20ceed service neutron] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:32.298597 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68667374-cae2-41b5-be47-1b70ab20ceed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:32.305830 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68667374-cae2-41b5-be47-1b70ab20ceed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:32.307268 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68667374-cae2-41b5-be47-1b70ab20ceed service neutron] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:32.316764 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 669/1340] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:40:36.873846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6437cb6-9a1e-4d05-b581-95040faab9d2 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:36.874454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6437cb6-9a1e-4d05-b581-95040faab9d2 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:36.874454 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6437cb6-9a1e-4d05-b581-95040faab9d2 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:36.875767 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 672/1341] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:36.882479 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3f30-7b27-465d-9325-b45b4d9562d5 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:36.883475 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3f30-7b27-465d-9325-b45b4d9562d5 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:36.883926 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60f3f30-7b27-465d-9325-b45b4d9562d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:36.919678 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-c60f3f30-7b27-465d-9325-b45b4d9562d5 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:36.920919 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-c60f3f30-7b27-465d-9325-b45b4d9562d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:37.003216 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 670/1342] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:37.041493 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6437cb6-9a1e-4d05-b581-95040faab9d2 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:37.053041 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6437cb6-9a1e-4d05-b581-95040faab9d2 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:37.056532 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0ae9da-c754-4fac-8d4f-ee4abcc4908b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:37.057183 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ae9da-c754-4fac-8d4f-ee4abcc4908b service neutron] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:37.057462 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ae9da-c754-4fac-8d4f-ee4abcc4908b service neutron] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:37.057731 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0ae9da-c754-4fac-8d4f-ee4abcc4908b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:37.067628 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ae9da-c754-4fac-8d4f-ee4abcc4908b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:37.069080 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0ae9da-c754-4fac-8d4f-ee4abcc4908b service neutron] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:37.078507 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 673/1343] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 20:40:40.347788 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee709b-d31a-4e3e-aa10-9bb06d95f6f7 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:40.348423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee709b-d31a-4e3e-aa10-9bb06d95f6f7 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:40.348423 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee709b-d31a-4e3e-aa10-9bb06d95f6f7 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:40.349756 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 671/1344] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:40.355072 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870889-2143-444c-9a39-ca93185953bf None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:40.355514 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870889-2143-444c-9a39-ca93185953bf None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:40.355653 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da870889-2143-444c-9a39-ca93185953bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:40.373690 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-da870889-2143-444c-9a39-ca93185953bf None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:40.374884 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-da870889-2143-444c-9a39-ca93185953bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:40.383139 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 674/1345] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:40 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:40.420550 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee709b-d31a-4e3e-aa10-9bb06d95f6f7 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:40.432168 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee709b-d31a-4e3e-aa10-9bb06d95f6f7 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:40.435723 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d46f19-57bf-4368-ac9f-5799b7cc975f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:40.436381 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d46f19-57bf-4368-ac9f-5799b7cc975f service neutron] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:40.436668 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d46f19-57bf-4368-ac9f-5799b7cc975f service neutron] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:40.436945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d46f19-57bf-4368-ac9f-5799b7cc975f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:40.451354 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d46f19-57bf-4368-ac9f-5799b7cc975f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:40.452960 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d46f19-57bf-4368-ac9f-5799b7cc975f service neutron] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:40.463448 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 672/1346] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:40:45.079561 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79cda-0923-4aa3-8cfe-a9dda3ebf255 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:45.080128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79cda-0923-4aa3-8cfe-a9dda3ebf255 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:45.080128 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c79cda-0923-4aa3-8cfe-a9dda3ebf255 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:45.082073 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 675/1347] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:40:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:45.087445 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10543a-ee78-4496-a74a-daf4496e329e None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:45.087887 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10543a-ee78-4496-a74a-daf4496e329e None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:45.088012 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b10543a-ee78-4496-a74a-daf4496e329e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:45.106834 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-2b10543a-ee78-4496-a74a-daf4496e329e None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:45.108111 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-2b10543a-ee78-4496-a74a-daf4496e329e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:45.117090 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 673/1348] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:45.156063 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c79cda-0923-4aa3-8cfe-a9dda3ebf255 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:40:45.169010 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c79cda-0923-4aa3-8cfe-a9dda3ebf255 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:40:45.172829 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d21f77-475a-4965-9876-a508c4644214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b272b57eafcb4de9b85ce8a8a893a2d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:40:45.173488 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d21f77-475a-4965-9876-a508c4644214 service neutron] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:45.174161 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d21f77-475a-4965-9876-a508c4644214 service neutron] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:45.174563 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d21f77-475a-4965-9876-a508c4644214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:45.184937 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d21f77-475a-4965-9876-a508c4644214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:40:45.186468 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d21f77-475a-4965-9876-a508c4644214 service neutron] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:40:45.196379 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 676/1349] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:40:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:40:50.691558 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9ea00-1db5-4f43-96ef-b411b974e899 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:50.692076 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9ea00-1db5-4f43-96ef-b411b974e899 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:50.692175 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9ea00-1db5-4f43-96ef-b411b974e899 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:50.693784 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 674/1350] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 20:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:50.699006 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790a4c2-fd0b-47e7-963a-6d009fcf3c4c None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:50.699433 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790a4c2-fd0b-47e7-963a-6d009fcf3c4c None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:50.699545 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790a4c2-fd0b-47e7-963a-6d009fcf3c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:50.716282 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:50.778145 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-2790a4c2-fd0b-47e7-963a-6d009fcf3c4c None None] MFA Rules not processed for user `955b371144784fea84ee7a90886c2d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:50.779277 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-2790a4c2-fd0b-47e7-963a-6d009fcf3c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:50.876442 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 677/1351] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 20:40:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:40:55.101305 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=57 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:40:55.425584 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb309136-ff1f-4c32-9aa2-b3fca24f0127 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:55.425771 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb309136-ff1f-4c32-9aa2-b3fca24f0127 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:55.426035 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb309136-ff1f-4c32-9aa2-b3fca24f0127 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:55.427629 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 675/1352] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 20:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:55.433818 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb24f7-004b-460d-ac87-e259c7c30b43 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:55.434909 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb24f7-004b-460d-ac87-e259c7c30b43 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:55.435526 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb24f7-004b-460d-ac87-e259c7c30b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:55.470517 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-8dbb24f7-004b-460d-ac87-e259c7c30b43 None None] MFA Rules not processed for user `955b371144784fea84ee7a90886c2d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:55.472807 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-8dbb24f7-004b-460d-ac87-e259c7c30b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:55.483885 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 678/1353] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 20:40:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:40:56.928329 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c407e1-d06b-41f5-9936-65e13f9ecab4 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:56.928912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c407e1-d06b-41f5-9936-65e13f9ecab4 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:56.928912 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c407e1-d06b-41f5-9936-65e13f9ecab4 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:56.930283 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 676/1354] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 20:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:40:56.935603 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96170314-0cbb-4280-8a42-cd40d634326b None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:40:56.936038 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96170314-0cbb-4280-8a42-cd40d634326b None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:40:56.936364 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96170314-0cbb-4280-8a42-cd40d634326b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:40:56.955131 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-96170314-0cbb-4280-8a42-cd40d634326b None None] MFA Rules not processed for user `955b371144784fea84ee7a90886c2d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:40:56.957096 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-96170314-0cbb-4280-8a42-cd40d634326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:40:56.965681 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 679/1355] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 20:40:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:41:01.205695 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf384b-afb6-46e6-afe0-a6138a72ce27 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:01.206261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf384b-afb6-46e6-afe0-a6138a72ce27 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:01.206261 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf384b-afb6-46e6-afe0-a6138a72ce27 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:01.212178 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 677/1356] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:41:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:41:01.213826 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61916b1d-3c1e-4ade-8002-ef2db01b2e8e None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:01.214278 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61916b1d-3c1e-4ade-8002-ef2db01b2e8e None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:01.214787 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61916b1d-3c1e-4ade-8002-ef2db01b2e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:01.238181 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-61916b1d-3c1e-4ade-8002-ef2db01b2e8e None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:41:01.239453 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-61916b1d-3c1e-4ade-8002-ef2db01b2e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:41:01.247445 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 680/1357] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 20:41:01.306215 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc1909-b079-43bc-9cd8-cf303e366f8d None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:01.306716 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc1909-b079-43bc-9cd8-cf303e366f8d None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:01.307163 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc1909-b079-43bc-9cd8-cf303e366f8d None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:01.308956 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 678/1358] 10.222.0.88 () {54 vars in 961 bytes} [Thu May 19 20:41:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:41:01.327183 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b00c84-4d58-48f0-af22-6a4c98befbe0 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:01.327686 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b00c84-4d58-48f0-af22-6a4c98befbe0 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:01.328167 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b00c84-4d58-48f0-af22-6a4c98befbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:01.383279 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-30b00c84-4d58-48f0-af22-6a4c98befbe0 None None] MFA Rules not processed for user `e6a982ae15a6443b9b52529df47aa837`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:41:01.384633 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-30b00c84-4d58-48f0-af22-6a4c98befbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:41:01.476154 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=100 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:41:01.502904 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 681/1359] 10.222.0.88 () {58 vars in 1058 bytes} [Thu May 19 20:41:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:41:01.516427 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc1909-b079-43bc-9cd8-cf303e366f8d None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:41:01.534747 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc1909-b079-43bc-9cd8-cf303e366f8d None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:41:01.541631 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74467bd6-5929-408f-b650-afd4a78e6311 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a982ae15a6443b9b52529df47aa837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:41:01.542428 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467bd6-5929-408f-b650-afd4a78e6311 service nova] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:01.542735 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467bd6-5929-408f-b650-afd4a78e6311 service nova] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:01.543014 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74467bd6-5929-408f-b650-afd4a78e6311 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:01.553016 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74467bd6-5929-408f-b650-afd4a78e6311 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:41:01.554645 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74467bd6-5929-408f-b650-afd4a78e6311 service nova] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:41:01.565514 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 679/1360] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:41:04.014928 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fbb8b-2120-4730-ac36-a522c9bdc5f2 None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:04.015443 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fbb8b-2120-4730-ac36-a522c9bdc5f2 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:04.015569 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fbb8b-2120-4730-ac36-a522c9bdc5f2 None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:04.017054 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 682/1361] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:41:04.023133 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973679f4-5c91-4f55-a625-0e05cdcbe67a None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:04.023581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973679f4-5c91-4f55-a625-0e05cdcbe67a None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:04.023701 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973679f4-5c91-4f55-a625-0e05cdcbe67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:04.042734 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-973679f4-5c91-4f55-a625-0e05cdcbe67a None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:41:04.043945 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-973679f4-5c91-4f55-a625-0e05cdcbe67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:41:04.052248 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 680/1362] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:41:04.097477 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fbb8b-2120-4730-ac36-a522c9bdc5f2 None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:41:04.112541 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fbb8b-2120-4730-ac36-a522c9bdc5f2 None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:41:04.118776 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539359b-df88-43cb-83da-958baf31cf99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a982ae15a6443b9b52529df47aa837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:41:04.119643 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539359b-df88-43cb-83da-958baf31cf99 service nova] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:04.119932 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539359b-df88-43cb-83da-958baf31cf99 service nova] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:04.120245 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539359b-df88-43cb-83da-958baf31cf99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:04.129290 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539359b-df88-43cb-83da-958baf31cf99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:41:04.130894 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539359b-df88-43cb-83da-958baf31cf99 service nova] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:41:04.141415 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 683/1363] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:41:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:41:06.862267 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e093d9-e45e-4cfa-a94e-d563819a0196 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:06.862978 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e093d9-e45e-4cfa-a94e-d563819a0196 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:06.862978 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e093d9-e45e-4cfa-a94e-d563819a0196 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:06.864134 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 681/1364] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:41:06.869969 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921ea9f-49de-4ac3-8f15-25e9d56c6f88 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:06.870843 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921ea9f-49de-4ac3-8f15-25e9d56c6f88 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:06.871136 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921ea9f-49de-4ac3-8f15-25e9d56c6f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:06.892837 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-6921ea9f-49de-4ac3-8f15-25e9d56c6f88 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:41:06.894564 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-6921ea9f-49de-4ac3-8f15-25e9d56c6f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:41:06.908391 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 684/1365] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:41:06.957816 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f6a7d-b102-4623-9d0e-58ee7a362ca5 None None] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:06.958100 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f6a7d-b102-4623-9d0e-58ee7a362ca5 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:06.958562 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f6a7d-b102-4623-9d0e-58ee7a362ca5 None None] PATH_INFO: `/` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:06.960284 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 682/1366] 10.222.0.88 () {54 vars in 961 bytes} [Thu May 19 20:41:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:41:06.966482 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d62beb-5568-4fcd-a2c9-144cdb24aa20 None None] REQUEST_METHOD: `POST` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:06.966772 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d62beb-5568-4fcd-a2c9-144cdb24aa20 None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:06.967225 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d62beb-5568-4fcd-a2c9-144cdb24aa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:06.987635 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.auth.core [None req-21d62beb-5568-4fcd-a2c9-144cdb24aa20 None None] MFA Rules not processed for user `e6a982ae15a6443b9b52529df47aa837`. Rule list: `[]` (Enabled: `True`). {{(pid=208581) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:41:06.988867 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.fernet_utils [None req-21d62beb-5568-4fcd-a2c9-144cdb24aa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208581) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:41:06.998258 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 685/1367] 10.222.0.88 () {58 vars in 1058 bytes} [Thu May 19 20:41:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:41:07.005930 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f6a7d-b102-4623-9d0e-58ee7a362ca5 None None] Authenticating user token {{(pid=208582) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:41:07.021961 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413f6a7d-b102-4623-9d0e-58ee7a362ca5 None None] Validating token access rules against request {{(pid=208582) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:41:07.027866 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963c104-e543-4271-86db-91abae9cd63d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a982ae15a6443b9b52529df47aa837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208582) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:41:07.028545 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963c104-e543-4271-86db-91abae9cd63d service nova] REQUEST_METHOD: `GET` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:07.028829 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963c104-e543-4271-86db-91abae9cd63d service nova] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:07.029113 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963c104-e543-4271-86db-91abae9cd63d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:07.038528 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963c104-e543-4271-86db-91abae9cd63d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:41:07.040510 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963c104-e543-4271-86db-91abae9cd63d service nova] RBAC: Authorization granted {{(pid=208582) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:41:07.051143 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 683/1368] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:41:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:41:07.729192 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcb119-c17a-4582-8530-b41628c176ef None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:07.729557 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcb119-c17a-4582-8530-b41628c176ef None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:07.729760 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcb119-c17a-4582-8530-b41628c176ef None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:07.731399 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 686/1369] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:41:07.737728 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ef9c3-a35f-4f6f-bf2d-6f57b28d1fe2 None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:07.738125 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ef9c3-a35f-4f6f-bf2d-6f57b28d1fe2 None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:07.738302 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ef9c3-a35f-4f6f-bf2d-6f57b28d1fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:07.758231 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-7c3ef9c3-a35f-4f6f-bf2d-6f57b28d1fe2 None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:41:07.759462 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-7c3ef9c3-a35f-4f6f-bf2d-6f57b28d1fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:41:07.767789 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 684/1370] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:41:09.511383 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f46b1-11e3-48ec-bb0f-408844fc108b None None] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:09.511926 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f46b1-11e3-48ec-bb0f-408844fc108b None None] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:09.512052 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f46b1-11e3-48ec-bb0f-408844fc108b None None] PATH_INFO: `/` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:09.513422 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 687/1371] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:41:09.519581 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5f693-61f1-4c5a-92df-148925b1c41f None None] REQUEST_METHOD: `POST` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:09.520020 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5f693-61f1-4c5a-92df-148925b1c41f None None] SCRIPT_NAME: `/identity` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:09.520149 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5f693-61f1-4c5a-92df-148925b1c41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=208582) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:09.537725 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.auth.core [None req-3ce5f693-61f1-4c5a-92df-148925b1c41f None None] MFA Rules not processed for user `cc336177e5d04523af605ab7e66532eb`. Rule list: `[]` (Enabled: `True`). {{(pid=208582) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:41:09.538940 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG keystone.common.fernet_utils [None req-3ce5f693-61f1-4c5a-92df-148925b1c41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=208582) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:41:09.547605 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: [pid: 208582|app: 0|req: 685/1372] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 20:41:09.594449 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f46b1-11e3-48ec-bb0f-408844fc108b None None] Authenticating user token {{(pid=208581) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:41:09.609504 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f46b1-11e3-48ec-bb0f-408844fc108b None None] Validating token access rules against request {{(pid=208581) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:41:09.615337 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163e590-8cef-40ba-8f72-08352e01c740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a982ae15a6443b9b52529df47aa837', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '895c72b9c9614f5dbf18b4255821b478', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=208581) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:41:09.615951 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163e590-8cef-40ba-8f72-08352e01c740 service nova] REQUEST_METHOD: `GET` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:41:09.616242 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163e590-8cef-40ba-8f72-08352e01c740 service nova] SCRIPT_NAME: `/identity` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:41:09.616553 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163e590-8cef-40ba-8f72-08352e01c740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=208581) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:41:09.625329 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163e590-8cef-40ba-8f72-08352e01c740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:41:09.626923 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163e590-8cef-40ba-8f72-08352e01c740 service nova] RBAC: Authorization granted {{(pid=208581) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:41:09.637846 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: [pid: 208581|app: 0|req: 688/1373] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 20:41:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 20:41:19.532633 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: DEBUG dbcounter [-] [208582] Writing DB stats keystone:SELECT=21 {{(pid=208582) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:41:19.632889 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: DEBUG dbcounter [-] [208581] Writing DB stats keystone:SELECT=23 {{(pid=208581) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:41:31.214850 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: Thu May 19 20:41:31 2022 - graceful shutdown triggered... May 19 20:41:31.215353 dv-cinder-841302-4-iscsi devstack@keystone.service[208582]: Gracefully killing worker 2 (pid: 208582)... May 19 20:41:31.215737 dv-cinder-841302-4-iscsi devstack@keystone.service[208581]: Gracefully killing worker 1 (pid: 208581)... May 19 20:41:31.217734 dv-cinder-841302-4-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 20:41:32.216075 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: worker 1 buried after 1 seconds May 19 20:41:32.216075 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: worker 2 buried after 1 seconds May 19 20:41:32.216075 dv-cinder-841302-4-iscsi devstack@keystone.service[208570]: goodbye to uWSGI. May 19 20:41:32.217644 dv-cinder-841302-4-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 20:41:32.218223 dv-cinder-841302-4-iscsi systemd[1]: Stopped Devstack devstack@keystone.service. May 19 20:48:25.123344 dv-cinder-841302-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 19 20:48:25.133953 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 19 20:48:25 2022] *** May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: nodename: dv-cinder-841302-4-iscsi May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: machine: x86_64 May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: clock source: unix May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: pcre jit disabled May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: detected number of CPU cores: 4 May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: current working directory: / May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: detected binary path: /usr/bin/uwsgi-core May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: your processes number limit is 47680 May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: your memory page size is 4096 bytes May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: detected max file descriptor number: 2048 May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: lock engine: pthread robust mutexes May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: thunder lock: enabled May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 19 20:48:25.140382 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 19 20:48:25.182503 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: Python main interpreter initialized at 0x55d75c4489c0 May 19 20:48:25.182503 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: python threads support enabled May 19 20:48:25.182503 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: your server socket listen backlog is limited to 100 connections May 19 20:48:25.182503 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: your mercy for graceful operations on workers is 90 seconds May 19 20:48:25.182917 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: mapped 403077 bytes (393 KB) for 2 cores May 19 20:48:25.182917 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: *** Operational MODE: preforking *** May 19 20:48:25.183150 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: *** uWSGI is running in multiple interpreter mode *** May 19 20:48:25.183150 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: spawned uWSGI master process (pid: 278902) May 19 20:48:25.183365 dv-cinder-841302-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 19 20:48:25.184647 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: spawned uWSGI worker 1 (pid: 278914, cores: 1) May 19 20:48:25.184647 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: spawned uWSGI worker 2 (pid: 278915, cores: 1) May 19 20:48:25.184647 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 19 20:48:30.959899 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:48:30.959899 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: from cryptography.utils import int_from_bytes May 19 20:48:30.960851 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:48:30.960851 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: from cryptography.utils import int_from_bytes May 19 20:48:31.007851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:48:31.007851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: from cryptography.utils import int_from_bytes May 19 20:48:31.008561 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 19 20:48:31.008561 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: from cryptography.utils import int_from_bytes May 19 20:48:31.011160 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:48:31.011364 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:48:31.011564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] command line args: [] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:48:31.011844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:48:31.012029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ================================================================================ {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:48:31.012292 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] admin_token = **** {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.012572 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] config_dir = [] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.012769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.012965 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] config_source = [] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.013152 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] debug = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.013364 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.014606 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.014606 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] insecure_debug = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.014606 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.014606 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.015088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] list_limit = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.015088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] log_config_append = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.015088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.015356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] log_dir = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.015356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] log_file = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.015549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.015807 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.016064 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.016322 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.016532 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.016716 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.016969 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.017224 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.017467 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.017708 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.017949 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.018190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.018475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.018732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] notification_format = cadf {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.019017 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.019529 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.019716 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] publish_errors = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.019956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.020194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.020407 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.020677 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.020921 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.021167 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] standard_threads = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.021443 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] strict_password_check = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.021660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.021844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] use_eventlog = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.022034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] use_journal = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.022197 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] use_json = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.022378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] use_stderr = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.022560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] use_syslog = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.022800 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] watch_log_file = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.023080 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.023334 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.023609 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.023860 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.024122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.024388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.024652 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.024902 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.025173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.025433 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.025695 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.025939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.026201 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.026448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.026707 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.026967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.027216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.027482 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.027749 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.028007 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.028254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.028554 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.028839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.029104 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.029368 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.029611 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.029856 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.030098 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.030345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.030604 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.030867 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.031108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.031349 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.031608 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.031869 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.032116 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.032400 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.032646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.032894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.033145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.033386 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.033626 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.033867 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.034133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.034380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.034651 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.034892 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.035141 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.035381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.035626 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.035870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.036129 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.036391 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.036642 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.036888 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.037133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.037379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.037627 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.037865 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.038126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.038370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.038617 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.038856 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.039125 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.039370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.039607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.039853 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.040098 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.040407 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.040657 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.040918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.041172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.041417 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.041725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.041974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.042152 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.enabled = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.042399 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.042644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.042894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.043145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.043392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.043636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.043881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.044125 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.044424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.044677 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.044930 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.045187 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.045440 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.045686 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.045931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.046177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.046416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.046655 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.046894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.047137 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.047374 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.047624 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.047870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.048129 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.048397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.048799 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.049074 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.049329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.049572 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] auth.external = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.050351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] auth.mapped = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.050660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.050905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.051148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] auth.password = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.051390 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] auth.token = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.051639 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.051886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] catalog.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.052144 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.052408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.052669 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.052927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.053168 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.053417 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] credential.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.053674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] credential.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.053932 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.054190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.054437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.054691 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.054937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.055211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.055473 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.055728 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.055983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.056083 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 19 20:48:31.056600 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.056815 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 19 20:48:31.056914 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.057008 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] command line args: [] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 19 20:48:31.057094 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.057188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 19 20:48:31.057287 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ================================================================================ {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 19 20:48:31.057379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.057552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] admin_token = **** {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.057655 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.057803 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] federation.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.057909 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] config_dir = [] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.058009 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.058101 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.058193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] config_source = [] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.058283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] federation.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.058374 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] debug = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.058535 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.058650 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.058771 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.058858 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.058949 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] insecure_debug = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.059035 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.059190 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.059289 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.059445 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.059544 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.059682 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] list_limit = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.059774 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.059871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] log_config_append = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.059984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.060084 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.060205 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] log_dir = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.060302 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.060422 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] log_file = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.060521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.060626 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.060718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.060883 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.060983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.061137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.061244 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.061391 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.061498 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.061591 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.061683 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.061862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.061958 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.062050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.062150 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.062237 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.062369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.062539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.062641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.062733 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.062870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.062962 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.063110 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.063216 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.063348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.063447 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.063592 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.063691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] notification_format = cadf {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.063844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.063950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.064100 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.064371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.064479 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.88/identity {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.064582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] publish_errors = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.064676 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.064789 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.064888 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.065024 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.065163 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.065269 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.065988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.066106 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.066241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.066350 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.066485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.066584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] standard_threads = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.066734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.066828 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] strict_password_check = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.066974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.067067 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.067158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] use_eventlog = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.067248 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.067348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] use_journal = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.067446 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.067538 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] use_json = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.067685 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] use_stderr = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.067774 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.067873 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] use_syslog = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.067959 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.068057 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] watch_log_file = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 19 20:48:31.068160 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.068289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.068429 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.068559 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.068687 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.068831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.068938 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.069079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.069186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.069335 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.069435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.069579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.069678 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.069834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.069941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.070077 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.070187 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.password = **** {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.070332 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.070432 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.070583 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.070682 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.070835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.070934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.071074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.071184 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.071318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.071417 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.071559 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.071657 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.071813 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.071912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.072050 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.072147 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.072290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.072434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.072578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.072709 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.072833 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.072977 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.073084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.073230 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.073337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.073475 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.073575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.073737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.073836 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.074000 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.074099 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.074257 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.074355 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.074494 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.074593 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.074736 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.074835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.074974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.075073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.075212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.075305 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.075454 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.075548 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.075689 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.075786 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.075932 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.076031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.076174 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.076279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.076433 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.076537 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.076670 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.076778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.076926 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.077053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.077150 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.077242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.077408 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.077503 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.077647 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.077747 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.077888 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.078003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.078128 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.078222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.078364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.078465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.078599 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.078700 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.078834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.078935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.079072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.079181 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.079313 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.079422 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.079564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.079658 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.079800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.079895 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.080043 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.080143 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.080279 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.080434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] policy.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.080538 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.080634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.080777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.080878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.081056 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.081158 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.081293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.081388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] receipt.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.081536 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.081642 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.081777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.081877 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.082018 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.082119 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.082258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.082364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.082505 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.082611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.082741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.082840 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] resource.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.082995 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.083089 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.083235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.083417 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] resource.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.083520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.083607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.083697 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.083789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.083951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.084046 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.084195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.084290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] revoke.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.084457 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.084560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.084701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.084795 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.084944 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.085054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] role.cache_time = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.085187 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] role.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.085286 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.085443 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] role.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.085546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.085683 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] role.list_limit = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.085777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.085930 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.086029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.086173 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.086272 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.086416 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.086515 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.086658 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.086758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.086910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.087009 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.enabled = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.087150 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.087248 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.087392 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.087492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.087639 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.087732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.087878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.087975 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.088115 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.088208 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.088379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.088520 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.088614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.088703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.088849 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.088938 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.089119 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.089315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.089408 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.089560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.089664 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.089820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.089920 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.090063 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.090156 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.090306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.090411 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.090547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.090641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.090789 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.090881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.091035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.091151 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.091272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.091378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.091509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.091619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.091752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.091845 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.091987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.092081 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.092229 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.092328 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.092480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.092573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.092737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.092837 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.092979 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.093079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.093245 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.093345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.093498 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.093591 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.093737 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] token.cache_time = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.093851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.093983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] token.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.094092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] auth.external = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.094230 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.094330 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] auth.mapped = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.094487 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] token.provider = fernet {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.094581 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.094732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.094834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.094981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.095078 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] auth.password = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.095341 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] auth.token = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.095487 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.095599 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.095730 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] catalog.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.095824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.095980 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.096072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.096216 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.096309 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] trust.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.096484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.096575 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.096764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.096865 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.097001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.097095 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.097244 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] credential.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.097346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.097500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] credential.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.097609 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.097739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.097848 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.097992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.098098 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.098234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.098327 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.098485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.098594 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.connection = **** {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.098728 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.098842 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.098997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.099098 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.099257 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.099357 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.099506 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.099627 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.099748 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.099851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.099994 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.100098 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.100237 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.100412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.100504 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.100603 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.100747 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.100931 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.101031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.101197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.101297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.101441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] federation.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.101552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.101685 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.101786 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.101930 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.102029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] federation.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.102182 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.102283 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.102421 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.102513 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.102668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.102775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.102910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.103016 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.103109 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=278915) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:48:31.103195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.103401 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.103661 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.103903 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.104155 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.104421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.104679 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.105056 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.105298 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.105541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.105794 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.106031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.106274 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.106528 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.106784 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.107022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.107258 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.107492 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.107740 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.107993 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.108244 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.108529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.108783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.109061 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.109304 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.109547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.109782 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.110024 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.110258 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.110498 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.110741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.110982 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.111246 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.111483 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.111737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.111990 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.112787 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.113060 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.113312 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.113548 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.113792 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.114027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.password = **** {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.114270 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.114513 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.114763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.115005 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.115248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.115502 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.115755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.115991 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.116226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.116503 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.116756 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.116998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.117239 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.117478 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.117825 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.118082 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.118349 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.118587 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.118842 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.119092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.119344 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.119583 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.119818 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.120058 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.120298 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.120563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.120800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.121052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.121303 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.121554 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.121807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.122056 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.122289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.122539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.122780 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.123025 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.123263 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.123501 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.123750 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.124002 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.124245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.124527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] policy.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.124763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.125013 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.125254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.125506 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] receipt.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.125747 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.126002 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.126275 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.126485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.126728 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.126974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] resource.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.127230 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.127485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] resource.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.127727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.128568 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.128828 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.129071 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] revoke.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.129325 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.129568 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.129813 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] role.cache_time = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.130054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] role.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.130307 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] role.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.130543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] role.list_limit = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.130797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.131053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.131306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.131558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.131809 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.132061 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.132313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.132589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.132833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.133085 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.133338 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.133590 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.133842 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.134272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.134515 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.134771 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.135027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.135119 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.135281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.135388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.135537 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.135637 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.135777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.136036 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.136293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.136561 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.136799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.137036 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.137272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.137527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.137928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.138187 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.138430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.138672 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.138908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] token.cache_time = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.139150 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] token.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.139392 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.139647 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] token.provider = fernet {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.139912 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.140178 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.140532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.140782 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.141034 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.141285 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.141540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] trust.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.141783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.142030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.142272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.142524 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.142783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.143018 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.143268 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.143597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.143863 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.connection = **** {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.144113 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.144390 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.144518 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.144636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.144884 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.145126 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.145367 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.145608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.145848 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.146098 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.146340 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.146587 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.146827 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.147079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.147318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.147566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.147801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.148047 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.148286 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 19 20:48:31.148483 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=278914) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 19 20:48:31.173376 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.173612 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.174623 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.175437 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.175670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.186426 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.205110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.205331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.227539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.243717 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.279393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.279632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.281050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.281297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.348325 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.348582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.348670 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.348868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.348963 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.349052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.349146 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.349242 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.349349 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.349451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.349542 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.349649 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.374922 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.375138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.375275 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.375363 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.375452 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.375547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.375636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.375736 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.375843 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.375973 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.376072 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.376196 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.376309 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.376453 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.388597 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.388813 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.402220 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.402454 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.402686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.402916 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.403141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.403365 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.403583 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=278914) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.403852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.404095 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.404326 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.404587 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.404811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.405034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.405257 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=278915) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 19 20:48:31.421634 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.421866 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.421979 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:48:31.422075 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:48:31.422190 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.422286 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.422389 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:48:31.422492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 19 20:48:31.422629 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.422736 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.422893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.422990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.423156 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.423308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.423408 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.423513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.435429 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.435712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.435990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.436279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.437067 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.437305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.437580 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.437869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.444551 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.446066 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.447873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.448161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.448589 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.448881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.449173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.449394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.449491 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.449676 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.449771 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.449961 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.450117 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.450376 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.450548 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.450675 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.450783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.450953 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.451066 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.451279 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.451556 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.451831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.452106 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.452402 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.480172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.481858 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.481960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.482143 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.483533 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.483817 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.488097 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.488406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.488695 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.489774 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.490065 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.490357 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.496123 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.496433 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.497882 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.498169 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.501132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.501428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.501720 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.502012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.502295 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.502579 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.502873 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.502983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.503162 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.503280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.503450 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.503731 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.504007 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.504282 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.504589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.504859 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.522031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.522317 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.523564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.523846 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.526895 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.528424 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.530563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.530867 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.532055 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.532371 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.536603 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.537897 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.540204 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.541497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.544123 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.544458 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.544754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.545046 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.545275 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.545389 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.545576 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.545672 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.545863 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.546146 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.546421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.546701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.566051 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.566476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.566762 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.566969 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.567100 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.567399 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.567676 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.568048 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.575375 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.576135 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.577220 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.577509 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.577967 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.578257 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.580695 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.581532 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.581825 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.581951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.582124 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.582243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.582410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.582530 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.582831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.583125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.596281 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.596733 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.597451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.597571 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.597749 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.597855 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.598046 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.598150 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.598326 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.598424 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.598610 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.598715 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.598889 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.599167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.617460 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.618902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.620877 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.622323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.624309 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.625733 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.625969 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.627395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.627674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.628012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.628311 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.629128 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.629424 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.629710 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.636008 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.637312 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.637728 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.638984 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.641048 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.641348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.641639 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.641920 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.642289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.642586 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.642871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.643148 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.652018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.652429 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.652768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.653137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.653444 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.653728 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.661609 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=278915) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.661919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.662207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.662490 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.662593 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=278914) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 19 20:48:31.662768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.662864 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.663077 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.663183 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.663359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.663465 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.663635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.663750 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.663910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.664017 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.664187 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.664291 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.664511 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.664610 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.664783 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.664881 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.665052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.665160 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.665326 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=278915) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.665440 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.665608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.665880 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.666149 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=278914) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 19 20:48:31.705637 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d75c4489c0 pid: 278915 (default app) May 19 20:48:31.706625 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d75c4489c0 pid: 278914 (default app) May 19 20:48:31.709916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bbf97-31b0-47fa-a055-48d1e8cca240 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:31.710217 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bbf97-31b0-47fa-a055-48d1e8cca240 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:31.710505 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bbf97-31b0-47fa-a055-48d1e8cca240 None None] PATH_INFO: `/v3` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:31.714943 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 1/1] 10.222.0.88 () {50 vars in 791 bytes} [Thu May 19 20:48:31 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:40.685027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce33f9c-a0cf-4047-b494-26e2384a322c None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:40.685577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce33f9c-a0cf-4047-b494-26e2384a322c None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:40.685676 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce33f9c-a0cf-4047-b494-26e2384a322c None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:40.690015 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 1/2] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:48:40.695515 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0abe-1408-4e69-b1b9-587c3d1f5540 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:40.695940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0abe-1408-4e69-b1b9-587c3d1f5540 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:40.696060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda0abe-1408-4e69-b1b9-587c3d1f5540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:40.764985 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: INFO dbcounter [None req-2cda0abe-1408-4e69-b1b9-587c3d1f5540 None None] Registered counter for database keystone May 19 20:48:40.788259 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG oslo_db.sqlalchemy.engines [None req-2cda0abe-1408-4e69-b1b9-587c3d1f5540 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=278915) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:48:40.794741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writer thread running {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:48:40.977525 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG passlib.handlers.bcrypt [None req-2cda0abe-1408-4e69-b1b9-587c3d1f5540 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=278915) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:48:40.977865 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG passlib.handlers.bcrypt [None req-2cda0abe-1408-4e69-b1b9-587c3d1f5540 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=278915) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:48:41.035219 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-2cda0abe-1408-4e69-b1b9-587c3d1f5540 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:41.036874 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-2cda0abe-1408-4e69-b1b9-587c3d1f5540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:41.164195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 2/3] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 20:48:41.187177 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce33f9c-a0cf-4047-b494-26e2384a322c None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:41.256482 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: INFO dbcounter [None req-2ce33f9c-a0cf-4047-b494-26e2384a322c None None] Registered counter for database keystone May 19 20:48:41.276951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG oslo_db.sqlalchemy.engines [None req-2ce33f9c-a0cf-4047-b494-26e2384a322c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=278914) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 19 20:48:41.283289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writer thread running {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 19 20:48:41.405685 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce33f9c-a0cf-4047-b494-26e2384a322c None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:41.410652 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252aefb-eb59-4105-9da2-19a03441e52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a03e4df28e472baf1019b7ccaefbc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:41.411343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252aefb-eb59-4105-9da2-19a03441e52b admin admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:41.411633 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252aefb-eb59-4105-9da2-19a03441e52b admin admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:41.411917 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252aefb-eb59-4105-9da2-19a03441e52b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:41.420558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252aefb-eb59-4105-9da2-19a03441e52b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:41.471315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9252aefb-eb59-4105-9da2-19a03441e52b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:48:41.472202 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252aefb-eb59-4105-9da2-19a03441e52b admin admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:41.482273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 2/4] 10.222.0.88 () {58 vars in 1324 bytes} [Thu May 19 20:48:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 19 20:48:41.488994 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda0abe-1408-4e69-b1b9-587c3d1f5540 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:41.503926 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda0abe-1408-4e69-b1b9-587c3d1f5540 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:41.508465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85882b44-5198-48e2-aa99-3f6188cdd1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a03e4df28e472baf1019b7ccaefbc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:41.509544 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85882b44-5198-48e2-aa99-3f6188cdd1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:41.509834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85882b44-5198-48e2-aa99-3f6188cdd1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:41.510113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85882b44-5198-48e2-aa99-3f6188cdd1b9 admin admin] PATH_INFO: `/v3/projects/06a03e4df28e472baf1019b7ccaefbc6` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:41.511663 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85882b44-5198-48e2-aa99-3f6188cdd1b9 admin admin] RBAC: Authorizing `identity:get_project(project_id=06a03e4df28e472baf1019b7ccaefbc6)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:41.673451 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: WARNING keystone.common.rbac_enforcer.enforcer [None req-85882b44-5198-48e2-aa99-3f6188cdd1b9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 19 20:48:41.673803 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85882b44-5198-48e2-aa99-3f6188cdd1b9 admin admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:41.679073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 3/5] 10.222.0.88 () {56 vars in 1207 bytes} [Thu May 19 20:48:41 2022] GET /identity/v3/projects/06a03e4df28e472baf1019b7ccaefbc6 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:43.909487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c1509-4275-4c3e-8726-5c602bf0f248 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:43.910072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c1509-4275-4c3e-8726-5c602bf0f248 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:43.910072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c1509-4275-4c3e-8726-5c602bf0f248 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:43.914331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 3/6] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:48:43.920736 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45d454-cf9c-4fb5-8c6e-d670d49eedc2 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:43.921297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45d454-cf9c-4fb5-8c6e-d670d49eedc2 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:43.921297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45d454-cf9c-4fb5-8c6e-d670d49eedc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:43.941520 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-4b45d454-cf9c-4fb5-8c6e-d670d49eedc2 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:43.943082 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-4b45d454-cf9c-4fb5-8c6e-d670d49eedc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:43.951318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 4/7] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 19 20:48:43.974077 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c1509-4275-4c3e-8726-5c602bf0f248 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:43.986306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c1509-4275-4c3e-8726-5c602bf0f248 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:43.991182 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bf1da-4961-4842-bf98-e61a0a5584a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a03e4df28e472baf1019b7ccaefbc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:43.991822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bf1da-4961-4842-bf98-e61a0a5584a0 admin admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:43.992111 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bf1da-4961-4842-bf98-e61a0a5584a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:43.992425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bf1da-4961-4842-bf98-e61a0a5584a0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:44.000450 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bf1da-4961-4842-bf98-e61a0a5584a0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:44.002229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bf1da-4961-4842-bf98-e61a0a5584a0 admin admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:44.011183 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 4/8] 10.222.0.88 () {58 vars in 1324 bytes} [Thu May 19 20:48:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 19 20:48:44.016998 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45d454-cf9c-4fb5-8c6e-d670d49eedc2 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:44.026411 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45d454-cf9c-4fb5-8c6e-d670d49eedc2 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:44.030903 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccab19-a3a5-4e1d-a610-ecd155a3dbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06a03e4df28e472baf1019b7ccaefbc6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:44.031789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccab19-a3a5-4e1d-a610-ecd155a3dbb0 admin admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:44.032072 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccab19-a3a5-4e1d-a610-ecd155a3dbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:44.032379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccab19-a3a5-4e1d-a610-ecd155a3dbb0 admin admin] PATH_INFO: `/v3/users/4f6c30de67e643ecb5582d374f07cb7c` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:44.033799 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccab19-a3a5-4e1d-a610-ecd155a3dbb0 admin admin] RBAC: Authorizing `identity:get_user(user_id=4f6c30de67e643ecb5582d374f07cb7c)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:44.036183 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccab19-a3a5-4e1d-a610-ecd155a3dbb0 admin admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:44.037864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 5/9] 10.222.0.88 () {56 vars in 1198 bytes} [Thu May 19 20:48:44 2022] GET /identity/v3/users/4f6c30de67e643ecb5582d374f07cb7c => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:46.378173 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22056fa-c960-4a3b-a7aa-b37033ebd785 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:46.378748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22056fa-c960-4a3b-a7aa-b37033ebd785 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:46.378748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22056fa-c960-4a3b-a7aa-b37033ebd785 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:46.380099 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 5/10] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:48:46.385670 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8716c2-dbbe-47f2-a9c3-1ae1e5427abc None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:46.386139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8716c2-dbbe-47f2-a9c3-1ae1e5427abc None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:46.386223 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8716c2-dbbe-47f2-a9c3-1ae1e5427abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:46.396708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29b3f3-30cf-4256-ba2a-51fdf1d9bc8c None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:46.397005 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29b3f3-30cf-4256-ba2a-51fdf1d9bc8c None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:46.397291 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29b3f3-30cf-4256-ba2a-51fdf1d9bc8c None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:46.398647 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 6/11] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:48:46.403586 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-ab8716c2-dbbe-47f2-a9c3-1ae1e5427abc None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:46.403900 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b5653-f6fc-4f65-a95b-eeb20d468a7a None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:46.404193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b5653-f6fc-4f65-a95b-eeb20d468a7a None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:46.404507 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b5653-f6fc-4f65-a95b-eeb20d468a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:46.404829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-ab8716c2-dbbe-47f2-a9c3-1ae1e5427abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:46.447463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG passlib.handlers.bcrypt [None req-114b5653-f6fc-4f65-a95b-eeb20d468a7a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=278914) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 19 20:48:46.447871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG passlib.handlers.bcrypt [None req-114b5653-f6fc-4f65-a95b-eeb20d468a7a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=278914) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 19 20:48:46.486671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 6/12] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:48:46.495852 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-114b5653-f6fc-4f65-a95b-eeb20d468a7a None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:46.497486 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-114b5653-f6fc-4f65-a95b-eeb20d468a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:46.509795 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8716c2-dbbe-47f2-a9c3-1ae1e5427abc None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:46.535483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:46.598703 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 7/13] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:48:46.617058 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8716c2-dbbe-47f2-a9c3-1ae1e5427abc None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:46.620658 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b5653-f6fc-4f65-a95b-eeb20d468a7a None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:46.653055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047c8c-d4bf-4a6d-aad8-2f3fa7fe46d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:46.653783 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047c8c-d4bf-4a6d-aad8-2f3fa7fe46d3 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:46.654073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047c8c-d4bf-4a6d-aad8-2f3fa7fe46d3 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:46.654354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5047c8c-d4bf-4a6d-aad8-2f3fa7fe46d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:46.666329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5047c8c-d4bf-4a6d-aad8-2f3fa7fe46d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-e5047c8c-d4bf-4a6d-aad8-2f3fa7fe46d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:48:46.677634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:48:46.679113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:48:46.680454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 7/14] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:48:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:48:46.684872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047c8c-d4bf-4a6d-aad8-2f3fa7fe46d3 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:46.726449 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b5653-f6fc-4f65-a95b-eeb20d468a7a None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:46.728820 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:46.764451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f20a05-7f5d-47d4-b5f8-3881fd68b004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:46.765374 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f20a05-7f5d-47d4-b5f8-3881fd68b004 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:46.765668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f20a05-7f5d-47d4-b5f8-3881fd68b004 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:46.765951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f20a05-7f5d-47d4-b5f8-3881fd68b004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:46.772378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:46.791551 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5047c8c-d4bf-4a6d-aad8-2f3fa7fe46d3 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:46.825285 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd905e8a-2a8e-4924-886d-5ee1848c52b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:46.825959 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd905e8a-2a8e-4924-886d-5ee1848c52b4 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:46.826246 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd905e8a-2a8e-4924-886d-5ee1848c52b4 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:46.826530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd905e8a-2a8e-4924-886d-5ee1848c52b4 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:46.827298 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd905e8a-2a8e-4924-886d-5ee1848c52b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:48:46.827614 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd905e8a-2a8e-4924-886d-5ee1848c52b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:46.829069 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd905e8a-2a8e-4924-886d-5ee1848c52b4 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:46.837598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f20a05-7f5d-47d4-b5f8-3881fd68b004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:46.839128 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f20a05-7f5d-47d4-b5f8-3881fd68b004 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:46.842108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 8/15] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:48:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:46.851309 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd905e8a-2a8e-4924-886d-5ee1848c52b4 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:46.943988 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 8/16] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:48:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:48:46.949688 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f20a05-7f5d-47d4-b5f8-3881fd68b004 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:46.952523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:46.959936 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd905e8a-2a8e-4924-886d-5ee1848c52b4 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:46.994199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695c127-0879-4f1c-9e9e-0632ad6c0e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:46.995091 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695c127-0879-4f1c-9e9e-0632ad6c0e83 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:46.995380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695c127-0879-4f1c-9e9e-0632ad6c0e83 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:46.995660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695c127-0879-4f1c-9e9e-0632ad6c0e83 None admin] PATH_INFO: `/v3/users/4f6c30de67e643ecb5582d374f07cb7c` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:46.996938 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695c127-0879-4f1c-9e9e-0632ad6c0e83 None admin] RBAC: Authorizing `identity:get_user(user_id=4f6c30de67e643ecb5582d374f07cb7c)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:46.998474 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695c127-0879-4f1c-9e9e-0632ad6c0e83 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:47.000115 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 9/17] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:48:46 2022] GET /identity/v3/users/4f6c30de67e643ecb5582d374f07cb7c => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:48:47.006010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695c127-0879-4f1c-9e9e-0632ad6c0e83 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:47.021077 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:47.051596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f20a05-7f5d-47d4-b5f8-3881fd68b004 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:47.087580 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721a769-d300-49dd-a037-29a249ca8501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:47.088411 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a769-d300-49dd-a037-29a249ca8501 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:47.088695 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a769-d300-49dd-a037-29a249ca8501 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:47.088969 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a769-d300-49dd-a037-29a249ca8501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:47.107463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721a769-d300-49dd-a037-29a249ca8501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:47.108980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721a769-d300-49dd-a037-29a249ca8501 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:47.110542 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695c127-0879-4f1c-9e9e-0632ad6c0e83 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-f721a769-d300-49dd-a037-29a249ca8501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:48:47.130273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:48:47.131394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:48:47.132412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:48:47.132412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:48:47.132412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:48:47.132412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:48:47.132412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 9/18] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:48:47.137584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721a769-d300-49dd-a037-29a249ca8501 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:47.147803 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c1ac7-e600-46a8-8713-b750c9e18bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:47.148642 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c1ac7-e600-46a8-8713-b750c9e18bb2 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:47.148934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c1ac7-e600-46a8-8713-b750c9e18bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:47.149277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c1ac7-e600-46a8-8713-b750c9e18bb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:47.150522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c1ac7-e600-46a8-8713-b750c9e18bb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:47.152018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c1ac7-e600-46a8-8713-b750c9e18bb2 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:47.153708 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 10/19] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:48:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:48:47.159350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c1ac7-e600-46a8-8713-b750c9e18bb2 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:47.200179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:47.248619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:47.271936 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c1ac7-e600-46a8-8713-b750c9e18bb2 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:47.320941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94f2c3-ca39-4633-ae59-d5588504ec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:47.321614 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94f2c3-ca39-4633-ae59-d5588504ec45 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:47.321902 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94f2c3-ca39-4633-ae59-d5588504ec45 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:47.322179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94f2c3-ca39-4633-ae59-d5588504ec45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:47.323031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94f2c3-ca39-4633-ae59-d5588504ec45 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.domain.id=default, user.id=4f6c30de67e643ecb5582d374f07cb7c) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:48:47.323343 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94f2c3-ca39-4633-ae59-d5588504ec45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:47.324882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94f2c3-ca39-4633-ae59-d5588504ec45 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:47.332495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 11/20] 10.222.0.88 () {56 vars in 1322 bytes} [Thu May 19 20:48:47 2022] GET /identity/v3/role_assignments?user.id=4f6c30de67e643ecb5582d374f07cb7c&scope.domain.id=default&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 233 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:48:47.361722 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721a769-d300-49dd-a037-29a249ca8501 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:47.399457 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cc57e-349f-4842-8668-a973fdc0d9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:47.400205 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cc57e-349f-4842-8668-a973fdc0d9de None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:47.400462 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cc57e-349f-4842-8668-a973fdc0d9de None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:47.400747 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cc57e-349f-4842-8668-a973fdc0d9de None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:47.401522 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cc57e-349f-4842-8668-a973fdc0d9de None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:48:47.401865 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cc57e-349f-4842-8668-a973fdc0d9de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:47.403285 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cc57e-349f-4842-8668-a973fdc0d9de None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:47.423410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 10/21] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:48:50.076263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce8ebc-b0cd-4822-b5c2-9f58752fe511 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:50.076808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce8ebc-b0cd-4822-b5c2-9f58752fe511 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:50.076927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce8ebc-b0cd-4822-b5c2-9f58752fe511 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:50.078231 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 12/22] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:48:50.085874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608acea9-1645-474f-b548-e83437e6e7b4 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:50.086370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608acea9-1645-474f-b548-e83437e6e7b4 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:50.086455 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608acea9-1645-474f-b548-e83437e6e7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:50.113421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-608acea9-1645-474f-b548-e83437e6e7b4 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:50.114641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-608acea9-1645-474f-b548-e83437e6e7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:50.211859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01514f-8de7-4e50-be36-8710cd47bce4 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:50.212045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01514f-8de7-4e50-be36-8710cd47bce4 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:50.212358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01514f-8de7-4e50-be36-8710cd47bce4 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:50.214058 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 13/23] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:48:50.215131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 11/24] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:48:50.219895 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165810-bf17-4e41-ae7d-ab7b307b8172 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:50.220389 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165810-bf17-4e41-ae7d-ab7b307b8172 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:50.220876 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28165810-bf17-4e41-ae7d-ab7b307b8172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:50.243726 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-28165810-bf17-4e41-ae7d-ab7b307b8172 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:50.245234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-28165810-bf17-4e41-ae7d-ab7b307b8172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:50.274114 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608acea9-1645-474f-b548-e83437e6e7b4 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:50.343645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 14/25] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:48:50.344088 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:50.373505 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28165810-bf17-4e41-ae7d-ab7b307b8172 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:50.459232 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608acea9-1645-474f-b548-e83437e6e7b4 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:50.479408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:50.489371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28165810-bf17-4e41-ae7d-ab7b307b8172 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:50.515798 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1d141-5a86-4828-ba24-b24886c80786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:50.516698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1d141-5a86-4828-ba24-b24886c80786 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:50.516996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1d141-5a86-4828-ba24-b24886c80786 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:50.517285 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1d141-5a86-4828-ba24-b24886c80786 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:50.532192 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1d141-5a86-4828-ba24-b24886c80786 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-01e1d141-5a86-4828-ba24-b24886c80786 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:48:50.543059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:48:50.544144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:48:50.545136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 12/26] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:48:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:48:50.546394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d1e3e-d056-4003-8e3a-c786b1681000 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:50.546394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d1e3e-d056-4003-8e3a-c786b1681000 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:50.547733 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d1e3e-d056-4003-8e3a-c786b1681000 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:50.548366 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 13/27] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:48:50.555620 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d1e3e-d056-4003-8e3a-c786b1681000 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:50.613233 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15318a-b78e-490d-95eb-f23a6b868b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:50.613951 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15318a-b78e-490d-95eb-f23a6b868b86 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:50.614241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15318a-b78e-490d-95eb-f23a6b868b86 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:50.614523 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e15318a-b78e-490d-95eb-f23a6b868b86 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:50.615335 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15318a-b78e-490d-95eb-f23a6b868b86 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:50.616877 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e15318a-b78e-490d-95eb-f23a6b868b86 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:50.699500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d1e3e-d056-4003-8e3a-c786b1681000 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:50.708795 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:50.742263 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db4060-f56e-44cc-9f0a-fe2e5f9b7878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:50.742886 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db4060-f56e-44cc-9f0a-fe2e5f9b7878 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:50.743181 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db4060-f56e-44cc-9f0a-fe2e5f9b7878 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:50.743470 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db4060-f56e-44cc-9f0a-fe2e5f9b7878 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:50.744425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db4060-f56e-44cc-9f0a-fe2e5f9b7878 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:48:50.744748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db4060-f56e-44cc-9f0a-fe2e5f9b7878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:50.746218 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db4060-f56e-44cc-9f0a-fe2e5f9b7878 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:50.760780 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 14/28] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:48:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:50.761867 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be6f6a-15ca-4e25-b46d-b7073605c3f0 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:50.762232 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be6f6a-15ca-4e25-b46d-b7073605c3f0 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:50.762456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be6f6a-15ca-4e25-b46d-b7073605c3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:50.781967 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-e1be6f6a-15ca-4e25-b46d-b7073605c3f0 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:50.783235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-e1be6f6a-15ca-4e25-b46d-b7073605c3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:50.932837 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 15/29] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:48:50 2022] POST /identity/v3/roles => generated 219 bytes in 560 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:48:50.934495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15318a-b78e-490d-95eb-f23a6b868b86 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:50.947028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 15/30] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:48:51.011586 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be6f6a-15ca-4e25-b46d-b7073605c3f0 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:51.135293 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e15318a-b78e-490d-95eb-f23a6b868b86 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:51.148907 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:51.153970 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be6f6a-15ca-4e25-b46d-b7073605c3f0 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:51.194749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae894b9-2d99-4f65-9aff-a2b66d57857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:51.195861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae894b9-2d99-4f65-9aff-a2b66d57857d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:51.196316 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae894b9-2d99-4f65-9aff-a2b66d57857d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:51.196909 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae894b9-2d99-4f65-9aff-a2b66d57857d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:51.210263 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae894b9-2d99-4f65-9aff-a2b66d57857d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:51.212164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae894b9-2d99-4f65-9aff-a2b66d57857d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:51.223508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c36d05-82f4-427a-8cef-83724be5c408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:51.224602 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c36d05-82f4-427a-8cef-83724be5c408 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:51.225171 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c36d05-82f4-427a-8cef-83724be5c408 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:51.225688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c36d05-82f4-427a-8cef-83724be5c408 None admin] PATH_INFO: `/v3/users/4f6c30de67e643ecb5582d374f07cb7c` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:51.233179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c36d05-82f4-427a-8cef-83724be5c408 None admin] RBAC: Authorizing `identity:get_user(user_id=4f6c30de67e643ecb5582d374f07cb7c)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-8ae894b9-2d99-4f65-9aff-a2b66d57857d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:48:51.233355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:48:51.234318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:48:51.235941 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:48:51.235941 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:48:51.235941 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:48:51.235941 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:48:51.235941 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 16/31] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:48:51.238235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c36d05-82f4-427a-8cef-83724be5c408 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:51.240360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae894b9-2d99-4f65-9aff-a2b66d57857d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:51.240525 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 16/32] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:48:50 2022] GET /identity/v3/users/4f6c30de67e643ecb5582d374f07cb7c => generated 247 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:51.246092 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c36d05-82f4-427a-8cef-83724be5c408 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:51.252703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:51.355813 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae894b9-2d99-4f65-9aff-a2b66d57857d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:51.397027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb2a12-c615-48b8-a59f-51dc2e81576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:51.398066 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb2a12-c615-48b8-a59f-51dc2e81576c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:51.398563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb2a12-c615-48b8-a59f-51dc2e81576c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:51.399011 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb2a12-c615-48b8-a59f-51dc2e81576c None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:51.399931 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb2a12-c615-48b8-a59f-51dc2e81576c None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:48:51.400459 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb2a12-c615-48b8-a59f-51dc2e81576c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:51.402062 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb2a12-c615-48b8-a59f-51dc2e81576c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:51.415830 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 17/33] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:48:51.417620 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92866b4d-00ee-4b77-9c23-ec4e860694ee None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:51.418077 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92866b4d-00ee-4b77-9c23-ec4e860694ee None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:51.418540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92866b4d-00ee-4b77-9c23-ec4e860694ee None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:51.420180 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 18/34] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:48:51.421575 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92866b4d-00ee-4b77-9c23-ec4e860694ee None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:51.425237 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c36d05-82f4-427a-8cef-83724be5c408 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:51.438328 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:51.469063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528934c3-71a3-4b94-8079-4f44c7e89493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:51.470284 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528934c3-71a3-4b94-8079-4f44c7e89493 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:51.470789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528934c3-71a3-4b94-8079-4f44c7e89493 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:51.471304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528934c3-71a3-4b94-8079-4f44c7e89493 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:51.472917 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528934c3-71a3-4b94-8079-4f44c7e89493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:51.474615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528934c3-71a3-4b94-8079-4f44c7e89493 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:51.476589 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 17/35] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:48:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:51.478138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df6875-8164-420f-ac75-bb3403a7738d None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:51.478433 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df6875-8164-420f-ac75-bb3403a7738d None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:51.478723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df6875-8164-420f-ac75-bb3403a7738d None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:51.480026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 18/36] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:48:51.482348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dfdd2-ac98-48e4-a153-4c88211e77fa None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:51.482348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dfdd2-ac98-48e4-a153-4c88211e77fa None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:51.482348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dfdd2-ac98-48e4-a153-4c88211e77fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:51.502338 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-e62dfdd2-ac98-48e4-a153-4c88211e77fa None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:51.504523 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-e62dfdd2-ac98-48e4-a153-4c88211e77fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:51.552591 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92866b4d-00ee-4b77-9c23-ec4e860694ee None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:51.577339 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:51.582921 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 19/37] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:48:51.584315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dfdd2-ac98-48e4-a153-4c88211e77fa None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:51.607797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3425c-7f31-4c3f-8d04-20d66fc84031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:51.608430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3425c-7f31-4c3f-8d04-20d66fc84031 None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:51.608880 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3425c-7f31-4c3f-8d04-20d66fc84031 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:51.609170 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3425c-7f31-4c3f-8d04-20d66fc84031 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:51.610140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3425c-7f31-4c3f-8d04-20d66fc84031 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:51.611610 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3425c-7f31-4c3f-8d04-20d66fc84031 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:51.691401 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dfdd2-ac98-48e4-a153-4c88211e77fa None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:51.729414 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47793281-6f3a-4c38-99ef-eff78e2cf457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:51.730329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47793281-6f3a-4c38-99ef-eff78e2cf457 None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:51.730622 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47793281-6f3a-4c38-99ef-eff78e2cf457 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:51.730917 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47793281-6f3a-4c38-99ef-eff78e2cf457 None admin] PATH_INFO: `/v3/domains/default/users/4f6c30de67e643ecb5582d374f07cb7c/roles/ed68c2195de147c0a30d9150b33e47cf` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:51.733579 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47793281-6f3a-4c38-99ef-eff78e2cf457 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4f6c30de67e643ecb5582d374f07cb7c, role_id=ed68c2195de147c0a30d9150b33e47cf)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:51.735142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47793281-6f3a-4c38-99ef-eff78e2cf457 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:51.805905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 20/38] 10.222.0.88 () {58 vars in 1361 bytes} [Thu May 19 20:48:51 2022] PUT /identity/v3/domains/default/users/4f6c30de67e643ecb5582d374f07cb7c/roles/ed68c2195de147c0a30d9150b33e47cf => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:48:51.818179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3cbe6-6d25-4d90-b155-c4400beca4c3 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:51.818475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3cbe6-6d25-4d90-b155-c4400beca4c3 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:51.818760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3cbe6-6d25-4d90-b155-c4400beca4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:51.840795 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 19/39] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:48:51 2022] POST /identity/v3/projects => generated 301 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:48:51.841977 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3425c-7f31-4c3f-8d04-20d66fc84031 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:51.866831 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-a3c3cbe6-6d25-4d90-b155-c4400beca4c3 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:51.868494 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-a3c3cbe6-6d25-4d90-b155-c4400beca4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:51.983674 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:51.985777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:52.004240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 21/40] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:48:52.005866 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c68da3-45ca-445b-9908-4041833af461 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:52.006162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c68da3-45ca-445b-9908-4041833af461 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:52.006447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c68da3-45ca-445b-9908-4041833af461 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:52.008899 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 22/41] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:48:52.019999 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f17a9-26a0-4a45-8493-29ab3a1b0a7a None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:52.019999 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f17a9-26a0-4a45-8493-29ab3a1b0a7a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:52.019999 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f17a9-26a0-4a45-8493-29ab3a1b0a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:52.043696 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-972f17a9-26a0-4a45-8493-29ab3a1b0a7a None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:52.045488 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-972f17a9-26a0-4a45-8493-29ab3a1b0a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:52.051608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3425c-7f31-4c3f-8d04-20d66fc84031 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:52.098501 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110126b-7460-4603-bf90-832facebb95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:52.099482 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110126b-7460-4603-bf90-832facebb95c None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:52.100322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110126b-7460-4603-bf90-832facebb95c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:52.100811 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110126b-7460-4603-bf90-832facebb95c None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:52.101762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110126b-7460-4603-bf90-832facebb95c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:52.103957 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110126b-7460-4603-bf90-832facebb95c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:52.138633 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 23/42] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:48:52.139431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f17a9-26a0-4a45-8493-29ab3a1b0a7a None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:52.190684 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 20/43] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:48:51 2022] POST /identity/v3/roles => generated 225 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:48:52.191602 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110126b-7460-4603-bf90-832facebb95c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:52.278220 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f17a9-26a0-4a45-8493-29ab3a1b0a7a None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:52.305026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:52.323725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488553d6-ed69-49b4-988a-c6c1b9014e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:52.324459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488553d6-ed69-49b4-988a-c6c1b9014e35 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:52.324757 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488553d6-ed69-49b4-988a-c6c1b9014e35 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:52.325045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488553d6-ed69-49b4-988a-c6c1b9014e35 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:52.325831 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488553d6-ed69-49b4-988a-c6c1b9014e35 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:52.327308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488553d6-ed69-49b4-988a-c6c1b9014e35 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:52.350830 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110126b-7460-4603-bf90-832facebb95c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:52.379176 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:52.391488 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba307e8-c62d-451f-af06-d228de6d5573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:52.392247 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba307e8-c62d-451f-af06-d228de6d5573 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:52.392612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba307e8-c62d-451f-af06-d228de6d5573 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:52.392892 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba307e8-c62d-451f-af06-d228de6d5573 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:52.394343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba307e8-c62d-451f-af06-d228de6d5573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:52.396005 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba307e8-c62d-451f-af06-d228de6d5573 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:52.397906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 21/44] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:48:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:52.406189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba307e8-c62d-451f-af06-d228de6d5573 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:52.406325 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 24/45] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:48:52 2022] POST /identity/v3/roles => generated 223 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:48:52.533160 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba307e8-c62d-451f-af06-d228de6d5573 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:52.571129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841da00a-8d05-4bb8-88c8-8a3e5f68c540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:52.571822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841da00a-8d05-4bb8-88c8-8a3e5f68c540 None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:52.572115 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841da00a-8d05-4bb8-88c8-8a3e5f68c540 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:52.572494 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841da00a-8d05-4bb8-88c8-8a3e5f68c540 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:52.573229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841da00a-8d05-4bb8-88c8-8a3e5f68c540 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:52.574758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841da00a-8d05-4bb8-88c8-8a3e5f68c540 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:52.663504 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 22/46] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:48:52 2022] POST /identity/v3/projects => generated 312 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:48:54.166392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced8b52-5966-43be-bb88-6875f36830fc None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:54.167016 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced8b52-5966-43be-bb88-6875f36830fc None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:54.167016 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced8b52-5966-43be-bb88-6875f36830fc None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:54.168581 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 25/47] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:48:54.173802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff49bc3-0368-4580-92ca-617b72ff12e3 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:54.174217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff49bc3-0368-4580-92ca-617b72ff12e3 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:54.175014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff49bc3-0368-4580-92ca-617b72ff12e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:54.193029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-7ff49bc3-0368-4580-92ca-617b72ff12e3 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:54.194508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-7ff49bc3-0368-4580-92ca-617b72ff12e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:54.230314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:54.263002 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 23/48] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:48:54.287361 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced8b52-5966-43be-bb88-6875f36830fc None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:54.390202 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced8b52-5966-43be-bb88-6875f36830fc None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:54.424992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da5042-0f89-4b04-9690-9f6b687f6f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:54.425792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da5042-0f89-4b04-9690-9f6b687f6f74 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:54.426081 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da5042-0f89-4b04-9690-9f6b687f6f74 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:54.426361 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da5042-0f89-4b04-9690-9f6b687f6f74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:54.437041 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da5042-0f89-4b04-9690-9f6b687f6f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-54da5042-0f89-4b04-9690-9f6b687f6f74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:48:54.445370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:48:54.446435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:48:54.447380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 26/49] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:48:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:48:54.454228 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff49bc3-0368-4580-92ca-617b72ff12e3 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:54.555052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff49bc3-0368-4580-92ca-617b72ff12e3 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:54.590047 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47187e-31c4-49da-a184-d389c7ad1d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:54.590731 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47187e-31c4-49da-a184-d389c7ad1d0d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:54.591023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47187e-31c4-49da-a184-d389c7ad1d0d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:54.591309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47187e-31c4-49da-a184-d389c7ad1d0d None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:54.592068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47187e-31c4-49da-a184-d389c7ad1d0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:48:54.592412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47187e-31c4-49da-a184-d389c7ad1d0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:54.593852 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47187e-31c4-49da-a184-d389c7ad1d0d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:54.602038 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 24/50] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:48:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:54.607632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da5042-0f89-4b04-9690-9f6b687f6f74 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:54.651075 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:54.709317 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da5042-0f89-4b04-9690-9f6b687f6f74 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:54.746699 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f7f21-c0a6-4ddf-a276-118e0e069f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:54.747605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7f21-c0a6-4ddf-a276-118e0e069f36 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:54.747898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7f21-c0a6-4ddf-a276-118e0e069f36 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:54.748185 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f7f21-c0a6-4ddf-a276-118e0e069f36 None admin] PATH_INFO: `/v3/users/4f6c30de67e643ecb5582d374f07cb7c` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:54.749522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f7f21-c0a6-4ddf-a276-118e0e069f36 None admin] RBAC: Authorizing `identity:get_user(user_id=4f6c30de67e643ecb5582d374f07cb7c)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:54.751043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162f7f21-c0a6-4ddf-a276-118e0e069f36 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:54.753290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 27/51] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:48:54 2022] GET /identity/v3/users/4f6c30de67e643ecb5582d374f07cb7c => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:54.758988 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47187e-31c4-49da-a184-d389c7ad1d0d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:54.793791 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:54.861026 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47187e-31c4-49da-a184-d389c7ad1d0d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:54.897821 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec0073-b2f6-4dca-aabb-ea0d34fd9de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:54.898626 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0073-b2f6-4dca-aabb-ea0d34fd9de1 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:54.898914 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0073-b2f6-4dca-aabb-ea0d34fd9de1 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:54.899195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec0073-b2f6-4dca-aabb-ea0d34fd9de1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:54.900408 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec0073-b2f6-4dca-aabb-ea0d34fd9de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:54.901868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec0073-b2f6-4dca-aabb-ea0d34fd9de1 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:54.903505 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 25/52] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:48:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:54.909755 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f7f21-c0a6-4ddf-a276-118e0e069f36 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:54.912297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382718e-a899-4f81-aadc-fdd142a1544d None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:54.912608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382718e-a899-4f81-aadc-fdd142a1544d None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:54.912901 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382718e-a899-4f81-aadc-fdd142a1544d None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:54.914202 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 26/53] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:48:54.919077 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592155d7-fdb4-40ae-bd4a-6307437838a5 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:54.919373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592155d7-fdb4-40ae-bd4a-6307437838a5 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:54.919659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592155d7-fdb4-40ae-bd4a-6307437838a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:54.938951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-592155d7-fdb4-40ae-bd4a-6307437838a5 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:54.941101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-592155d7-fdb4-40ae-bd4a-6307437838a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:55.013886 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 27/54] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:48:55.019380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f7f21-c0a6-4ddf-a276-118e0e069f36 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:55.028977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:55.037596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592155d7-fdb4-40ae-bd4a-6307437838a5 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:55.057894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8933359-4641-4225-a984-6506006675c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:55.058575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8933359-4641-4225-a984-6506006675c6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:55.058873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8933359-4641-4225-a984-6506006675c6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:55.059165 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8933359-4641-4225-a984-6506006675c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:55.060080 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8933359-4641-4225-a984-6506006675c6 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.domain.id=default, user.id=4f6c30de67e643ecb5582d374f07cb7c) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:48:55.060696 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8933359-4641-4225-a984-6506006675c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:55.062447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8933359-4641-4225-a984-6506006675c6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:55.068931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 28/55] 10.222.0.88 () {56 vars in 1322 bytes} [Thu May 19 20:48:54 2022] GET /identity/v3/role_assignments?user.id=4f6c30de67e643ecb5582d374f07cb7c&scope.domain.id=default&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 530 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:55.069617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:55.147020 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592155d7-fdb4-40ae-bd4a-6307437838a5 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:55.184758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a7995-ec55-4353-9e81-958827077907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:55.185584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a7995-ec55-4353-9e81-958827077907 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:55.185869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a7995-ec55-4353-9e81-958827077907 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:55.186147 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a7995-ec55-4353-9e81-958827077907 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:55.187392 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a7995-ec55-4353-9e81-958827077907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:55.188917 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a7995-ec55-4353-9e81-958827077907 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:55.191151 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 28/56] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:48:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:48:55.196933 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8933359-4641-4225-a984-6506006675c6 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:55.303252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8933359-4641-4225-a984-6506006675c6 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:55.338719 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c2811-7059-4a6e-a2b4-d78d2f0ee9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:55.339389 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2811-7059-4a6e-a2b4-d78d2f0ee9a8 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:55.339683 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2811-7059-4a6e-a2b4-d78d2f0ee9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:55.339969 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c2811-7059-4a6e-a2b4-d78d2f0ee9a8 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:55.340734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c2811-7059-4a6e-a2b4-d78d2f0ee9a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:55.342289 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c2811-7059-4a6e-a2b4-d78d2f0ee9a8 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:55.450951 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 29/57] 10.222.0.88 () {60 vars in 1140 bytes} [Thu May 19 20:48:55 2022] POST /identity/v3/projects => generated 298 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:48:57.719469 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bdca0-c96d-44bf-a079-2d86cdeb7b01 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:57.720047 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bdca0-c96d-44bf-a079-2d86cdeb7b01 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:57.720047 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bdca0-c96d-44bf-a079-2d86cdeb7b01 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:57.722103 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 29/58] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:48:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:48:57.727644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad60a4-5e64-4de8-9193-157d660ef4aa None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:57.728148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad60a4-5e64-4de8-9193-157d660ef4aa None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:57.728267 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad60a4-5e64-4de8-9193-157d660ef4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:57.745973 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-a2ad60a4-5e64-4de8-9193-157d660ef4aa None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:48:57.747201 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-a2ad60a4-5e64-4de8-9193-157d660ef4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:48:57.765865 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:57.823278 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 30/59] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:48:57.848822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bdca0-c96d-44bf-a079-2d86cdeb7b01 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:57.951347 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:48:57.955238 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bdca0-c96d-44bf-a079-2d86cdeb7b01 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:57.990675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3af38b-b443-460b-951a-8e77c51728cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:57.991404 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3af38b-b443-460b-951a-8e77c51728cf None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:57.991692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3af38b-b443-460b-951a-8e77c51728cf None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:57.991976 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3af38b-b443-460b-951a-8e77c51728cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:57.993282 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3af38b-b443-460b-951a-8e77c51728cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:57.994730 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3af38b-b443-460b-951a-8e77c51728cf None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:57.996398 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 30/60] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:48:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:48:58.002000 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad60a4-5e64-4de8-9193-157d660ef4aa None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:48:58.104857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad60a4-5e64-4de8-9193-157d660ef4aa None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:48:58.140205 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c91f3-36d9-46d4-a5fc-73476c8146c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:48:58.140918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c91f3-36d9-46d4-a5fc-73476c8146c9 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:48:58.141210 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c91f3-36d9-46d4-a5fc-73476c8146c9 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:48:58.141493 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c91f3-36d9-46d4-a5fc-73476c8146c9 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:48:58.142231 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c91f3-36d9-46d4-a5fc-73476c8146c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:48:58.144281 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c91f3-36d9-46d4-a5fc-73476c8146c9 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:48:58.249489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 31/61] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:48:58 2022] POST /identity/v3/users => generated 275 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:49:01.614184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687785e-187c-41b4-9176-9bd66ce4c1ad None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:01.614818 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687785e-187c-41b4-9176-9bd66ce4c1ad None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:01.615171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687785e-187c-41b4-9176-9bd66ce4c1ad None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:01.616745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 31/62] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:01.633312 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6aef6e-01bb-4d62-baa4-74bfa2b23eae None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:01.634390 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6aef6e-01bb-4d62-baa4-74bfa2b23eae None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:01.634937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6aef6e-01bb-4d62-baa4-74bfa2b23eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:01.661547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-8a6aef6e-01bb-4d62-baa4-74bfa2b23eae None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:01.663226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-8a6aef6e-01bb-4d62-baa4-74bfa2b23eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:01.666894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:01.758584 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 32/63] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:01.793027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5687785e-187c-41b4-9176-9bd66ce4c1ad None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:01.849889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b5cd7-79ed-4098-a528-5ccbfeb082d3 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:01.850641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b5cd7-79ed-4098-a528-5ccbfeb082d3 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:01.851095 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b5cd7-79ed-4098-a528-5ccbfeb082d3 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:01.853514 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 33/64] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:49:01.867082 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf8aba-9740-4890-ae14-c8a14698552d None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:01.867849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf8aba-9740-4890-ae14-c8a14698552d None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:01.868426 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf8aba-9740-4890-ae14-c8a14698552d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:01.910622 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-16bf8aba-9740-4890-ae14-c8a14698552d None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:01.911886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-16bf8aba-9740-4890-ae14-c8a14698552d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:01.961180 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5687785e-187c-41b4-9176-9bd66ce4c1ad None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:02.003250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55ca33-b29b-4976-8cc2-4759fb8e66c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:02.004078 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55ca33-b29b-4976-8cc2-4759fb8e66c4 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.004398 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55ca33-b29b-4976-8cc2-4759fb8e66c4 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.004695 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55ca33-b29b-4976-8cc2-4759fb8e66c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.017422 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55ca33-b29b-4976-8cc2-4759fb8e66c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-7b55ca33-b29b-4976-8cc2-4759fb8e66c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:02.026252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:02.027461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:02.028874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 32/65] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:49:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:02.035791 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55ca33-b29b-4976-8cc2-4759fb8e66c4 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:02.071837 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:02.113769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 34/66] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:02.115531 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee70b6-fcdd-485c-af83-8b2fbbdb6221 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.116061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee70b6-fcdd-485c-af83-8b2fbbdb6221 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.116606 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee70b6-fcdd-485c-af83-8b2fbbdb6221 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.118243 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 35/67] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:02.123853 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bf9ca-8bf1-489a-b8da-b4624d2b061c None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.124543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bf9ca-8bf1-489a-b8da-b4624d2b061c None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.125061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736bf9ca-8bf1-489a-b8da-b4624d2b061c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.151683 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-736bf9ca-8bf1-489a-b8da-b4624d2b061c None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:02.153317 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-736bf9ca-8bf1-489a-b8da-b4624d2b061c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:02.157214 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55ca33-b29b-4976-8cc2-4759fb8e66c4 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:02.201151 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8eb6d-cf6f-4c86-a90c-ac82c58c858f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:02.202186 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8eb6d-cf6f-4c86-a90c-ac82c58c858f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.202677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8eb6d-cf6f-4c86-a90c-ac82c58c858f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.203191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8eb6d-cf6f-4c86-a90c-ac82c58c858f None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.204176 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8eb6d-cf6f-4c86-a90c-ac82c58c858f None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:02.204763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8eb6d-cf6f-4c86-a90c-ac82c58c858f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:02.206420 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8eb6d-cf6f-4c86-a90c-ac82c58c858f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:02.218112 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 33/68] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:02.219318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8eb6d-cf6f-4c86-a90c-ac82c58c858f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:02.242870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:02.249878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 36/69] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:02.250735 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bf9ca-8bf1-489a-b8da-b4624d2b061c None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:02.337609 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:02.349671 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8eb6d-cf6f-4c86-a90c-ac82c58c858f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:02.394984 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5ab2a-9b56-4c0e-96f1-adb2bdd43cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:02.396184 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736bf9ca-8bf1-489a-b8da-b4624d2b061c None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:02.396352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ab2a-9b56-4c0e-96f1-adb2bdd43cf2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.396858 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ab2a-9b56-4c0e-96f1-adb2bdd43cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.397410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ab2a-9b56-4c0e-96f1-adb2bdd43cf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.414757 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f5ab2a-9b56-4c0e-96f1-adb2bdd43cf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-f5f5ab2a-9b56-4c0e-96f1-adb2bdd43cf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:02.426550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:02.427856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:02.428851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:02.430661 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 34/70] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:02.430887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5ab2a-9b56-4c0e-96f1-adb2bdd43cf2 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:02.448369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98121fa7-fba0-4169-af8d-196a21333a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:02.449657 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98121fa7-fba0-4169-af8d-196a21333a18 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.450196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98121fa7-fba0-4169-af8d-196a21333a18 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.450699 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98121fa7-fba0-4169-af8d-196a21333a18 None admin] PATH_INFO: `/v3/users/4f6c30de67e643ecb5582d374f07cb7c` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.452198 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98121fa7-fba0-4169-af8d-196a21333a18 None admin] RBAC: Authorizing `identity:get_user(user_id=4f6c30de67e643ecb5582d374f07cb7c)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:02.454083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98121fa7-fba0-4169-af8d-196a21333a18 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:02.456657 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 37/71] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:02 2022] GET /identity/v3/users/4f6c30de67e643ecb5582d374f07cb7c => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:02.457720 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98121fa7-fba0-4169-af8d-196a21333a18 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:02.531351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:02.544994 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f5ab2a-9b56-4c0e-96f1-adb2bdd43cf2 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:02.569160 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98121fa7-fba0-4169-af8d-196a21333a18 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:02.582364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f2d4e-4ff7-41eb-aeac-dcc1d496bb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:02.583196 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f2d4e-4ff7-41eb-aeac-dcc1d496bb0d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.583558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f2d4e-4ff7-41eb-aeac-dcc1d496bb0d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.583855 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f2d4e-4ff7-41eb-aeac-dcc1d496bb0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.596816 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f2d4e-4ff7-41eb-aeac-dcc1d496bb0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-2e0f2d4e-4ff7-41eb-aeac-dcc1d496bb0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:02.605518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:02.606670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 35/72] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:02.607628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f2d4e-4ff7-41eb-aeac-dcc1d496bb0d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:02.615114 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de0fb9-5508-456c-b364-c0c9f1616618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:02.615823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de0fb9-5508-456c-b364-c0c9f1616618 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.616120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de0fb9-5508-456c-b364-c0c9f1616618 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.616442 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de0fb9-5508-456c-b364-c0c9f1616618 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.617238 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de0fb9-5508-456c-b364-c0c9f1616618 None admin] RBAC: Adding query filter params (name=member) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:02.617561 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de0fb9-5508-456c-b364-c0c9f1616618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:02.619053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de0fb9-5508-456c-b364-c0c9f1616618 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:02.622961 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:02.628921 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 38/73] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:02.630421 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790b548-9c6b-42f3-9705-7ddc33f3ff8b None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.630734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790b548-9c6b-42f3-9705-7ddc33f3ff8b None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.631024 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790b548-9c6b-42f3-9705-7ddc33f3ff8b None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.632486 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 39/74] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:02.634238 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2af13-0d4b-4205-b7f1-934262eedbd8 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.634843 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2af13-0d4b-4205-b7f1-934262eedbd8 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.634843 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2af13-0d4b-4205-b7f1-934262eedbd8 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.636688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 40/75] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:02.641546 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2af13-0d4b-4205-b7f1-934262eedbd8 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:02.719948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f2d4e-4ff7-41eb-aeac-dcc1d496bb0d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:02.749556 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2af13-0d4b-4205-b7f1-934262eedbd8 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:02.753751 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59212bcc-7c89-4101-92b4-8154d59d1128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:02.754410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59212bcc-7c89-4101-92b4-8154d59d1128 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.754698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59212bcc-7c89-4101-92b4-8154d59d1128 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.754982 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59212bcc-7c89-4101-92b4-8154d59d1128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.778806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:02.784854 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d058a45-dd48-477c-9197-ec4ba64395a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:02.785530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d058a45-dd48-477c-9197-ec4ba64395a5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.785822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d058a45-dd48-477c-9197-ec4ba64395a5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.786102 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d058a45-dd48-477c-9197-ec4ba64395a5 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.786868 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d058a45-dd48-477c-9197-ec4ba64395a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:02.787186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d058a45-dd48-477c-9197-ec4ba64395a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:02.788636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d058a45-dd48-477c-9197-ec4ba64395a5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:02.802729 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 41/76] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:02.802729 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d058a45-dd48-477c-9197-ec4ba64395a5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:02.828391 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59212bcc-7c89-4101-92b4-8154d59d1128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:02.829927 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59212bcc-7c89-4101-92b4-8154d59d1128 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:02.843134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:02.918475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d058a45-dd48-477c-9197-ec4ba64395a5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:02.943190 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 36/77] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:02.944567 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1e573-d241-426d-b391-3a3a8708cbfd None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.944874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1e573-d241-426d-b391-3a3a8708cbfd None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.945164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1e573-d241-426d-b391-3a3a8708cbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.960947 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d32e1b-0062-4961-8210-2a443709359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:02.962248 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d32e1b-0062-4961-8210-2a443709359c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.962617 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d32e1b-0062-4961-8210-2a443709359c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.962905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d32e1b-0062-4961-8210-2a443709359c None admin] PATH_INFO: `/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:02.963450 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-3bc1e573-d241-426d-b391-3a3a8708cbfd None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:02.964849 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-3bc1e573-d241-426d-b391-3a3a8708cbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:02.980426 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d32e1b-0062-4961-8210-2a443709359c None admin] RBAC: Authorizing `identity:get_user(user_id=136b6f5ff4004c75b8b76ea91e0ec8d4)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:02.981816 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d32e1b-0062-4961-8210-2a443709359c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:02.984815 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 42/78] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:02 2022] GET /identity/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:02.986123 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d43f92-7fcd-4338-ba24-bd667d1449a4 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:02.986545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d43f92-7fcd-4338-ba24-bd667d1449a4 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:02.986751 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d43f92-7fcd-4338-ba24-bd667d1449a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:03.005887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-e9d43f92-7fcd-4338-ba24-bd667d1449a4 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:03.007150 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-e9d43f92-7fcd-4338-ba24-bd667d1449a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:03.045241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 37/79] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:03.047344 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1e573-d241-426d-b391-3a3a8708cbfd None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:03.062653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:03.086189 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 43/80] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:03.087426 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d43f92-7fcd-4338-ba24-bd667d1449a4 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:03.094908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:03.160523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1e573-d241-426d-b391-3a3a8708cbfd None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:03.197728 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fcd41-55c4-4c5b-b09d-ee93bd4bec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:03.198635 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fcd41-55c4-4c5b-b09d-ee93bd4bec47 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:03.198921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fcd41-55c4-4c5b-b09d-ee93bd4bec47 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:03.199216 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fcd41-55c4-4c5b-b09d-ee93bd4bec47 None admin] PATH_INFO: `/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:03.199216 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fcd41-55c4-4c5b-b09d-ee93bd4bec47 None admin] RBAC: Authorizing `identity:get_user(user_id=136b6f5ff4004c75b8b76ea91e0ec8d4)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:03.199216 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fcd41-55c4-4c5b-b09d-ee93bd4bec47 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:03.205080 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d43f92-7fcd-4338-ba24-bd667d1449a4 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:03.205205 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 38/81] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:03 2022] GET /identity/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:03.209437 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fcd41-55c4-4c5b-b09d-ee93bd4bec47 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:03.249864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6789d9-b793-481f-8bd8-39493b690b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:03.250658 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6789d9-b793-481f-8bd8-39493b690b69 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:03.250951 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6789d9-b793-481f-8bd8-39493b690b69 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:03.251238 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6789d9-b793-481f-8bd8-39493b690b69 None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:03.253187 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6789d9-b793-481f-8bd8-39493b690b69 None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:03.254738 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6789d9-b793-481f-8bd8-39493b690b69 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:03.256602 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 44/82] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:03 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:03.260812 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6789d9-b793-481f-8bd8-39493b690b69 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:03.318912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:03.328822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fcd41-55c4-4c5b-b09d-ee93bd4bec47 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:03.339384 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:03.379958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d7e1-b6a2-4407-be5d-c90afb0a1e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:03.379958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d7e1-b6a2-4407-be5d-c90afb0a1e11 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:03.380451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d7e1-b6a2-4407-be5d-c90afb0a1e11 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:03.380451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d7e1-b6a2-4407-be5d-c90afb0a1e11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:03.385214 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6789d9-b793-481f-8bd8-39493b690b69 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:03.424156 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d056d72-2fe5-4803-bf54-a0a53136de13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:03.424986 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d056d72-2fe5-4803-bf54-a0a53136de13 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:03.425281 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d056d72-2fe5-4803-bf54-a0a53136de13 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:03.425564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d056d72-2fe5-4803-bf54-a0a53136de13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:03.426788 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d056d72-2fe5-4803-bf54-a0a53136de13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:03.428287 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d056d72-2fe5-4803-bf54-a0a53136de13 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:03.429983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 45/83] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:49:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:03.430776 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d056d72-2fe5-4803-bf54-a0a53136de13 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:03.460449 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418d7e1-b6a2-4407-be5d-c90afb0a1e11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:03.461961 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418d7e1-b6a2-4407-be5d-c90afb0a1e11 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:03.544328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d056d72-2fe5-4803-bf54-a0a53136de13 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:03.567905 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 39/84] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:03.568944 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d7e1-b6a2-4407-be5d-c90afb0a1e11 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:03.574412 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:03.580004 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:03.593969 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b30ef-205d-40e3-b89b-c92439899057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:03.594781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b30ef-205d-40e3-b89b-c92439899057 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:03.595069 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b30ef-205d-40e3-b89b-c92439899057 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:03.595350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b30ef-205d-40e3-b89b-c92439899057 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:03.606453 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b30ef-205d-40e3-b89b-c92439899057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-ed4b30ef-205d-40e3-b89b-c92439899057 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:03.614379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:03.615941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 46/85] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:03.616967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b30ef-205d-40e3-b89b-c92439899057 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:03.693506 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d7e1-b6a2-4407-be5d-c90afb0a1e11 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:03.730657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b0a03-cb07-404d-9245-c5e915708bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:03.731318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b0a03-cb07-404d-9245-c5e915708bd3 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:03.731613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b0a03-cb07-404d-9245-c5e915708bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:03.732017 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b0a03-cb07-404d-9245-c5e915708bd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:03.755616 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b30ef-205d-40e3-b89b-c92439899057 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:03.793293 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a28359-b8f2-4e11-80f1-3cf70a910d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:03.793970 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a28359-b8f2-4e11-80f1-3cf70a910d7f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:03.794303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a28359-b8f2-4e11-80f1-3cf70a910d7f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:03.794636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a28359-b8f2-4e11-80f1-3cf70a910d7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:03.795960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a28359-b8f2-4e11-80f1-3cf70a910d7f None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=834a091f1d39471ba13d3156d015bf3d, user.id=4f6c30de67e643ecb5582d374f07cb7c) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:03.796369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a28359-b8f2-4e11-80f1-3cf70a910d7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:03.798351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a28359-b8f2-4e11-80f1-3cf70a910d7f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:03.805770 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 47/86] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:03 2022] GET /identity/v3/role_assignments?user.id=4f6c30de67e643ecb5582d374f07cb7c&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:03.808376 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a28359-b8f2-4e11-80f1-3cf70a910d7f None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:03.812981 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b0a03-cb07-404d-9245-c5e915708bd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:03.814497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b0a03-cb07-404d-9245-c5e915708bd3 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:03.827947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:03.858670 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:03.944049 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a28359-b8f2-4e11-80f1-3cf70a910d7f None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:03.948914 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 40/87] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:03.949655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b0a03-cb07-404d-9245-c5e915708bd3 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:03.988800 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15a575-75bf-440e-9000-34406dbea9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:03.989495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15a575-75bf-440e-9000-34406dbea9c1 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:03.989786 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15a575-75bf-440e-9000-34406dbea9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:03.990069 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15a575-75bf-440e-9000-34406dbea9c1 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:03.990810 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15a575-75bf-440e-9000-34406dbea9c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:03.992369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15a575-75bf-440e-9000-34406dbea9c1 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:04.073210 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 48/88] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:49:03 2022] POST /identity/v3/users => generated 289 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:49:04.074099 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15a575-75bf-440e-9000-34406dbea9c1 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:04.099697 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b0a03-cb07-404d-9245-c5e915708bd3 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:04.113665 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:04.144740 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6f55d-30d0-4e56-b8e0-d0e748a3e878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:04.146074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6f55d-30d0-4e56-b8e0-d0e748a3e878 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:04.146710 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6f55d-30d0-4e56-b8e0-d0e748a3e878 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:04.147302 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6f55d-30d0-4e56-b8e0-d0e748a3e878 None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:04.149046 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6f55d-30d0-4e56-b8e0-d0e748a3e878 None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:04.151510 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6f55d-30d0-4e56-b8e0-d0e748a3e878 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:04.159869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 41/89] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:03 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:04.161340 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6f55d-30d0-4e56-b8e0-d0e748a3e878 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:04.209515 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:04.217208 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15a575-75bf-440e-9000-34406dbea9c1 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:04.260936 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e997f-2ece-4099-b7a2-2e8567cfeedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:04.261963 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e997f-2ece-4099-b7a2-2e8567cfeedc None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:04.262434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e997f-2ece-4099-b7a2-2e8567cfeedc None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:04.262949 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e997f-2ece-4099-b7a2-2e8567cfeedc None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:04.264043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e997f-2ece-4099-b7a2-2e8567cfeedc None admin] RBAC: Adding query filter params (name=member) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:04.264785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e997f-2ece-4099-b7a2-2e8567cfeedc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:04.266435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e997f-2ece-4099-b7a2-2e8567cfeedc None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:04.276997 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 49/90] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:04.278613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e997f-2ece-4099-b7a2-2e8567cfeedc None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:04.286141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6f55d-30d0-4e56-b8e0-d0e748a3e878 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:04.327341 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a17953-059b-4b54-baf9-ae688428fccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:04.328138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a17953-059b-4b54-baf9-ae688428fccf None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:04.328493 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a17953-059b-4b54-baf9-ae688428fccf None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:04.329229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a17953-059b-4b54-baf9-ae688428fccf None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:04.330613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a17953-059b-4b54-baf9-ae688428fccf None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:04.332092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a17953-059b-4b54-baf9-ae688428fccf None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:04.334023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 42/91] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:04 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:04.334822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a17953-059b-4b54-baf9-ae688428fccf None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:04.392539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:04.401675 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e997f-2ece-4099-b7a2-2e8567cfeedc None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:04.457871 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9f9f5-2f31-4ba9-86c4-f2307a15612b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:04.458843 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f9f5-2f31-4ba9-86c4-f2307a15612b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:04.459306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f9f5-2f31-4ba9-86c4-f2307a15612b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:04.459650 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9f9f5-2f31-4ba9-86c4-f2307a15612b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:04.460543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9f9f5-2f31-4ba9-86c4-f2307a15612b None admin] RBAC: Adding query filter params (role.id=2e3b001c16d24c279be5c68aa3847ca7, scope.project.id=834a091f1d39471ba13d3156d015bf3d, user.id=136b6f5ff4004c75b8b76ea91e0ec8d4) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:04.460860 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9f9f5-2f31-4ba9-86c4-f2307a15612b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:04.462378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9f9f5-2f31-4ba9-86c4-f2307a15612b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:04.473730 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 50/92] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:04 2022] GET /identity/v3/role_assignments?user.id=136b6f5ff4004c75b8b76ea91e0ec8d4&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:04.474602 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9f9f5-2f31-4ba9-86c4-f2307a15612b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:04.493320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a17953-059b-4b54-baf9-ae688428fccf None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:04.510118 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:04.548976 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d8404-a03c-4421-865f-b6fa66cc3c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:04.551012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8404-a03c-4421-865f-b6fa66cc3c02 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:04.551587 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8404-a03c-4421-865f-b6fa66cc3c02 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:04.552090 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d8404-a03c-4421-865f-b6fa66cc3c02 None admin] PATH_INFO: `/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:04.553808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d8404-a03c-4421-865f-b6fa66cc3c02 None admin] RBAC: Authorizing `identity:get_user(user_id=136b6f5ff4004c75b8b76ea91e0ec8d4)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:04.555613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d8404-a03c-4421-865f-b6fa66cc3c02 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:04.560403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 43/93] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:04 2022] GET /identity/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:04.566221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d8404-a03c-4421-865f-b6fa66cc3c02 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:04.632806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9f9f5-2f31-4ba9-86c4-f2307a15612b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:04.685906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617b18d-5964-400e-a3ec-533dfcec4252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:04.687256 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617b18d-5964-400e-a3ec-533dfcec4252 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:04.688387 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617b18d-5964-400e-a3ec-533dfcec4252 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:04.688876 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617b18d-5964-400e-a3ec-533dfcec4252 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:04.690822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617b18d-5964-400e-a3ec-533dfcec4252 None admin] RBAC: Adding query filter params (role.id=4313b1c9df5345778a249b04355b90a0, scope.project.id=834a091f1d39471ba13d3156d015bf3d, user.id=136b6f5ff4004c75b8b76ea91e0ec8d4) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:04.691351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617b18d-5964-400e-a3ec-533dfcec4252 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:04.694076 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617b18d-5964-400e-a3ec-533dfcec4252 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:04.700252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d8404-a03c-4421-865f-b6fa66cc3c02 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:04.703700 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 51/94] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:04 2022] GET /identity/v3/role_assignments?user.id=136b6f5ff4004c75b8b76ea91e0ec8d4&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=4313b1c9df5345778a249b04355b90a0 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:04.717404 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:04.745409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad408fc1-b8dc-47ea-a081-f7e1fe2c71dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:04.746628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad408fc1-b8dc-47ea-a081-f7e1fe2c71dd None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:04.747192 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad408fc1-b8dc-47ea-a081-f7e1fe2c71dd None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:04.747710 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad408fc1-b8dc-47ea-a081-f7e1fe2c71dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:04.861041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad408fc1-b8dc-47ea-a081-f7e1fe2c71dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:04.863493 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad408fc1-b8dc-47ea-a081-f7e1fe2c71dd None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:05.012444 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 44/95] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:49:05.018160 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617b18d-5964-400e-a3ec-533dfcec4252 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:05.151954 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:05.174884 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617b18d-5964-400e-a3ec-533dfcec4252 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:05.224752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6ad32c-3094-4b0c-a5ae-816b00be68a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:05.225992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ad32c-3094-4b0c-a5ae-816b00be68a9 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:05.226498 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ad32c-3094-4b0c-a5ae-816b00be68a9 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:05.227087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ad32c-3094-4b0c-a5ae-816b00be68a9 None admin] PATH_INFO: `/v3/projects/7ece7251349140ca9d947cf7aec97a90` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:05.228564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ad32c-3094-4b0c-a5ae-816b00be68a9 None admin] RBAC: Authorizing `identity:get_project(project_id=7ece7251349140ca9d947cf7aec97a90)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:05.230544 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6ad32c-3094-4b0c-a5ae-816b00be68a9 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:05.232575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 52/96] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:05 2022] GET /identity/v3/projects/7ece7251349140ca9d947cf7aec97a90 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:05.239428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad408fc1-b8dc-47ea-a081-f7e1fe2c71dd None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:05.255047 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:05.379593 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad408fc1-b8dc-47ea-a081-f7e1fe2c71dd None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:05.424246 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01594519-1602-478b-addc-d4d67dfd34db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:05.425347 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594519-1602-478b-addc-d4d67dfd34db None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:05.425844 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594519-1602-478b-addc-d4d67dfd34db None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:05.426526 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594519-1602-478b-addc-d4d67dfd34db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:05.427593 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01594519-1602-478b-addc-d4d67dfd34db None admin] RBAC: Adding query filter params (role.id=2e3b001c16d24c279be5c68aa3847ca7, scope.project.id=7ece7251349140ca9d947cf7aec97a90, user.id=136b6f5ff4004c75b8b76ea91e0ec8d4) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:05.428151 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01594519-1602-478b-addc-d4d67dfd34db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:05.429839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01594519-1602-478b-addc-d4d67dfd34db None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:05.443374 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 45/97] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:05 2022] GET /identity/v3/role_assignments?user.id=136b6f5ff4004c75b8b76ea91e0ec8d4&scope.project.id=7ece7251349140ca9d947cf7aec97a90&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:06.827719 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ff647-b237-4a3e-b186-9df7603f6610 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:06.828824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ff647-b237-4a3e-b186-9df7603f6610 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:06.829366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ff647-b237-4a3e-b186-9df7603f6610 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:06.831185 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 53/98] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:06.836873 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50649d61-6201-46fd-9d83-66f4442eb806 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:06.837737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50649d61-6201-46fd-9d83-66f4442eb806 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:06.838406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50649d61-6201-46fd-9d83-66f4442eb806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:06.860812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-50649d61-6201-46fd-9d83-66f4442eb806 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:06.862917 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-50649d61-6201-46fd-9d83-66f4442eb806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:06.966287 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 46/99] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:07.000770 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ff647-b237-4a3e-b186-9df7603f6610 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:07.145868 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ff647-b237-4a3e-b186-9df7603f6610 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:07.198560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952e4c3-09fb-48cf-85de-d24d437ccb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:07.199708 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952e4c3-09fb-48cf-85de-d24d437ccb27 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:07.200170 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952e4c3-09fb-48cf-85de-d24d437ccb27 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:07.200729 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952e4c3-09fb-48cf-85de-d24d437ccb27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:07.222104 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952e4c3-09fb-48cf-85de-d24d437ccb27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-c952e4c3-09fb-48cf-85de-d24d437ccb27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:07.230252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:07.231424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:07.232392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 54/100] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:49:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:07.241538 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50649d61-6201-46fd-9d83-66f4442eb806 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:07.255291 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:07.389383 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50649d61-6201-46fd-9d83-66f4442eb806 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:07.463120 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773a5c0-f0eb-4d30-8b2d-b63a8ba18cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:07.464212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773a5c0-f0eb-4d30-8b2d-b63a8ba18cd4 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:07.464774 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773a5c0-f0eb-4d30-8b2d-b63a8ba18cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:07.465295 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773a5c0-f0eb-4d30-8b2d-b63a8ba18cd4 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:07.466304 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773a5c0-f0eb-4d30-8b2d-b63a8ba18cd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:07.466846 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773a5c0-f0eb-4d30-8b2d-b63a8ba18cd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:07.468540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773a5c0-f0eb-4d30-8b2d-b63a8ba18cd4 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:07.484985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 47/101] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:07.491819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952e4c3-09fb-48cf-85de-d24d437ccb27 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:07.531021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:07.638274 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952e4c3-09fb-48cf-85de-d24d437ccb27 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:07.696411 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c329180-3c05-4ccb-9d5e-ace456db8333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:07.697356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c329180-3c05-4ccb-9d5e-ace456db8333 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:07.697651 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c329180-3c05-4ccb-9d5e-ace456db8333 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:07.697939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c329180-3c05-4ccb-9d5e-ace456db8333 None admin] PATH_INFO: `/v3/users/4f6c30de67e643ecb5582d374f07cb7c` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:07.705979 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c329180-3c05-4ccb-9d5e-ace456db8333 None admin] RBAC: Authorizing `identity:get_user(user_id=4f6c30de67e643ecb5582d374f07cb7c)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:07.707569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c329180-3c05-4ccb-9d5e-ace456db8333 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:07.709460 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 55/102] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:07 2022] GET /identity/v3/users/4f6c30de67e643ecb5582d374f07cb7c => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:07.715553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773a5c0-f0eb-4d30-8b2d-b63a8ba18cd4 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:07.880701 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa56a6-66d2-4521-983b-84fbef3b1a7c None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:07.881293 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa56a6-66d2-4521-983b-84fbef3b1a7c None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:07.881293 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fa56a6-66d2-4521-983b-84fbef3b1a7c None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:07.881894 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:07.884539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 56/103] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:07.890704 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592c1e6-48d6-4e0d-b7e3-b688f785b3af None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:07.890704 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592c1e6-48d6-4e0d-b7e3-b688f785b3af None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:07.890704 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592c1e6-48d6-4e0d-b7e3-b688f785b3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:07.917884 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773a5c0-f0eb-4d30-8b2d-b63a8ba18cd4 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:07.949485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-2592c1e6-48d6-4e0d-b7e3-b688f785b3af None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:07.950751 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-2592c1e6-48d6-4e0d-b7e3-b688f785b3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:07.972775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc7fa8-40d3-43f5-92d1-b1470d628c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:07.973962 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc7fa8-40d3-43f5-92d1-b1470d628c8e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:07.974421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc7fa8-40d3-43f5-92d1-b1470d628c8e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:07.974877 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc7fa8-40d3-43f5-92d1-b1470d628c8e None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:07.976357 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc7fa8-40d3-43f5-92d1-b1470d628c8e None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:07.978000 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc7fa8-40d3-43f5-92d1-b1470d628c8e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:07.980291 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 48/104] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:07 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:07.987204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc7fa8-40d3-43f5-92d1-b1470d628c8e None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:08.082822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 57/105] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:08.084398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f1973-82fd-4b31-b946-9f34e9eb9d69 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:08.084705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f1973-82fd-4b31-b946-9f34e9eb9d69 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:08.084994 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f1973-82fd-4b31-b946-9f34e9eb9d69 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:08.086484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 58/106] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:08.097949 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680e42-4698-4182-a27a-b3ac0fdc635c None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:08.098223 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680e42-4698-4182-a27a-b3ac0fdc635c None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:08.098712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b680e42-4698-4182-a27a-b3ac0fdc635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:08.122092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc7fa8-40d3-43f5-92d1-b1470d628c8e None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:08.133724 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-6b680e42-4698-4182-a27a-b3ac0fdc635c None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:08.134794 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-6b680e42-4698-4182-a27a-b3ac0fdc635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:08.157589 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:08.170909 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e55bd-0793-46f2-833e-5fb96df2f57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:08.172099 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e55bd-0793-46f2-833e-5fb96df2f57c None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:08.173061 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e55bd-0793-46f2-833e-5fb96df2f57c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:08.173061 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e55bd-0793-46f2-833e-5fb96df2f57c None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d/users/4f6c30de67e643ecb5582d374f07cb7c/roles/ed68c2195de147c0a30d9150b33e47cf` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:08.177772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e55bd-0793-46f2-833e-5fb96df2f57c None admin] RBAC: Authorizing `identity:create_grant(project_id=834a091f1d39471ba13d3156d015bf3d, user_id=4f6c30de67e643ecb5582d374f07cb7c, role_id=ed68c2195de147c0a30d9150b33e47cf)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:08.179235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e55bd-0793-46f2-833e-5fb96df2f57c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:08.184090 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:08.226541 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 59/107] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:08.227433 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b680e42-4698-4182-a27a-b3ac0fdc635c None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:08.264160 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 49/108] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:49:07 2022] PUT /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d/users/4f6c30de67e643ecb5582d374f07cb7c/roles/ed68c2195de147c0a30d9150b33e47cf => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:49:08.265539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5263eb-eb97-452c-8681-0b087da1507b None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:08.265937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5263eb-eb97-452c-8681-0b087da1507b None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:08.266228 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5263eb-eb97-452c-8681-0b087da1507b None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:08.267973 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 50/109] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:08.274147 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5263eb-eb97-452c-8681-0b087da1507b None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:08.395774 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b680e42-4698-4182-a27a-b3ac0fdc635c None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:08.437244 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb1ad7-c0bd-4c28-87d4-b1cf0b98320d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:08.438181 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb1ad7-c0bd-4c28-87d4-b1cf0b98320d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:08.438477 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb1ad7-c0bd-4c28-87d4-b1cf0b98320d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:08.438767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb1ad7-c0bd-4c28-87d4-b1cf0b98320d None admin] PATH_INFO: `/v3/domains/default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:08.440257 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb1ad7-c0bd-4c28-87d4-b1cf0b98320d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:08.441845 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb1ad7-c0bd-4c28-87d4-b1cf0b98320d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:08.449935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 60/110] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:49:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:08.449935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52952b35-d823-481d-8768-7cb20c9eaf23 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:08.449935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52952b35-d823-481d-8768-7cb20c9eaf23 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:08.449935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52952b35-d823-481d-8768-7cb20c9eaf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:08.466046 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-52952b35-d823-481d-8768-7cb20c9eaf23 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:08.467285 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-52952b35-d823-481d-8768-7cb20c9eaf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:08.476817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:08.506145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5263eb-eb97-452c-8681-0b087da1507b None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:08.558894 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6af70e-35da-4757-811f-865093232adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:08.560112 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6af70e-35da-4757-811f-865093232adb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:08.560648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6af70e-35da-4757-811f-865093232adb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:08.561135 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6af70e-35da-4757-811f-865093232adb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:08.561352 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 61/111] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:08.562937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd8268-bdcc-41fa-bfef-747a7523ee58 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:08.563231 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd8268-bdcc-41fa-bfef-747a7523ee58 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:08.563517 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd8268-bdcc-41fa-bfef-747a7523ee58 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:08.565920 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 62/112] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:08.566658 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd8268-bdcc-41fa-bfef-747a7523ee58 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:08.584368 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6af70e-35da-4757-811f-865093232adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-3a6af70e-35da-4757-811f-865093232adb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:08.593950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:08.595076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:08.596628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 51/113] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:08.597434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffc69f-0c44-4dab-a31f-0647bf1b6d6f None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:08.597434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffc69f-0c44-4dab-a31f-0647bf1b6d6f None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:08.597979 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffc69f-0c44-4dab-a31f-0647bf1b6d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:08.619692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-33ffc69f-0c44-4dab-a31f-0647bf1b6d6f None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:08.621390 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-33ffc69f-0c44-4dab-a31f-0647bf1b6d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:08.688382 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:08.709619 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 52/114] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:08.709856 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd8268-bdcc-41fa-bfef-747a7523ee58 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:08.711152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffc69f-0c44-4dab-a31f-0647bf1b6d6f None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:08.761986 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ddd81-ef6b-4bfe-a23a-3a7f511cac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:08.762817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ddd81-ef6b-4bfe-a23a-3a7f511cac80 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:08.762958 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ddd81-ef6b-4bfe-a23a-3a7f511cac80 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:08.763144 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ddd81-ef6b-4bfe-a23a-3a7f511cac80 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:08.763855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ddd81-ef6b-4bfe-a23a-3a7f511cac80 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:08.765399 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ddd81-ef6b-4bfe-a23a-3a7f511cac80 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:08.775956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:08.821131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffc69f-0c44-4dab-a31f-0647bf1b6d6f None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:08.837299 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 63/115] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:49:08 2022] POST /identity/v3/projects => generated 302 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:49:08.844280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ddd81-ef6b-4bfe-a23a-3a7f511cac80 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:08.863874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321dbe7-324f-4295-af05-a9eb33bbbbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:08.864847 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321dbe7-324f-4295-af05-a9eb33bbbbcf None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:08.865192 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321dbe7-324f-4295-af05-a9eb33bbbbcf None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:08.865482 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321dbe7-324f-4295-af05-a9eb33bbbbcf None admin] PATH_INFO: `/v3/roles/member` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:08.878481 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321dbe7-324f-4295-af05-a9eb33bbbbcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-4321dbe7-324f-4295-af05-a9eb33bbbbcf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:08.887097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:08.888934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:08.890281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 53/116] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:08.894675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321dbe7-324f-4295-af05-a9eb33bbbbcf None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:08.958513 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:08.992519 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ddd81-ef6b-4bfe-a23a-3a7f511cac80 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:09.023969 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321dbe7-324f-4295-af05-a9eb33bbbbcf None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:09.032745 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29415d54-cc50-434a-b7ab-f41cac00048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:09.033487 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29415d54-cc50-434a-b7ab-f41cac00048d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.033826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29415d54-cc50-434a-b7ab-f41cac00048d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.034039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29415d54-cc50-434a-b7ab-f41cac00048d None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.034811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29415d54-cc50-434a-b7ab-f41cac00048d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:09.035128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29415d54-cc50-434a-b7ab-f41cac00048d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:09.036588 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29415d54-cc50-434a-b7ab-f41cac00048d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:09.047223 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 64/117] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:09.053681 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97b152-bbcf-44d6-b659-316882bdb8c4 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.054012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97b152-bbcf-44d6-b659-316882bdb8c4 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.054303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97b152-bbcf-44d6-b659-316882bdb8c4 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.056081 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 65/118] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:09.057382 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97b152-bbcf-44d6-b659-316882bdb8c4 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:09.072035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a573a-be0c-4e11-8299-fd7cfa767994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:09.072919 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a573a-be0c-4e11-8299-fd7cfa767994 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.073211 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a573a-be0c-4e11-8299-fd7cfa767994 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.073502 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a573a-be0c-4e11-8299-fd7cfa767994 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.091154 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a573a-be0c-4e11-8299-fd7cfa767994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-381a573a-be0c-4e11-8299-fd7cfa767994 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:09.099668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:09.100876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 54/119] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:09.103397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a573a-be0c-4e11-8299-fd7cfa767994 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:09.138744 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:09.193716 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97b152-bbcf-44d6-b659-316882bdb8c4 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:09.233140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d1f97-2831-4280-9005-836f6b0e912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:09.234397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1f97-2831-4280-9005-836f6b0e912c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.234894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1f97-2831-4280-9005-836f6b0e912c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.235377 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1f97-2831-4280-9005-836f6b0e912c None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.236368 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d1f97-2831-4280-9005-836f6b0e912c None admin] RBAC: Adding query filter params (name=member) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:09.237403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d1f97-2831-4280-9005-836f6b0e912c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:09.238132 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a573a-be0c-4e11-8299-fd7cfa767994 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:09.239231 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d1f97-2831-4280-9005-836f6b0e912c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:09.245872 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:09.249405 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 66/120] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:09.256138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f1de4-309f-4dbf-b9e1-671d8fb5bce8 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.256710 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f1de4-309f-4dbf-b9e1-671d8fb5bce8 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.257361 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f1de4-309f-4dbf-b9e1-671d8fb5bce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.275276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-d68f1de4-309f-4dbf-b9e1-671d8fb5bce8 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:09.275956 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f39314-acfd-404a-afd3-6970062665d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:09.277041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f39314-acfd-404a-afd3-6970062665d1 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.277297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-d68f1de4-309f-4dbf-b9e1-671d8fb5bce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:09.277588 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f39314-acfd-404a-afd3-6970062665d1 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.277983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f39314-acfd-404a-afd3-6970062665d1 None admin] PATH_INFO: `/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.279339 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f39314-acfd-404a-afd3-6970062665d1 None admin] RBAC: Authorizing `identity:get_user(user_id=136b6f5ff4004c75b8b76ea91e0ec8d4)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:09.281761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f39314-acfd-404a-afd3-6970062665d1 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:09.287294 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 55/121] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:09 2022] GET /identity/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:09.287294 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f39314-acfd-404a-afd3-6970062665d1 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:09.356355 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 67/122] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:09.357860 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f1de4-309f-4dbf-b9e1-671d8fb5bce8 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:09.393312 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f39314-acfd-404a-afd3-6970062665d1 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:09.412786 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:09.430592 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40befb-cd51-4ec0-8bbb-f648b12f2ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:09.431371 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40befb-cd51-4ec0-8bbb-f648b12f2ae2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.431726 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40befb-cd51-4ec0-8bbb-f648b12f2ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.432329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40befb-cd51-4ec0-8bbb-f648b12f2ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.433190 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40befb-cd51-4ec0-8bbb-f648b12f2ae2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:09.433596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40befb-cd51-4ec0-8bbb-f648b12f2ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:09.435054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c40befb-cd51-4ec0-8bbb-f648b12f2ae2 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:09.444447 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 56/123] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:09.449204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40befb-cd51-4ec0-8bbb-f648b12f2ae2 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:09.472653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f1de4-309f-4dbf-b9e1-671d8fb5bce8 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:09.485879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:09.515858 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34e35f-64fa-4109-a933-84e9591cae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:09.517010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34e35f-64fa-4109-a933-84e9591cae7d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.517314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34e35f-64fa-4109-a933-84e9591cae7d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.517960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34e35f-64fa-4109-a933-84e9591cae7d None admin] PATH_INFO: `/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.519771 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34e35f-64fa-4109-a933-84e9591cae7d None admin] RBAC: Authorizing `identity:get_user(user_id=136b6f5ff4004c75b8b76ea91e0ec8d4)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:09.521307 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34e35f-64fa-4109-a933-84e9591cae7d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:09.523249 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 68/124] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:09 2022] GET /identity/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:09.524044 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34e35f-64fa-4109-a933-84e9591cae7d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:09.560903 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c40befb-cd51-4ec0-8bbb-f648b12f2ae2 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:09.598894 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d86b0-3c38-4c9a-a6b7-05858353abbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:09.599594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d86b0-3c38-4c9a-a6b7-05858353abbc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.599927 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d86b0-3c38-4c9a-a6b7-05858353abbc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.600222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d86b0-3c38-4c9a-a6b7-05858353abbc None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.601484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d86b0-3c38-4c9a-a6b7-05858353abbc None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:09.603082 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d86b0-3c38-4c9a-a6b7-05858353abbc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:09.609275 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 57/125] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:09 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:09.609275 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d86b0-3c38-4c9a-a6b7-05858353abbc None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:09.656147 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34e35f-64fa-4109-a933-84e9591cae7d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:09.672614 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:09.702155 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9563a-8e7b-48e7-b003-8241f506f22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:09.703770 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9563a-8e7b-48e7-b003-8241f506f22f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.704288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9563a-8e7b-48e7-b003-8241f506f22f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.705080 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9563a-8e7b-48e7-b003-8241f506f22f None admin] PATH_INFO: `/v3/roles/member` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.718896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9563a-8e7b-48e7-b003-8241f506f22f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:09.728111 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d86b0-3c38-4c9a-a6b7-05858353abbc None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-3fa9563a-8e7b-48e7-b003-8241f506f22f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:09.729910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:09.730934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:09.731905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:09.732646 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:09.736408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 69/126] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:09.738134 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9563a-8e7b-48e7-b003-8241f506f22f None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:09.768080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3657f7-2356-45d7-a74f-0a415928624c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:09.769107 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3657f7-2356-45d7-a74f-0a415928624c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.769408 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3657f7-2356-45d7-a74f-0a415928624c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.769693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3657f7-2356-45d7-a74f-0a415928624c None admin] PATH_INFO: `/v3/users/c38801f1b5fb429592a3393dd72f3c4f` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.787148 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3657f7-2356-45d7-a74f-0a415928624c None admin] RBAC: Authorizing `identity:get_user(user_id=c38801f1b5fb429592a3393dd72f3c4f)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:09.788785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3657f7-2356-45d7-a74f-0a415928624c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:09.790827 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 58/127] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:09 2022] GET /identity/v3/users/c38801f1b5fb429592a3393dd72f3c4f => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:09.791449 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3657f7-2356-45d7-a74f-0a415928624c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:09.861259 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9563a-8e7b-48e7-b003-8241f506f22f None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:09.904172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb5445-13db-42e3-b07c-ca2fa92c9e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:09.905148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb5445-13db-42e3-b07c-ca2fa92c9e43 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.905448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb5445-13db-42e3-b07c-ca2fa92c9e43 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.905735 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb5445-13db-42e3-b07c-ca2fa92c9e43 None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.907058 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb5445-13db-42e3-b07c-ca2fa92c9e43 None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:09.909257 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb5445-13db-42e3-b07c-ca2fa92c9e43 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:09.915510 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 70/128] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:09 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:09.915510 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb5445-13db-42e3-b07c-ca2fa92c9e43 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:09.935908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3657f7-2356-45d7-a74f-0a415928624c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:09.967098 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:09.984444 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cece46-db9a-4d34-a181-120a721af17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:09.986273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cece46-db9a-4d34-a181-120a721af17f None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:09.986779 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cece46-db9a-4d34-a181-120a721af17f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:09.987478 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cece46-db9a-4d34-a181-120a721af17f None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d/users/136b6f5ff4004c75b8b76ea91e0ec8d4/roles/4313b1c9df5345778a249b04355b90a0` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:09.990662 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cece46-db9a-4d34-a181-120a721af17f None admin] RBAC: Authorizing `identity:create_grant(project_id=834a091f1d39471ba13d3156d015bf3d, user_id=136b6f5ff4004c75b8b76ea91e0ec8d4, role_id=4313b1c9df5345778a249b04355b90a0)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:09.993090 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cece46-db9a-4d34-a181-120a721af17f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:10.029774 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb5445-13db-42e3-b07c-ca2fa92c9e43 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:10.069400 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3393b39-1d2f-4439-8a3f-cfff6a69d101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:10.070079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3393b39-1d2f-4439-8a3f-cfff6a69d101 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:10.070373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3393b39-1d2f-4439-8a3f-cfff6a69d101 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:10.070658 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3393b39-1d2f-4439-8a3f-cfff6a69d101 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:10.071516 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3393b39-1d2f-4439-8a3f-cfff6a69d101 None admin] RBAC: Adding query filter params (name=member) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:10.071833 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3393b39-1d2f-4439-8a3f-cfff6a69d101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:10.073280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3393b39-1d2f-4439-8a3f-cfff6a69d101 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:10.082600 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 71/129] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:10.083200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3393b39-1d2f-4439-8a3f-cfff6a69d101 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:10.103179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 59/130] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:49:09 2022] PUT /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d/users/136b6f5ff4004c75b8b76ea91e0ec8d4/roles/4313b1c9df5345778a249b04355b90a0 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:10.104089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cece46-db9a-4d34-a181-120a721af17f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:10.138377 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:10.248322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cece46-db9a-4d34-a181-120a721af17f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:10.296797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf976068-3a26-4d8c-86b2-d6f37c4abfdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:10.299792 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf976068-3a26-4d8c-86b2-d6f37c4abfdb None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:10.301920 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf976068-3a26-4d8c-86b2-d6f37c4abfdb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:10.302386 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf976068-3a26-4d8c-86b2-d6f37c4abfdb None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d/users/136b6f5ff4004c75b8b76ea91e0ec8d4/roles/2e3b001c16d24c279be5c68aa3847ca7` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:10.309314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf976068-3a26-4d8c-86b2-d6f37c4abfdb None admin] RBAC: Authorizing `identity:create_grant(project_id=834a091f1d39471ba13d3156d015bf3d, user_id=136b6f5ff4004c75b8b76ea91e0ec8d4, role_id=2e3b001c16d24c279be5c68aa3847ca7)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:10.311324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf976068-3a26-4d8c-86b2-d6f37c4abfdb None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:10.318588 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3393b39-1d2f-4439-8a3f-cfff6a69d101 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:10.325043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:10.365737 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477e1ca-1546-4b81-97f7-238efd6a460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:10.366872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477e1ca-1546-4b81-97f7-238efd6a460c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:10.367332 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477e1ca-1546-4b81-97f7-238efd6a460c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:10.367800 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477e1ca-1546-4b81-97f7-238efd6a460c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:10.394879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 60/131] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:49:10 2022] PUT /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d/users/136b6f5ff4004c75b8b76ea91e0ec8d4/roles/2e3b001c16d24c279be5c68aa3847ca7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:10.394879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf976068-3a26-4d8c-86b2-d6f37c4abfdb None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:10.552353 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477e1ca-1546-4b81-97f7-238efd6a460c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:10.555267 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477e1ca-1546-4b81-97f7-238efd6a460c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:10.612405 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:10.662738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf976068-3a26-4d8c-86b2-d6f37c4abfdb None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:10.721893 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 72/132] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:10.729517 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477e1ca-1546-4b81-97f7-238efd6a460c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:10.735198 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:10.744742 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a39184-2b44-4ecb-b8e1-0f0fc8bfe460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:10.745675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a39184-2b44-4ecb-b8e1-0f0fc8bfe460 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:10.745966 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a39184-2b44-4ecb-b8e1-0f0fc8bfe460 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:10.746257 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a39184-2b44-4ecb-b8e1-0f0fc8bfe460 None admin] PATH_INFO: `/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:10.747841 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a39184-2b44-4ecb-b8e1-0f0fc8bfe460 None admin] RBAC: Authorizing `identity:get_user(user_id=136b6f5ff4004c75b8b76ea91e0ec8d4)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:10.752770 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a39184-2b44-4ecb-b8e1-0f0fc8bfe460 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:10.752770 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 61/133] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:10 2022] GET /identity/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4 => generated 275 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:10.781870 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a39184-2b44-4ecb-b8e1-0f0fc8bfe460 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:11.011160 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477e1ca-1546-4b81-97f7-238efd6a460c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:11.054942 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a39184-2b44-4ecb-b8e1-0f0fc8bfe460 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:11.079127 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f724f71-6255-449b-87a2-941921f4400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:11.080932 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f724f71-6255-449b-87a2-941921f4400c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:11.081405 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f724f71-6255-449b-87a2-941921f4400c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:11.081841 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f724f71-6255-449b-87a2-941921f4400c None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:11.083567 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f724f71-6255-449b-87a2-941921f4400c None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:11.086087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f724f71-6255-449b-87a2-941921f4400c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:11.088352 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 73/134] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:10 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:11.090297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a607b-53c5-4512-9efc-13902157464f None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:11.090861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a607b-53c5-4512-9efc-13902157464f None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:11.091350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a607b-53c5-4512-9efc-13902157464f None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:11.093773 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 74/135] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:11.097056 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a607b-53c5-4512-9efc-13902157464f None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:11.143849 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774af90b-35bb-4bb0-a2b0-cfed24aca5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:11.144839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774af90b-35bb-4bb0-a2b0-cfed24aca5cc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:11.145175 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774af90b-35bb-4bb0-a2b0-cfed24aca5cc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:11.145455 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774af90b-35bb-4bb0-a2b0-cfed24aca5cc None admin] PATH_INFO: `/v3/projects/7ece7251349140ca9d947cf7aec97a90` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:11.146936 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774af90b-35bb-4bb0-a2b0-cfed24aca5cc None admin] RBAC: Authorizing `identity:get_project(project_id=7ece7251349140ca9d947cf7aec97a90)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:11.155526 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774af90b-35bb-4bb0-a2b0-cfed24aca5cc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:11.157193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 62/136] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:10 2022] GET /identity/v3/projects/7ece7251349140ca9d947cf7aec97a90 => generated 312 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:11.159104 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33841c5c-7c1e-4a4b-b5e4-3fef31cdee4d None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:11.159401 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33841c5c-7c1e-4a4b-b5e4-3fef31cdee4d None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:11.159689 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33841c5c-7c1e-4a4b-b5e4-3fef31cdee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:11.198925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-33841c5c-7c1e-4a4b-b5e4-3fef31cdee4d None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:11.200073 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-33841c5c-7c1e-4a4b-b5e4-3fef31cdee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:11.220722 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:11.239483 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:11.253411 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a607b-53c5-4512-9efc-13902157464f None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:11.292944 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db637c0-6243-48d3-bb05-b51a3fe05025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:11.294183 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db637c0-6243-48d3-bb05-b51a3fe05025 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:11.294678 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db637c0-6243-48d3-bb05-b51a3fe05025 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:11.295135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db637c0-6243-48d3-bb05-b51a3fe05025 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:11.296214 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db637c0-6243-48d3-bb05-b51a3fe05025 None admin] RBAC: Adding query filter params (role.id=83321c42fce7438aa5e29c1acb4f07af, scope.project.id=834a091f1d39471ba13d3156d015bf3d, user.id=c38801f1b5fb429592a3393dd72f3c4f) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:11.296731 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db637c0-6243-48d3-bb05-b51a3fe05025 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:11.298471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db637c0-6243-48d3-bb05-b51a3fe05025 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:11.306331 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 75/137] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:11 2022] GET /identity/v3/role_assignments?user.id=c38801f1b5fb429592a3393dd72f3c4f&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=83321c42fce7438aa5e29c1acb4f07af => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:11.307962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db637c0-6243-48d3-bb05-b51a3fe05025 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:11.350908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 63/138] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:11.433481 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33841c5c-7c1e-4a4b-b5e4-3fef31cdee4d None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:11.434296 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db637c0-6243-48d3-bb05-b51a3fe05025 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:11.494855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a9559-4971-4de0-b622-7f4aa3928956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:11.495694 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a9559-4971-4de0-b622-7f4aa3928956 None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:11.495979 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a9559-4971-4de0-b622-7f4aa3928956 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:11.496257 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a9559-4971-4de0-b622-7f4aa3928956 None admin] PATH_INFO: `/v3/projects/7ece7251349140ca9d947cf7aec97a90/users/136b6f5ff4004c75b8b76ea91e0ec8d4/roles/2e3b001c16d24c279be5c68aa3847ca7` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:11.504565 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a9559-4971-4de0-b622-7f4aa3928956 None admin] RBAC: Authorizing `identity:create_grant(project_id=7ece7251349140ca9d947cf7aec97a90, user_id=136b6f5ff4004c75b8b76ea91e0ec8d4, role_id=2e3b001c16d24c279be5c68aa3847ca7)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:11.506183 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a9559-4971-4de0-b622-7f4aa3928956 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:11.516816 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:11.591006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33841c5c-7c1e-4a4b-b5e4-3fef31cdee4d None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:11.655255 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 76/139] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:49:11 2022] PUT /identity/v3/projects/7ece7251349140ca9d947cf7aec97a90/users/136b6f5ff4004c75b8b76ea91e0ec8d4/roles/2e3b001c16d24c279be5c68aa3847ca7 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:11.657726 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7a885-ec36-4c2f-bf9e-6890d8cf9800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:11.658428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673c4e9-3ad1-496a-863f-2c34db14722a None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:11.659275 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673c4e9-3ad1-496a-863f-2c34db14722a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:11.659605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673c4e9-3ad1-496a-863f-2c34db14722a None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:11.660292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7a885-ec36-4c2f-bf9e-6890d8cf9800 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:11.660650 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7a885-ec36-4c2f-bf9e-6890d8cf9800 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:11.660930 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7a885-ec36-4c2f-bf9e-6890d8cf9800 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:11.668044 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 77/140] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:11.672198 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d42cc8-c889-400e-bbbb-dbe1aeff07db None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:11.672562 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d42cc8-c889-400e-bbbb-dbe1aeff07db None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:11.672856 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d42cc8-c889-400e-bbbb-dbe1aeff07db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:11.700002 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7a885-ec36-4c2f-bf9e-6890d8cf9800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-6fc7a885-ec36-4c2f-bf9e-6890d8cf9800 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:11.709419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:11.710557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:11.711529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 64/141] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:49:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:11.714025 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-c0d42cc8-c889-400e-bbbb-dbe1aeff07db None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:11.715835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-c0d42cc8-c889-400e-bbbb-dbe1aeff07db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:11.717155 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7a885-ec36-4c2f-bf9e-6890d8cf9800 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:11.737863 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:11.855299 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7a885-ec36-4c2f-bf9e-6890d8cf9800 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:11.900568 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 78/142] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:11.928521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d42cc8-c889-400e-bbbb-dbe1aeff07db None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:11.942246 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6307a03-4f5b-4708-8652-4a018d23d44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:11.943384 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6307a03-4f5b-4708-8652-4a018d23d44c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:11.944072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6307a03-4f5b-4708-8652-4a018d23d44c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:11.944617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6307a03-4f5b-4708-8652-4a018d23d44c None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:11.945663 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6307a03-4f5b-4708-8652-4a018d23d44c None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:11.946217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6307a03-4f5b-4708-8652-4a018d23d44c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:11.947852 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6307a03-4f5b-4708-8652-4a018d23d44c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:11.964719 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 65/143] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:11.982772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6307a03-4f5b-4708-8652-4a018d23d44c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:12.059949 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d42cc8-c889-400e-bbbb-dbe1aeff07db None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:12.093775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:12.101015 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:12.109829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdb43d-7787-49be-8c9f-d35c364e055e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:12.109829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdb43d-7787-49be-8c9f-d35c364e055e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:12.109829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdb43d-7787-49be-8c9f-d35c364e055e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:12.109829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdb43d-7787-49be-8c9f-d35c364e055e None admin] PATH_INFO: `/v3/domains/default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:12.110942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdb43d-7787-49be-8c9f-d35c364e055e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:12.113275 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdb43d-7787-49be-8c9f-d35c364e055e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:12.115303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 79/144] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:49:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:12.121731 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdb43d-7787-49be-8c9f-d35c364e055e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:12.138729 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6307a03-4f5b-4708-8652-4a018d23d44c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:12.176737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994154d-20be-4a43-b3c0-3c29e4933f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:12.177732 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994154d-20be-4a43-b3c0-3c29e4933f17 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:12.178028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994154d-20be-4a43-b3c0-3c29e4933f17 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:12.178320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994154d-20be-4a43-b3c0-3c29e4933f17 None admin] PATH_INFO: `/v3/users/4f6c30de67e643ecb5582d374f07cb7c` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:12.179608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994154d-20be-4a43-b3c0-3c29e4933f17 None admin] RBAC: Authorizing `identity:get_user(user_id=4f6c30de67e643ecb5582d374f07cb7c)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:12.181125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994154d-20be-4a43-b3c0-3c29e4933f17 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:12.183084 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 66/145] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:11 2022] GET /identity/v3/users/4f6c30de67e643ecb5582d374f07cb7c => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:12.190580 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994154d-20be-4a43-b3c0-3c29e4933f17 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:12.252036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdb43d-7787-49be-8c9f-d35c364e055e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:12.295967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5dc14-60d7-4ff9-aa24-4a913a41217a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:12.297182 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5dc14-60d7-4ff9-aa24-4a913a41217a None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:12.297723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5dc14-60d7-4ff9-aa24-4a913a41217a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:12.298351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba5dc14-60d7-4ff9-aa24-4a913a41217a None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:12.299960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5dc14-60d7-4ff9-aa24-4a913a41217a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:12.301944 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba5dc14-60d7-4ff9-aa24-4a913a41217a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:12.310926 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994154d-20be-4a43-b3c0-3c29e4933f17 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:12.352285 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9b2ee-8b1e-40a5-8c34-fa7206b7c0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:12.353047 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9b2ee-8b1e-40a5-8c34-fa7206b7c0f4 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:12.353341 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9b2ee-8b1e-40a5-8c34-fa7206b7c0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:12.353630 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9b2ee-8b1e-40a5-8c34-fa7206b7c0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:12.370011 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:12.412376 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 80/146] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:49:12 2022] POST /identity/v3/users => generated 283 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:49:12.440694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9b2ee-8b1e-40a5-8c34-fa7206b7c0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:12.442234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9b2ee-8b1e-40a5-8c34-fa7206b7c0f4 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:12.623540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 67/147] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:12.629848 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5dc14-60d7-4ff9-aa24-4a913a41217a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:12.756252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:12.844635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba5dc14-60d7-4ff9-aa24-4a913a41217a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:12.887020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c3784-9d6e-4376-9dda-8eafa4e0993b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:12.887973 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c3784-9d6e-4376-9dda-8eafa4e0993b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:12.888261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c3784-9d6e-4376-9dda-8eafa4e0993b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:12.888570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c3784-9d6e-4376-9dda-8eafa4e0993b None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:12.889909 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c3784-9d6e-4376-9dda-8eafa4e0993b None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:12.892188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c3784-9d6e-4376-9dda-8eafa4e0993b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:12.894170 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 81/148] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:12 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:12.902293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9b2ee-8b1e-40a5-8c34-fa7206b7c0f4 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:12.949501 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:13.017398 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9b2ee-8b1e-40a5-8c34-fa7206b7c0f4 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:13.055159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6cb58-eef6-4113-8466-4c837d7d9cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:13.055847 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6cb58-eef6-4113-8466-4c837d7d9cd6 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:13.056141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6cb58-eef6-4113-8466-4c837d7d9cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:13.056467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6cb58-eef6-4113-8466-4c837d7d9cd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:13.057972 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6cb58-eef6-4113-8466-4c837d7d9cd6 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=834a091f1d39471ba13d3156d015bf3d, user.id=4f6c30de67e643ecb5582d374f07cb7c) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:13.058315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6cb58-eef6-4113-8466-4c837d7d9cd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:13.059934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6cb58-eef6-4113-8466-4c837d7d9cd6 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:13.067423 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 68/149] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:12 2022] GET /identity/v3/role_assignments?user.id=4f6c30de67e643ecb5582d374f07cb7c&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:13.811688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537db1c0-d365-42f1-bb01-cca698ff942e None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:13.812247 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537db1c0-d365-42f1-bb01-cca698ff942e None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:13.812247 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537db1c0-d365-42f1-bb01-cca698ff942e None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:13.813860 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 82/150] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:13.820703 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865787-4267-4e31-af0e-26ae95285266 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:13.821221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865787-4267-4e31-af0e-26ae95285266 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:13.821333 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865787-4267-4e31-af0e-26ae95285266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:13.843337 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-72865787-4267-4e31-af0e-26ae95285266 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:13.844624 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-72865787-4267-4e31-af0e-26ae95285266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:13.937995 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 69/151] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:13.968494 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537db1c0-d365-42f1-bb01-cca698ff942e None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:14.088901 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537db1c0-d365-42f1-bb01-cca698ff942e None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:14.112801 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:14.113517 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8890a77-6b14-4bd5-8b3d-12af550b2e06 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:14.113812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8890a77-6b14-4bd5-8b3d-12af550b2e06 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:14.114098 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8890a77-6b14-4bd5-8b3d-12af550b2e06 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:14.115823 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 70/152] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:49:14.122117 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5478a-422f-45cf-8a45-62fc69f81a01 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:14.122483 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5478a-422f-45cf-8a45-62fc69f81a01 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:14.127537 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5478a-422f-45cf-8a45-62fc69f81a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:14.146889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0f5bc-55af-4fb5-becf-209b4cfec00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:14.148129 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0f5bc-55af-4fb5-becf-209b4cfec00e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:14.148789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0f5bc-55af-4fb5-becf-209b4cfec00e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:14.149296 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0f5bc-55af-4fb5-becf-209b4cfec00e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:14.165572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-6bf5478a-422f-45cf-8a45-62fc69f81a01 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:14.166911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-6bf5478a-422f-45cf-8a45-62fc69f81a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:14.167461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0f5bc-55af-4fb5-becf-209b4cfec00e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-31b0f5bc-55af-4fb5-becf-209b4cfec00e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:14.177939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:14.179252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:14.180366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:14.183901 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 83/153] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:14.190375 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0f5bc-55af-4fb5-becf-209b4cfec00e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:14.208870 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:14.272832 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 71/154] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:14.314434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5478a-422f-45cf-8a45-62fc69f81a01 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:14.384632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0f5bc-55af-4fb5-becf-209b4cfec00e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:14.475981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de49c66-0914-423e-a365-cf0e142b6427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:14.476699 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49c66-0914-423e-a365-cf0e142b6427 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:14.476984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49c66-0914-423e-a365-cf0e142b6427 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:14.477257 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de49c66-0914-423e-a365-cf0e142b6427 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:14.478423 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de49c66-0914-423e-a365-cf0e142b6427 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:14.478898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de49c66-0914-423e-a365-cf0e142b6427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:14.480369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de49c66-0914-423e-a365-cf0e142b6427 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:14.499292 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 84/155] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:14.505840 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de49c66-0914-423e-a365-cf0e142b6427 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:14.539901 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5478a-422f-45cf-8a45-62fc69f81a01 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:14.606873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:14.628129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea6084-f959-4b46-8cbc-f67eae3c43bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:14.629566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea6084-f959-4b46-8cbc-f67eae3c43bb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:14.629566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea6084-f959-4b46-8cbc-f67eae3c43bb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:14.629566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea6084-f959-4b46-8cbc-f67eae3c43bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:14.657110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea6084-f959-4b46-8cbc-f67eae3c43bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-49ea6084-f959-4b46-8cbc-f67eae3c43bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:14.666839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:14.667992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:14.668987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:14.673036 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 72/156] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:14.678828 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea6084-f959-4b46-8cbc-f67eae3c43bb None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:14.736793 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:14.774727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de49c66-0914-423e-a365-cf0e142b6427 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:14.833195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea6084-f959-4b46-8cbc-f67eae3c43bb None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:14.875726 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f144c-df7e-4735-9bb9-fda8796ada98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:14.877379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f144c-df7e-4735-9bb9-fda8796ada98 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:14.877825 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f144c-df7e-4735-9bb9-fda8796ada98 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:14.878278 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f144c-df7e-4735-9bb9-fda8796ada98 None admin] PATH_INFO: `/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:14.885153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f144c-df7e-4735-9bb9-fda8796ada98 None admin] RBAC: Authorizing `identity:get_user(user_id=136b6f5ff4004c75b8b76ea91e0ec8d4)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:14.894768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f144c-df7e-4735-9bb9-fda8796ada98 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:14.900924 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 85/157] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:14 2022] GET /identity/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4 => generated 275 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:14.906792 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fc5b0-8d2f-4887-8c21-734373ad341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:14.907331 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f144c-df7e-4735-9bb9-fda8796ada98 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:14.907523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fc5b0-8d2f-4887-8c21-734373ad341d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:14.907775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fc5b0-8d2f-4887-8c21-734373ad341d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:14.908057 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fc5b0-8d2f-4887-8c21-734373ad341d None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:14.910510 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fc5b0-8d2f-4887-8c21-734373ad341d None admin] RBAC: Adding query filter params (name=member) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:14.910939 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fc5b0-8d2f-4887-8c21-734373ad341d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:14.912510 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fc5b0-8d2f-4887-8c21-734373ad341d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:14.929989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 73/158] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:14.935705 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fc5b0-8d2f-4887-8c21-734373ad341d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:15.105193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f144c-df7e-4735-9bb9-fda8796ada98 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:15.118198 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:15.160776 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:15.191576 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b1d3b-e119-4830-bba6-131897627b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:15.192251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b1d3b-e119-4830-bba6-131897627b3d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:15.192730 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b1d3b-e119-4830-bba6-131897627b3d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:15.193015 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b1d3b-e119-4830-bba6-131897627b3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:15.210558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fc5b0-8d2f-4887-8c21-734373ad341d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:15.302754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07d844-8781-4a8b-b0de-f588281e0469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:15.303686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07d844-8781-4a8b-b0de-f588281e0469 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:15.303972 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07d844-8781-4a8b-b0de-f588281e0469 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:15.304252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07d844-8781-4a8b-b0de-f588281e0469 None admin] PATH_INFO: `/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:15.309320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07d844-8781-4a8b-b0de-f588281e0469 None admin] RBAC: Authorizing `identity:get_user(user_id=136b6f5ff4004c75b8b76ea91e0ec8d4)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:15.310879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07d844-8781-4a8b-b0de-f588281e0469 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:15.312998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 74/159] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:14 2022] GET /identity/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4 => generated 275 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:15.314121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d79f96-26be-41ed-87d3-123b76eb0890 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:15.314409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d79f96-26be-41ed-87d3-123b76eb0890 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:15.314721 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d79f96-26be-41ed-87d3-123b76eb0890 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:15.316535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 75/160] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:15.323297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0238-97e3-466b-9414-ca9a91f1697f None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:15.323808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0238-97e3-466b-9414-ca9a91f1697f None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:15.324289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0238-97e3-466b-9414-ca9a91f1697f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:15.360696 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-533d0238-97e3-466b-9414-ca9a91f1697f None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:15.361988 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-533d0238-97e3-466b-9414-ca9a91f1697f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:15.394250 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b1d3b-e119-4830-bba6-131897627b3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:15.395975 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b1d3b-e119-4830-bba6-131897627b3d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:15.528606 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 76/161] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:15.529440 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d0238-97e3-466b-9414-ca9a91f1697f None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:15.660712 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:15.673137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d0238-97e3-466b-9414-ca9a91f1697f None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:15.706818 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 86/162] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 800 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:49:15.708485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fddcc7-c4ea-414c-a1c9-7d46e9b2774d None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:15.708777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fddcc7-c4ea-414c-a1c9-7d46e9b2774d None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:15.709057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fddcc7-c4ea-414c-a1c9-7d46e9b2774d None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:15.711014 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 87/163] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:15.711895 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fddcc7-c4ea-414c-a1c9-7d46e9b2774d None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:15.724817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:15.765697 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c80728-5b41-4b1d-929d-07039c12cbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:15.766380 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c80728-5b41-4b1d-929d-07039c12cbaa None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:15.766668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c80728-5b41-4b1d-929d-07039c12cbaa None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:15.766949 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c80728-5b41-4b1d-929d-07039c12cbaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:15.865215 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fddcc7-c4ea-414c-a1c9-7d46e9b2774d None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:15.930047 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c80728-5b41-4b1d-929d-07039c12cbaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:15.932006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c80728-5b41-4b1d-929d-07039c12cbaa None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:15.949685 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bdf6c-ca74-4bcd-9629-9adde628cad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:15.950526 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bdf6c-ca74-4bcd-9629-9adde628cad2 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:15.950813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bdf6c-ca74-4bcd-9629-9adde628cad2 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:15.951487 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bdf6c-ca74-4bcd-9629-9adde628cad2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:15.978720 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bdf6c-ca74-4bcd-9629-9adde628cad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-c77bdf6c-ca74-4bcd-9629-9adde628cad2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:15.995258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:15.996615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:15.997769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:16.004736 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 88/164] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:16.005704 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bdf6c-ca74-4bcd-9629-9adde628cad2 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:16.034083 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:16.057828 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 77/165] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:16.063893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a3afd-dc20-4aaa-838d-d680ee6932ec None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:16.065366 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a3afd-dc20-4aaa-838d-d680ee6932ec None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:16.065841 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519a3afd-dc20-4aaa-838d-d680ee6932ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:16.085083 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-519a3afd-dc20-4aaa-838d-d680ee6932ec None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:16.086617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-519a3afd-dc20-4aaa-838d-d680ee6932ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:16.167116 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 78/166] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:16.168379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a3afd-dc20-4aaa-838d-d680ee6932ec None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:16.185883 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:16.280207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bdf6c-ca74-4bcd-9629-9adde628cad2 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:16.283118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519a3afd-dc20-4aaa-838d-d680ee6932ec None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:16.298260 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:16.321676 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831c6b1-3d5a-4e18-9b34-d726bcdd4705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:16.322687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831c6b1-3d5a-4e18-9b34-d726bcdd4705 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:16.323163 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831c6b1-3d5a-4e18-9b34-d726bcdd4705 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:16.323595 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831c6b1-3d5a-4e18-9b34-d726bcdd4705 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:16.324557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831c6b1-3d5a-4e18-9b34-d726bcdd4705 None admin] RBAC: Adding query filter params (name=member) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:16.325024 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831c6b1-3d5a-4e18-9b34-d726bcdd4705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:16.326630 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831c6b1-3d5a-4e18-9b34-d726bcdd4705 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:16.336427 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 79/167] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:16.337955 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831c6b1-3d5a-4e18-9b34-d726bcdd4705 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:16.393727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db302c-0e75-46d2-b83c-448d7a762273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:16.394732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db302c-0e75-46d2-b83c-448d7a762273 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:16.394916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db302c-0e75-46d2-b83c-448d7a762273 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:16.395200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43db302c-0e75-46d2-b83c-448d7a762273 None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:16.401246 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db302c-0e75-46d2-b83c-448d7a762273 None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:16.405810 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43db302c-0e75-46d2-b83c-448d7a762273 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:16.405810 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 89/168] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:16 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:16.405810 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db302c-0e75-46d2-b83c-448d7a762273 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:16.456239 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831c6b1-3d5a-4e18-9b34-d726bcdd4705 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:16.498290 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09270731-c1fa-4500-bbc1-ba27b1b6389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:16.499328 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09270731-c1fa-4500-bbc1-ba27b1b6389b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:16.499719 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09270731-c1fa-4500-bbc1-ba27b1b6389b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:16.500247 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09270731-c1fa-4500-bbc1-ba27b1b6389b None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:16.501743 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09270731-c1fa-4500-bbc1-ba27b1b6389b None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:16.503377 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09270731-c1fa-4500-bbc1-ba27b1b6389b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:16.510089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 80/169] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:16 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:16.510089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09270731-c1fa-4500-bbc1-ba27b1b6389b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:16.551990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43db302c-0e75-46d2-b83c-448d7a762273 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:16.557868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:16.599824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf14ef-a98b-4b02-bc34-9e2d2327d3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:16.600872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf14ef-a98b-4b02-bc34-9e2d2327d3f4 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:16.601260 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf14ef-a98b-4b02-bc34-9e2d2327d3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:16.601642 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf14ef-a98b-4b02-bc34-9e2d2327d3f4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:16.605321 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:16.616913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf14ef-a98b-4b02-bc34-9e2d2327d3f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-e3bf14ef-a98b-4b02-bc34-9e2d2327d3f4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:16.625967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:16.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:16.628764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:16.630651 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 90/170] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:16.631488 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf14ef-a98b-4b02-bc34-9e2d2327d3f4 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:16.632300 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09270731-c1fa-4500-bbc1-ba27b1b6389b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:16.685164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7d5f4-89dc-4351-a2f9-def37ac16d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:16.686086 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7d5f4-89dc-4351-a2f9-def37ac16d67 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:16.686368 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7d5f4-89dc-4351-a2f9-def37ac16d67 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:16.686646 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7d5f4-89dc-4351-a2f9-def37ac16d67 None admin] PATH_INFO: `/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:16.692243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7d5f4-89dc-4351-a2f9-def37ac16d67 None admin] RBAC: Authorizing `identity:get_user(user_id=136b6f5ff4004c75b8b76ea91e0ec8d4)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:16.696069 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7d5f4-89dc-4351-a2f9-def37ac16d67 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:16.699274 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 81/171] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:16 2022] GET /identity/v3/users/136b6f5ff4004c75b8b76ea91e0ec8d4 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:16.708011 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3879ec3-418d-4825-814d-d4183f07309e None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:16.708628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3879ec3-418d-4825-814d-d4183f07309e None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:16.709131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3879ec3-418d-4825-814d-d4183f07309e None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:16.716005 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 82/172] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:16.717476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3879ec3-418d-4825-814d-d4183f07309e None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:16.802290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf14ef-a98b-4b02-bc34-9e2d2327d3f4 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:16.837670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3879ec3-418d-4825-814d-d4183f07309e None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:16.850863 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:16.873930 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799528da-668b-49fc-a701-5ca5768952ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:16.875283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799528da-668b-49fc-a701-5ca5768952ca None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:16.875897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799528da-668b-49fc-a701-5ca5768952ca None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:16.876521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799528da-668b-49fc-a701-5ca5768952ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:16.877771 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839b948-1e29-4d45-983e-c4c668fd86d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:16.878305 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799528da-668b-49fc-a701-5ca5768952ca None admin] RBAC: Adding query filter params (role.id=4313b1c9df5345778a249b04355b90a0, scope.project.id=834a091f1d39471ba13d3156d015bf3d, user.id=136b6f5ff4004c75b8b76ea91e0ec8d4) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:16.878879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839b948-1e29-4d45-983e-c4c668fd86d4 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:16.878996 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799528da-668b-49fc-a701-5ca5768952ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:16.879448 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839b948-1e29-4d45-983e-c4c668fd86d4 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:16.880004 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839b948-1e29-4d45-983e-c4c668fd86d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:16.881076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839b948-1e29-4d45-983e-c4c668fd86d4 None admin] RBAC: Adding query filter params (role.id=2e3b001c16d24c279be5c68aa3847ca7, scope.project.id=834a091f1d39471ba13d3156d015bf3d, user.id=136b6f5ff4004c75b8b76ea91e0ec8d4) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:16.881908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839b948-1e29-4d45-983e-c4c668fd86d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:16.883711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839b948-1e29-4d45-983e-c4c668fd86d4 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:16.884367 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799528da-668b-49fc-a701-5ca5768952ca None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:16.891823 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 83/173] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:16 2022] GET /identity/v3/role_assignments?user.id=136b6f5ff4004c75b8b76ea91e0ec8d4&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:16.893783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839b948-1e29-4d45-983e-c4c668fd86d4 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:16.904372 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 91/174] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:16 2022] GET /identity/v3/role_assignments?user.id=136b6f5ff4004c75b8b76ea91e0ec8d4&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=4313b1c9df5345778a249b04355b90a0 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:16.904372 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799528da-668b-49fc-a701-5ca5768952ca None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:17.054117 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:17.136474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839b948-1e29-4d45-983e-c4c668fd86d4 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:17.161524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799528da-668b-49fc-a701-5ca5768952ca None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:17.211161 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c8a0b-bf0b-4823-99b9-03924121dcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:17.212795 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c8a0b-bf0b-4823-99b9-03924121dcc5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.213628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c8a0b-bf0b-4823-99b9-03924121dcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.214510 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c8a0b-bf0b-4823-99b9-03924121dcc5 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.216329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c8a0b-bf0b-4823-99b9-03924121dcc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:17.217538 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c8a0b-bf0b-4823-99b9-03924121dcc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:17.220228 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc777d-dbee-4b23-8081-1320592e0894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:17.220990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc777d-dbee-4b23-8081-1320592e0894 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.221273 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc777d-dbee-4b23-8081-1320592e0894 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.221559 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc777d-dbee-4b23-8081-1320592e0894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.221971 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c8a0b-bf0b-4823-99b9-03924121dcc5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:17.237484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 84/175] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:17.239422 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ac08e-22b1-49ec-9448-94f5fc6f4a05 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.239942 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ac08e-22b1-49ec-9448-94f5fc6f4a05 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.240467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ac08e-22b1-49ec-9448-94f5fc6f4a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.261497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-466ac08e-22b1-49ec-9448-94f5fc6f4a05 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:17.263135 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-466ac08e-22b1-49ec-9448-94f5fc6f4a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:17.300099 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:17.315329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc777d-dbee-4b23-8081-1320592e0894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:17.315329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc777d-dbee-4b23-8081-1320592e0894 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:17.349163 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 85/176] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:17.353705 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ac08e-22b1-49ec-9448-94f5fc6f4a05 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:17.388580 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:17.481913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 92/177] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:17.483273 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6fec7-2abc-4499-95e8-06ebfd0074ea None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.483570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6fec7-2abc-4499-95e8-06ebfd0074ea None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.483850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6fec7-2abc-4499-95e8-06ebfd0074ea None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.485244 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 93/178] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:17.485905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6fec7-2abc-4499-95e8-06ebfd0074ea None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:17.519683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ac08e-22b1-49ec-9448-94f5fc6f4a05 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:17.554951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe6971-b83e-44cb-86ec-d81418d4fa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:17.555831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe6971-b83e-44cb-86ec-d81418d4fa40 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.556163 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe6971-b83e-44cb-86ec-d81418d4fa40 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.556468 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe6971-b83e-44cb-86ec-d81418d4fa40 None admin] PATH_INFO: `/v3/users/c38801f1b5fb429592a3393dd72f3c4f` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.557935 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe6971-b83e-44cb-86ec-d81418d4fa40 None admin] RBAC: Authorizing `identity:get_user(user_id=c38801f1b5fb429592a3393dd72f3c4f)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:17.559382 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe6971-b83e-44cb-86ec-d81418d4fa40 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:17.561066 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 86/179] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:17 2022] GET /identity/v3/users/c38801f1b5fb429592a3393dd72f3c4f => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:17.566167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bddb26d-704e-46f6-af01-46ac23ae675c None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.566167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bddb26d-704e-46f6-af01-46ac23ae675c None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.566167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bddb26d-704e-46f6-af01-46ac23ae675c None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.566167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 87/180] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:17.566167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bddb26d-704e-46f6-af01-46ac23ae675c None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:17.593683 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6fec7-2abc-4499-95e8-06ebfd0074ea None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:17.604371 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:17.630307 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fee143-8d0c-437b-9742-c85cfc1ab7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:17.631108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fee143-8d0c-437b-9742-c85cfc1ab7d7 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.631389 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fee143-8d0c-437b-9742-c85cfc1ab7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.631663 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fee143-8d0c-437b-9742-c85cfc1ab7d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.643958 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fee143-8d0c-437b-9742-c85cfc1ab7d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-68fee143-8d0c-437b-9742-c85cfc1ab7d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:17.651834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:17.652984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:17.653984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 94/181] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:17.655308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927d53e-b768-4358-8b80-5e8f4dc6a781 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.655308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927d53e-b768-4358-8b80-5e8f4dc6a781 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.655308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927d53e-b768-4358-8b80-5e8f4dc6a781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.672874 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-2927d53e-b768-4358-8b80-5e8f4dc6a781 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:17.674112 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-2927d53e-b768-4358-8b80-5e8f4dc6a781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:17.677175 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bddb26d-704e-46f6-af01-46ac23ae675c None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:17.688871 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:17.714189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88563949-c5b7-4615-b714-1a11df2058e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:17.715074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88563949-c5b7-4615-b714-1a11df2058e4 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.715362 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88563949-c5b7-4615-b714-1a11df2058e4 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.715644 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88563949-c5b7-4615-b714-1a11df2058e4 None admin] PATH_INFO: `/v3/projects/7ece7251349140ca9d947cf7aec97a90` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.716888 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88563949-c5b7-4615-b714-1a11df2058e4 None admin] RBAC: Authorizing `identity:get_project(project_id=7ece7251349140ca9d947cf7aec97a90)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:17.718337 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88563949-c5b7-4615-b714-1a11df2058e4 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:17.720073 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 88/182] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:17 2022] GET /identity/v3/projects/7ece7251349140ca9d947cf7aec97a90 => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:17.720927 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88563949-c5b7-4615-b714-1a11df2058e4 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:17.745850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 95/183] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:17.753690 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec022e9-bee8-4519-990c-13623b613e86 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.754109 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec022e9-bee8-4519-990c-13623b613e86 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.754388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec022e9-bee8-4519-990c-13623b613e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.775313 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-bec022e9-bee8-4519-990c-13623b613e86 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:17.776553 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-bec022e9-bee8-4519-990c-13623b613e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:17.835937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88563949-c5b7-4615-b714-1a11df2058e4 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:17.837486 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:17.845739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 96/184] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:17.845739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec022e9-bee8-4519-990c-13623b613e86 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:17.874776 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883d4a0-6c0e-4b1f-b016-473113619eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:17.875659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883d4a0-6c0e-4b1f-b016-473113619eb6 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.875945 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883d4a0-6c0e-4b1f-b016-473113619eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.876222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883d4a0-6c0e-4b1f-b016-473113619eb6 None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.877484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883d4a0-6c0e-4b1f-b016-473113619eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:17.878938 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883d4a0-6c0e-4b1f-b016-473113619eb6 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:17.880689 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 89/185] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:17 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:17.881525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883d4a0-6c0e-4b1f-b016-473113619eb6 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:17.932630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:17.956770 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec022e9-bee8-4519-990c-13623b613e86 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:17.987791 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ca9dd-147e-4a5b-978a-31f605dba639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:17.988504 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca9dd-147e-4a5b-978a-31f605dba639 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:17.988793 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca9dd-147e-4a5b-978a-31f605dba639 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:17.989067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ca9dd-147e-4a5b-978a-31f605dba639 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:17.989826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ca9dd-147e-4a5b-978a-31f605dba639 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:17.990136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883d4a0-6c0e-4b1f-b016-473113619eb6 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:17.990247 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ca9dd-147e-4a5b-978a-31f605dba639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:17.991572 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ca9dd-147e-4a5b-978a-31f605dba639 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:18.000234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 97/186] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:18.000234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ca9dd-147e-4a5b-978a-31f605dba639 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:18.030550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593d0b7-004b-492e-8f33-510f6dc8a51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:18.031232 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593d0b7-004b-492e-8f33-510f6dc8a51b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:18.031515 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593d0b7-004b-492e-8f33-510f6dc8a51b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:18.031798 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593d0b7-004b-492e-8f33-510f6dc8a51b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:18.032681 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593d0b7-004b-492e-8f33-510f6dc8a51b None admin] RBAC: Adding query filter params (role.id=2e3b001c16d24c279be5c68aa3847ca7, scope.project.id=7ece7251349140ca9d947cf7aec97a90, user.id=136b6f5ff4004c75b8b76ea91e0ec8d4) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:18.032996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593d0b7-004b-492e-8f33-510f6dc8a51b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:18.035002 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593d0b7-004b-492e-8f33-510f6dc8a51b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:18.041928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 90/187] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:17 2022] GET /identity/v3/role_assignments?user.id=136b6f5ff4004c75b8b76ea91e0ec8d4&scope.project.id=7ece7251349140ca9d947cf7aec97a90&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:18.041928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593d0b7-004b-492e-8f33-510f6dc8a51b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:18.087704 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:18.126949 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ca9dd-147e-4a5b-978a-31f605dba639 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:18.171713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f6e6d-df7d-44e3-b84d-f920e2e6a701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:18.172873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f6e6d-df7d-44e3-b84d-f920e2e6a701 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:18.173167 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f6e6d-df7d-44e3-b84d-f920e2e6a701 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:18.173451 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f6e6d-df7d-44e3-b84d-f920e2e6a701 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:18.177691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593d0b7-004b-492e-8f33-510f6dc8a51b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:18.188108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f6e6d-df7d-44e3-b84d-f920e2e6a701 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-4d3f6e6d-df7d-44e3-b84d-f920e2e6a701 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:18.196713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:18.203047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 98/188] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:18.204139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f6e6d-df7d-44e3-b84d-f920e2e6a701 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:18.205030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:18.229643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f73d4-a124-410a-87f0-d456129a5cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:18.230466 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f73d4-a124-410a-87f0-d456129a5cdd None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:18.230767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f73d4-a124-410a-87f0-d456129a5cdd None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:18.231008 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f73d4-a124-410a-87f0-d456129a5cdd None admin] PATH_INFO: `/v3/domains/default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:18.232265 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f73d4-a124-410a-87f0-d456129a5cdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:18.233766 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f73d4-a124-410a-87f0-d456129a5cdd None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:18.235803 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 91/189] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:49:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:18.236680 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f73d4-a124-410a-87f0-d456129a5cdd None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:18.333147 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f6e6d-df7d-44e3-b84d-f920e2e6a701 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:18.358622 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f73d4-a124-410a-87f0-d456129a5cdd None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:18.360129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:18.369420 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1abfa-9342-49ab-bbdf-2ab35b114495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:18.370310 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1abfa-9342-49ab-bbdf-2ab35b114495 None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:18.370595 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1abfa-9342-49ab-bbdf-2ab35b114495 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:18.370872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1abfa-9342-49ab-bbdf-2ab35b114495 None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d/users/c38801f1b5fb429592a3393dd72f3c4f/roles/83321c42fce7438aa5e29c1acb4f07af` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:18.372939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1abfa-9342-49ab-bbdf-2ab35b114495 None admin] RBAC: Authorizing `identity:create_grant(project_id=834a091f1d39471ba13d3156d015bf3d, user_id=c38801f1b5fb429592a3393dd72f3c4f, role_id=83321c42fce7438aa5e29c1acb4f07af)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:18.374507 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1abfa-9342-49ab-bbdf-2ab35b114495 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:18.393144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513c562-3944-4c3c-b63a-54ce9064abac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:18.394042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513c562-3944-4c3c-b63a-54ce9064abac None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:18.394625 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513c562-3944-4c3c-b63a-54ce9064abac None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:18.394901 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513c562-3944-4c3c-b63a-54ce9064abac None admin] PATH_INFO: `/v3/users/da341dea30f049cf832df018785674b8` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:18.408132 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513c562-3944-4c3c-b63a-54ce9064abac None admin] RBAC: Authorizing `identity:get_user(user_id=da341dea30f049cf832df018785674b8)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:18.410222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513c562-3944-4c3c-b63a-54ce9064abac None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:18.411895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 92/190] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:18 2022] GET /identity/v3/users/da341dea30f049cf832df018785674b8 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:18.412704 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513c562-3944-4c3c-b63a-54ce9064abac None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:18.472937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 99/191] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:49:18 2022] PUT /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d/users/c38801f1b5fb429592a3393dd72f3c4f/roles/83321c42fce7438aa5e29c1acb4f07af => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:18.473765 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1abfa-9342-49ab-bbdf-2ab35b114495 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:18.532726 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513c562-3944-4c3c-b63a-54ce9064abac None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:18.549656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:18.574144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07f647-08a2-492a-a19f-b94d67bcc0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:18.574850 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07f647-08a2-492a-a19f-b94d67bcc0b7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:18.575201 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07f647-08a2-492a-a19f-b94d67bcc0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:18.575488 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07f647-08a2-492a-a19f-b94d67bcc0b7 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:18.576277 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07f647-08a2-492a-a19f-b94d67bcc0b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:18.576634 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07f647-08a2-492a-a19f-b94d67bcc0b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:18.578141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07f647-08a2-492a-a19f-b94d67bcc0b7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:18.595696 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 93/192] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:18.596265 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07f647-08a2-492a-a19f-b94d67bcc0b7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:18.600132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1abfa-9342-49ab-bbdf-2ab35b114495 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:18.629746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:18.640290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639a31a-3c6d-4db2-846d-4d4fa6398885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:18.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639a31a-3c6d-4db2-846d-4d4fa6398885 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:18.646024 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639a31a-3c6d-4db2-846d-4d4fa6398885 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:18.646024 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639a31a-3c6d-4db2-846d-4d4fa6398885 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:18.646024 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639a31a-3c6d-4db2-846d-4d4fa6398885 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:18.646024 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639a31a-3c6d-4db2-846d-4d4fa6398885 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:18.717998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07f647-08a2-492a-a19f-b94d67bcc0b7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:18.753425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33e7cb-2641-406f-a671-43327bc31ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:18.754298 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33e7cb-2641-406f-a671-43327bc31ae1 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:18.754918 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33e7cb-2641-406f-a671-43327bc31ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:18.755213 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33e7cb-2641-406f-a671-43327bc31ae1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:18.768681 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 100/193] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:49:18 2022] POST /identity/v3/users => generated 297 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:49:18.768681 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639a31a-3c6d-4db2-846d-4d4fa6398885 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:18.853792 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33e7cb-2641-406f-a671-43327bc31ae1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:18.855837 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33e7cb-2641-406f-a671-43327bc31ae1 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:18.900448 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:18.912437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639a31a-3c6d-4db2-846d-4d4fa6398885 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:18.939153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:18.959931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b434d-9362-4034-b40a-ef06cbee66ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:18.961350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b434d-9362-4034-b40a-ef06cbee66ed None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:18.961868 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b434d-9362-4034-b40a-ef06cbee66ed None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:18.962514 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b434d-9362-4034-b40a-ef06cbee66ed None admin] PATH_INFO: `/v3/users/da341dea30f049cf832df018785674b8` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:18.964248 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b434d-9362-4034-b40a-ef06cbee66ed None admin] RBAC: Authorizing `identity:get_user(user_id=da341dea30f049cf832df018785674b8)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:18.966086 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b434d-9362-4034-b40a-ef06cbee66ed None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:18.968770 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 101/194] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:18 2022] GET /identity/v3/users/da341dea30f049cf832df018785674b8 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:18.974049 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b434d-9362-4034-b40a-ef06cbee66ed None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:19.009939 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 94/195] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:19.016466 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33e7cb-2641-406f-a671-43327bc31ae1 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:19.114004 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b434d-9362-4034-b40a-ef06cbee66ed None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:19.157097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33e7cb-2641-406f-a671-43327bc31ae1 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:19.163414 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3cd28-1203-4ae1-a651-2040320af78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:19.164179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3cd28-1203-4ae1-a651-2040320af78e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:19.164509 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3cd28-1203-4ae1-a651-2040320af78e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:19.164794 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3cd28-1203-4ae1-a651-2040320af78e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:19.193736 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:19.208434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56dfe5-23bd-41f5-9f69-96f36b23c0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:19.209374 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56dfe5-23bd-41f5-9f69-96f36b23c0e4 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:19.209660 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56dfe5-23bd-41f5-9f69-96f36b23c0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:19.209936 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56dfe5-23bd-41f5-9f69-96f36b23c0e4 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:19.211459 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56dfe5-23bd-41f5-9f69-96f36b23c0e4 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:19.213224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56dfe5-23bd-41f5-9f69-96f36b23c0e4 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:19.216130 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 95/196] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:19 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:19.221999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56dfe5-23bd-41f5-9f69-96f36b23c0e4 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:19.242106 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:19.256973 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3cd28-1203-4ae1-a651-2040320af78e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:19.258902 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3cd28-1203-4ae1-a651-2040320af78e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:19.356988 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56dfe5-23bd-41f5-9f69-96f36b23c0e4 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:19.373403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 102/197] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:19.379372 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3cd28-1203-4ae1-a651-2040320af78e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:19.400243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b783d74-a039-4ffc-bbb1-d0af3d620449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:19.401057 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b783d74-a039-4ffc-bbb1-d0af3d620449 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:19.401344 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b783d74-a039-4ffc-bbb1-d0af3d620449 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:19.401673 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b783d74-a039-4ffc-bbb1-d0af3d620449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:19.402786 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b783d74-a039-4ffc-bbb1-d0af3d620449 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2, user.id=da341dea30f049cf832df018785674b8) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:19.403007 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b783d74-a039-4ffc-bbb1-d0af3d620449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:19.404593 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b783d74-a039-4ffc-bbb1-d0af3d620449 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:19.413621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 96/198] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:19 2022] GET /identity/v3/role_assignments?user.id=da341dea30f049cf832df018785674b8&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:19.497252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:19.512061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3cd28-1203-4ae1-a651-2040320af78e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:19.569728 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27909de1-fd61-4f26-8e88-a2714ba2e6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:19.570760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27909de1-fd61-4f26-8e88-a2714ba2e6d0 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:19.571055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27909de1-fd61-4f26-8e88-a2714ba2e6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:19.571325 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27909de1-fd61-4f26-8e88-a2714ba2e6d0 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:19.572704 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27909de1-fd61-4f26-8e88-a2714ba2e6d0 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:19.574291 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27909de1-fd61-4f26-8e88-a2714ba2e6d0 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:19.576167 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 103/199] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:19 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:19.583035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b783d74-a039-4ffc-bbb1-d0af3d620449 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:19.673933 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:19.743739 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b783d74-a039-4ffc-bbb1-d0af3d620449 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:19.785336 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e524a5e-3a2d-4fd5-9034-de7a2e6bdeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:19.786008 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e524a5e-3a2d-4fd5-9034-de7a2e6bdeb2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:19.786295 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e524a5e-3a2d-4fd5-9034-de7a2e6bdeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:19.786577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e524a5e-3a2d-4fd5-9034-de7a2e6bdeb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:19.787426 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e524a5e-3a2d-4fd5-9034-de7a2e6bdeb2 None admin] RBAC: Adding query filter params (role.id=4313b1c9df5345778a249b04355b90a0, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2, user.id=da341dea30f049cf832df018785674b8) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:19.787743 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e524a5e-3a2d-4fd5-9034-de7a2e6bdeb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:19.789220 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e524a5e-3a2d-4fd5-9034-de7a2e6bdeb2 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:19.796380 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 97/200] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:19 2022] GET /identity/v3/role_assignments?user.id=da341dea30f049cf832df018785674b8&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=4313b1c9df5345778a249b04355b90a0 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:21.245375 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689bb89-bcac-4896-8ecd-1997f5466b58 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:21.245953 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689bb89-bcac-4896-8ecd-1997f5466b58 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:21.245953 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689bb89-bcac-4896-8ecd-1997f5466b58 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:21.247473 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 104/201] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:21.253772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77210612-76cb-4413-a6df-ad41487e9c59 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:21.254430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77210612-76cb-4413-a6df-ad41487e9c59 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:21.254430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77210612-76cb-4413-a6df-ad41487e9c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:21.277700 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-77210612-76cb-4413-a6df-ad41487e9c59 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:21.278937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-77210612-76cb-4413-a6df-ad41487e9c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:21.370963 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 98/202] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:21.415490 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689bb89-bcac-4896-8ecd-1997f5466b58 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:21.555873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689bb89-bcac-4896-8ecd-1997f5466b58 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:21.621820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e3ba8a-4d4a-4240-abd7-6b62e81ab3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:21.622649 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3ba8a-4d4a-4240-abd7-6b62e81ab3e7 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:21.622936 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3ba8a-4d4a-4240-abd7-6b62e81ab3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:21.623381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e3ba8a-4d4a-4240-abd7-6b62e81ab3e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:21.645520 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e3ba8a-4d4a-4240-abd7-6b62e81ab3e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-48e3ba8a-4d4a-4240-abd7-6b62e81ab3e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:21.653931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:21.656668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:21.658846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 105/203] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:21.665160 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77210612-76cb-4413-a6df-ad41487e9c59 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:21.748964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:21.780466 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9df2c5-b3f0-4784-b387-b6725c7abd1a None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:21.781632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9df2c5-b3f0-4784-b387-b6725c7abd1a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:21.781632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9df2c5-b3f0-4784-b387-b6725c7abd1a None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:21.782841 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 106/204] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:21.788968 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcf92f-25cc-4ba1-b6c8-03bbe0921441 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:21.789648 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcf92f-25cc-4ba1-b6c8-03bbe0921441 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:21.790155 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bcf92f-25cc-4ba1-b6c8-03bbe0921441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:21.819277 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77210612-76cb-4413-a6df-ad41487e9c59 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:21.833105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-07bcf92f-25cc-4ba1-b6c8-03bbe0921441 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:21.834350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-07bcf92f-25cc-4ba1-b6c8-03bbe0921441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:21.860372 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:21.879652 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0e3fe-5241-498a-ac4f-0caa68e85407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:21.880371 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0e3fe-5241-498a-ac4f-0caa68e85407 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:21.880654 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0e3fe-5241-498a-ac4f-0caa68e85407 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:21.880937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0e3fe-5241-498a-ac4f-0caa68e85407 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:21.881708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0e3fe-5241-498a-ac4f-0caa68e85407 None admin] RBAC: Adding query filter params (name=reader) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:21.882023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0e3fe-5241-498a-ac4f-0caa68e85407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:21.883540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0e3fe-5241-498a-ac4f-0caa68e85407 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:21.901329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 99/205] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:21.909826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0e3fe-5241-498a-ac4f-0caa68e85407 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:21.953724 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 107/206] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:21.978806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcf92f-25cc-4ba1-b6c8-03bbe0921441 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:22.104428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0e3fe-5241-498a-ac4f-0caa68e85407 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:22.108526 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bcf92f-25cc-4ba1-b6c8-03bbe0921441 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:22.118346 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:22.152478 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f39694-0af2-4811-9bc2-8ee4aca7bc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:22.153398 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f39694-0af2-4811-9bc2-8ee4aca7bc0f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.153676 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f39694-0af2-4811-9bc2-8ee4aca7bc0f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.153954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f39694-0af2-4811-9bc2-8ee4aca7bc0f None admin] PATH_INFO: `/v3/users/c38801f1b5fb429592a3393dd72f3c4f` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.155332 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f39694-0af2-4811-9bc2-8ee4aca7bc0f None admin] RBAC: Authorizing `identity:get_user(user_id=c38801f1b5fb429592a3393dd72f3c4f)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:22.155949 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b33d6da-e8aa-4712-bdd8-9079d86b9b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:22.156854 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f39694-0af2-4811-9bc2-8ee4aca7bc0f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:22.157406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33d6da-e8aa-4712-bdd8-9079d86b9b34 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.157940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33d6da-e8aa-4712-bdd8-9079d86b9b34 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.159005 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b33d6da-e8aa-4712-bdd8-9079d86b9b34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.160424 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 100/207] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:21 2022] GET /identity/v3/users/c38801f1b5fb429592a3393dd72f3c4f => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:22.162104 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641223f-afe6-4e35-b9f9-37005529436f None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.163797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641223f-afe6-4e35-b9f9-37005529436f None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.164620 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641223f-afe6-4e35-b9f9-37005529436f None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.166116 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 101/208] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:22.168430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641223f-afe6-4e35-b9f9-37005529436f None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:22.184176 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b33d6da-e8aa-4712-bdd8-9079d86b9b34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-0b33d6da-e8aa-4712-bdd8-9079d86b9b34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:22.198397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:22.200239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:22.201242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:22.202739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 108/209] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:49:22.207866 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d319a8-759b-47e9-81d2-06a3e52c6f14 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.207866 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d319a8-759b-47e9-81d2-06a3e52c6f14 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.207866 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d319a8-759b-47e9-81d2-06a3e52c6f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.220433 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:22.227498 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-f5d319a8-759b-47e9-81d2-06a3e52c6f14 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:22.229007 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-f5d319a8-759b-47e9-81d2-06a3e52c6f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:22.305947 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 109/210] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:22.306854 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d319a8-759b-47e9-81d2-06a3e52c6f14 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:22.313446 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641223f-afe6-4e35-b9f9-37005529436f None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:22.351903 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707757f-86df-420b-b4a7-ba7b1a893c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:22.352589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707757f-86df-420b-b4a7-ba7b1a893c44 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.352873 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707757f-86df-420b-b4a7-ba7b1a893c44 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.353162 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707757f-86df-420b-b4a7-ba7b1a893c44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.405465 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:22.428678 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707757f-86df-420b-b4a7-ba7b1a893c44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:22.430152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707757f-86df-420b-b4a7-ba7b1a893c44 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:22.430359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d319a8-759b-47e9-81d2-06a3e52c6f14 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:22.474794 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587d73c-7c29-4be9-a637-90a1c0897c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:22.475480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d73c-7c29-4be9-a637-90a1c0897c3f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.475767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d73c-7c29-4be9-a637-90a1c0897c3f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.476047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587d73c-7c29-4be9-a637-90a1c0897c3f None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.477008 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587d73c-7c29-4be9-a637-90a1c0897c3f None admin] RBAC: Adding query filter params (name=member) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:22.477323 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587d73c-7c29-4be9-a637-90a1c0897c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:22.478741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587d73c-7c29-4be9-a637-90a1c0897c3f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:22.480802 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:22.491458 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 110/211] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:22.492837 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73ce94-e2b8-4b71-9c5e-be7688a7d0a9 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.499933 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73ce94-e2b8-4b71-9c5e-be7688a7d0a9 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.499933 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73ce94-e2b8-4b71-9c5e-be7688a7d0a9 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.499933 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 111/212] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:22.500237 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73ce94-e2b8-4b71-9c5e-be7688a7d0a9 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:22.553585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 102/213] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:22.553585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707757f-86df-420b-b4a7-ba7b1a893c44 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:22.635314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73ce94-e2b8-4b71-9c5e-be7688a7d0a9 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:22.636418 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:22.663273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707757f-86df-420b-b4a7-ba7b1a893c44 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:22.674274 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e7f60-ae99-4884-ab36-935fa5f82aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:22.675382 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e7f60-ae99-4884-ab36-935fa5f82aea None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.675731 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e7f60-ae99-4884-ab36-935fa5f82aea None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.676020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e7f60-ae99-4884-ab36-935fa5f82aea None admin] PATH_INFO: `/v3/domains/default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.677489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e7f60-ae99-4884-ab36-935fa5f82aea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:22.679042 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e7f60-ae99-4884-ab36-935fa5f82aea None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:22.680925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 112/214] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:49:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:22.682240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f30c75-57bb-4ce6-a1cd-002c3a93a006 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.682846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f30c75-57bb-4ce6-a1cd-002c3a93a006 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.683136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f30c75-57bb-4ce6-a1cd-002c3a93a006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.701653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-84f30c75-57bb-4ce6-a1cd-002c3a93a006 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:22.702865 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-84f30c75-57bb-4ce6-a1cd-002c3a93a006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:22.715943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed384613-d074-4308-b3ca-83ada1db04bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:22.717229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed384613-d074-4308-b3ca-83ada1db04bf None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.717517 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed384613-d074-4308-b3ca-83ada1db04bf None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.718063 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed384613-d074-4308-b3ca-83ada1db04bf None admin] PATH_INFO: `/v3/users/c41b01ba1dda49d989972952eeb81619` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.736176 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed384613-d074-4308-b3ca-83ada1db04bf None admin] RBAC: Authorizing `identity:get_user(user_id=c41b01ba1dda49d989972952eeb81619)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:22.738534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed384613-d074-4308-b3ca-83ada1db04bf None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:22.740865 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 103/215] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:22 2022] GET /identity/v3/users/c41b01ba1dda49d989972952eeb81619 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:22.741467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed384613-d074-4308-b3ca-83ada1db04bf None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:22.774289 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:22.783703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 113/216] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:22.784329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f30c75-57bb-4ce6-a1cd-002c3a93a006 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:22.867718 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed384613-d074-4308-b3ca-83ada1db04bf None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:22.901944 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f30c75-57bb-4ce6-a1cd-002c3a93a006 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:22.902803 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692731e-026a-4e29-ad3e-031f9c30ea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:22.903695 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692731e-026a-4e29-ad3e-031f9c30ea78 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.903980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692731e-026a-4e29-ad3e-031f9c30ea78 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.904268 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692731e-026a-4e29-ad3e-031f9c30ea78 None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.905560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692731e-026a-4e29-ad3e-031f9c30ea78 None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:22.907034 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692731e-026a-4e29-ad3e-031f9c30ea78 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:22.909369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 104/217] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:22 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:22.910155 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692731e-026a-4e29-ad3e-031f9c30ea78 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:22.914923 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:22.941629 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8e4f6-f789-4604-9c83-1ff75ca993c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:22.942319 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8e4f6-f789-4604-9c83-1ff75ca993c8 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:22.942632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8e4f6-f789-4604-9c83-1ff75ca993c8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:22.942915 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8e4f6-f789-4604-9c83-1ff75ca993c8 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:22.943669 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8e4f6-f789-4604-9c83-1ff75ca993c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:22.945476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b8e4f6-f789-4604-9c83-1ff75ca993c8 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:23.016018 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692731e-026a-4e29-ad3e-031f9c30ea78 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:23.044386 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 114/218] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:49:22 2022] POST /identity/v3/users => generated 297 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:49:23.050360 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8e4f6-f789-4604-9c83-1ff75ca993c8 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:23.052894 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2a5e8-13f4-4c0b-9773-69a63612fb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:23.053581 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2a5e8-13f4-4c0b-9773-69a63612fb91 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:23.053865 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2a5e8-13f4-4c0b-9773-69a63612fb91 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:23.054210 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2a5e8-13f4-4c0b-9773-69a63612fb91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:23.125508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:23.134856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2a5e8-13f4-4c0b-9773-69a63612fb91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:23.136412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2a5e8-13f4-4c0b-9773-69a63612fb91 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:23.140991 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:23.191015 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8e4f6-f789-4604-9c83-1ff75ca993c8 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:23.237140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88a17d-d625-4280-86b3-1adf401d012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:23.237919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88a17d-d625-4280-86b3-1adf401d012a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:23.238207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88a17d-d625-4280-86b3-1adf401d012a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:23.238476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e88a17d-d625-4280-86b3-1adf401d012a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:23.251349 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e88a17d-d625-4280-86b3-1adf401d012a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:23.256289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 105/219] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-4e88a17d-d625-4280-86b3-1adf401d012a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:23.259992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:23.261145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:23.262135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:23.264319 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f09c8-ad37-4eba-97bd-cbf26c2fb1d8 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:23.264319 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f09c8-ad37-4eba-97bd-cbf26c2fb1d8 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:23.264523 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 115/220] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:49:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:23.264602 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f09c8-ad37-4eba-97bd-cbf26c2fb1d8 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:23.264676 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88a17d-d625-4280-86b3-1adf401d012a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:23.265275 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 106/221] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:23.265275 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f09c8-ad37-4eba-97bd-cbf26c2fb1d8 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:23.400638 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f09c8-ad37-4eba-97bd-cbf26c2fb1d8 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:23.405407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:23.407713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e88a17d-d625-4280-86b3-1adf401d012a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:23.458450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:23.488368 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb6233-92d1-4356-aec0-636830089ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:23.489713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb6233-92d1-4356-aec0-636830089ac0 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:23.490006 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb6233-92d1-4356-aec0-636830089ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:23.490291 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb6233-92d1-4356-aec0-636830089ac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:23.491450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb6233-92d1-4356-aec0-636830089ac0 None admin] RBAC: Adding query filter params (role.id=83321c42fce7438aa5e29c1acb4f07af, scope.project.id=834a091f1d39471ba13d3156d015bf3d, user.id=c38801f1b5fb429592a3393dd72f3c4f) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:23.491768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb6233-92d1-4356-aec0-636830089ac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:23.493559 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb6233-92d1-4356-aec0-636830089ac0 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:23.504976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 116/222] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:23 2022] GET /identity/v3/role_assignments?user.id=c38801f1b5fb429592a3393dd72f3c4f&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=83321c42fce7438aa5e29c1acb4f07af => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:23.506043 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0844c98-285a-48fa-a98d-6410f5d1ca9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:23.506241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ce6f9-42a7-4c3a-a256-11679cc99027 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:23.506582 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ce6f9-42a7-4c3a-a256-11679cc99027 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:23.506827 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ce6f9-42a7-4c3a-a256-11679cc99027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:23.507837 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0844c98-285a-48fa-a98d-6410f5d1ca9f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:23.508329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0844c98-285a-48fa-a98d-6410f5d1ca9f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:23.508867 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0844c98-285a-48fa-a98d-6410f5d1ca9f None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:23.511164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0844c98-285a-48fa-a98d-6410f5d1ca9f None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:23.513963 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0844c98-285a-48fa-a98d-6410f5d1ca9f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:23.517435 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 107/223] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:23 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:23.519632 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0844c98-285a-48fa-a98d-6410f5d1ca9f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:23.527929 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-136ce6f9-42a7-4c3a-a256-11679cc99027 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:23.529315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-136ce6f9-42a7-4c3a-a256-11679cc99027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:23.690979 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 117/224] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:23.691962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ce6f9-42a7-4c3a-a256-11679cc99027 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:23.730219 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0844c98-285a-48fa-a98d-6410f5d1ca9f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:23.774222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6631f1b-4438-44df-90e6-ee4956835e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:23.774980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631f1b-4438-44df-90e6-ee4956835e28 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:23.775277 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631f1b-4438-44df-90e6-ee4956835e28 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:23.775556 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6631f1b-4438-44df-90e6-ee4956835e28 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:23.776391 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6631f1b-4438-44df-90e6-ee4956835e28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:23.776687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6631f1b-4438-44df-90e6-ee4956835e28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:23.778077 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6631f1b-4438-44df-90e6-ee4956835e28 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:23.787835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 108/225] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:23.788846 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6631f1b-4438-44df-90e6-ee4956835e28 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:23.833835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:23.863891 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ce6f9-42a7-4c3a-a256-11679cc99027 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:23.867850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:23.918991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047e62d-7a90-43d9-ab9f-82539094702d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:23.920153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e62d-7a90-43d9-ab9f-82539094702d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:23.920746 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e62d-7a90-43d9-ab9f-82539094702d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:23.921257 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047e62d-7a90-43d9-ab9f-82539094702d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:23.921591 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6631f1b-4438-44df-90e6-ee4956835e28 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:23.922682 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047e62d-7a90-43d9-ab9f-82539094702d None admin] RBAC: Adding query filter params (role.id=2e3b001c16d24c279be5c68aa3847ca7, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2, user.id=c41b01ba1dda49d989972952eeb81619) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:23.923378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047e62d-7a90-43d9-ab9f-82539094702d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:23.925586 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047e62d-7a90-43d9-ab9f-82539094702d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:23.935721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 118/226] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:23 2022] GET /identity/v3/role_assignments?user.id=c41b01ba1dda49d989972952eeb81619&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:23.937001 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047e62d-7a90-43d9-ab9f-82539094702d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:24.018340 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a3e62-9307-4c90-8e00-c9e4fcd8de37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:24.019134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a3e62-9307-4c90-8e00-c9e4fcd8de37 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:24.019500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a3e62-9307-4c90-8e00-c9e4fcd8de37 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:24.019798 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a3e62-9307-4c90-8e00-c9e4fcd8de37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:24.054891 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a3e62-9307-4c90-8e00-c9e4fcd8de37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-5e1a3e62-9307-4c90-8e00-c9e4fcd8de37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:24.066948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:24.068101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:24.069121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 109/227] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:24.077543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a3e62-9307-4c90-8e00-c9e4fcd8de37 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:24.119912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047e62d-7a90-43d9-ab9f-82539094702d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:24.200154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7e103-cf7a-432d-be36-67b538174454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:24.201076 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7e103-cf7a-432d-be36-67b538174454 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:24.201363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7e103-cf7a-432d-be36-67b538174454 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:24.201645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7e103-cf7a-432d-be36-67b538174454 None admin] PATH_INFO: `/v3/users/da341dea30f049cf832df018785674b8` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:24.203929 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7e103-cf7a-432d-be36-67b538174454 None admin] RBAC: Authorizing `identity:get_user(user_id=da341dea30f049cf832df018785674b8)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:24.206576 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7e103-cf7a-432d-be36-67b538174454 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:24.210759 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 119/228] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:23 2022] GET /identity/v3/users/da341dea30f049cf832df018785674b8 => generated 283 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:24.219527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7e103-cf7a-432d-be36-67b538174454 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:24.262522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:24.278260 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:24.290583 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a3e62-9307-4c90-8e00-c9e4fcd8de37 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:24.338310 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7e103-cf7a-432d-be36-67b538174454 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:24.378617 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8bd18-21a8-4cfe-82b4-636075eb9eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:24.379505 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8bd18-21a8-4cfe-82b4-636075eb9eb6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:24.379852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8bd18-21a8-4cfe-82b4-636075eb9eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:24.380246 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e8bd18-21a8-4cfe-82b4-636075eb9eb6 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:24.381983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8bd18-21a8-4cfe-82b4-636075eb9eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:24.384404 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e8bd18-21a8-4cfe-82b4-636075eb9eb6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:24.386579 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 120/229] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:24 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:24.387850 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18070225-391f-496d-9123-52b5b354308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:24.389210 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18070225-391f-496d-9123-52b5b354308b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:24.389210 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18070225-391f-496d-9123-52b5b354308b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:24.389528 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18070225-391f-496d-9123-52b5b354308b None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:24.390494 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18070225-391f-496d-9123-52b5b354308b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:24.390815 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18070225-391f-496d-9123-52b5b354308b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:24.392419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8bd18-21a8-4cfe-82b4-636075eb9eb6 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:24.394310 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18070225-391f-496d-9123-52b5b354308b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:24.426753 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 110/230] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:24.437447 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18070225-391f-496d-9123-52b5b354308b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:24.510739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e8bd18-21a8-4cfe-82b4-636075eb9eb6 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:24.513233 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:24.549749 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f34ea8-5aa1-49ca-9cf6-a507e007027a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:24.550873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f34ea8-5aa1-49ca-9cf6-a507e007027a None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:24.551159 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f34ea8-5aa1-49ca-9cf6-a507e007027a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:24.551441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f34ea8-5aa1-49ca-9cf6-a507e007027a None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/users/da341dea30f049cf832df018785674b8/roles/ed68c2195de147c0a30d9150b33e47cf` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:24.553839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f34ea8-5aa1-49ca-9cf6-a507e007027a None admin] RBAC: Authorizing `identity:create_grant(project_id=7e20e05ff6d243f3ab00b511081c5bf2, user_id=da341dea30f049cf832df018785674b8, role_id=ed68c2195de147c0a30d9150b33e47cf)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:24.555680 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f34ea8-5aa1-49ca-9cf6-a507e007027a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:24.627495 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18070225-391f-496d-9123-52b5b354308b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:24.668527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 121/231] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:49:24 2022] PUT /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/users/da341dea30f049cf832df018785674b8/roles/ed68c2195de147c0a30d9150b33e47cf => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:24.675849 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d3477-6b31-44eb-95fd-d39146b9faf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:24.677475 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d3477-6b31-44eb-95fd-d39146b9faf0 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:24.678088 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d3477-6b31-44eb-95fd-d39146b9faf0 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:24.678561 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d3477-6b31-44eb-95fd-d39146b9faf0 None admin] PATH_INFO: `/v3/users/da341dea30f049cf832df018785674b8` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:24.680236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d3477-6b31-44eb-95fd-d39146b9faf0 None admin] RBAC: Authorizing `identity:get_user(user_id=da341dea30f049cf832df018785674b8)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:24.682216 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d3477-6b31-44eb-95fd-d39146b9faf0 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:24.685107 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 111/232] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:24 2022] GET /identity/v3/users/da341dea30f049cf832df018785674b8 => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:24.691021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f34ea8-5aa1-49ca-9cf6-a507e007027a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:24.851452 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f34ea8-5aa1-49ca-9cf6-a507e007027a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:24.890394 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212c0da-3bb9-4c3a-b5c0-1cd02b2f04e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:24.890394 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212c0da-3bb9-4c3a-b5c0-1cd02b2f04e8 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:24.890394 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212c0da-3bb9-4c3a-b5c0-1cd02b2f04e8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:24.890394 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212c0da-3bb9-4c3a-b5c0-1cd02b2f04e8 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:24.891234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212c0da-3bb9-4c3a-b5c0-1cd02b2f04e8 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:24.893457 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212c0da-3bb9-4c3a-b5c0-1cd02b2f04e8 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:24.900045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 122/233] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:24 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:24.901472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d3477-6b31-44eb-95fd-d39146b9faf0 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:24.940071 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:25.081958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d3477-6b31-44eb-95fd-d39146b9faf0 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:25.150970 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cf8fc-c8d7-4633-84e7-611078733f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:25.153108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cf8fc-c8d7-4633-84e7-611078733f82 None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:25.153957 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cf8fc-c8d7-4633-84e7-611078733f82 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:25.154754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cf8fc-c8d7-4633-84e7-611078733f82 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/users/da341dea30f049cf832df018785674b8/roles/4313b1c9df5345778a249b04355b90a0` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:25.159808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5cf8fc-c8d7-4633-84e7-611078733f82 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e20e05ff6d243f3ab00b511081c5bf2, user_id=da341dea30f049cf832df018785674b8, role_id=4313b1c9df5345778a249b04355b90a0)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:25.163042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5cf8fc-c8d7-4633-84e7-611078733f82 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:25.266544 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 112/234] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:49:24 2022] PUT /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/users/da341dea30f049cf832df018785674b8/roles/4313b1c9df5345778a249b04355b90a0 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:25.842962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e827b-fe40-49f6-b61c-56676ca7680d None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:25.843499 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e827b-fe40-49f6-b61c-56676ca7680d None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:25.843499 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e827b-fe40-49f6-b61c-56676ca7680d None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:25.845021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 123/235] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:25.851261 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8a62-4c40-4c54-b14f-f3567f8e81ae None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:25.851758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8a62-4c40-4c54-b14f-f3567f8e81ae None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:25.851856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8a62-4c40-4c54-b14f-f3567f8e81ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:25.873402 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-61bf8a62-4c40-4c54-b14f-f3567f8e81ae None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:25.874681 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-61bf8a62-4c40-4c54-b14f-f3567f8e81ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:25.964085 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 113/236] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:26.001588 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e827b-fe40-49f6-b61c-56676ca7680d None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:26.018691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627340c-95ce-4576-8480-204d3658cc1b None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:26.019038 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627340c-95ce-4576-8480-204d3658cc1b None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:26.019273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627340c-95ce-4576-8480-204d3658cc1b None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:26.021074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 114/237] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:49:26.027996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483e863-0c88-452b-9f10-30177d9c714a None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:26.028280 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483e863-0c88-452b-9f10-30177d9c714a None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:26.028607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483e863-0c88-452b-9f10-30177d9c714a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:26.049102 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-8483e863-0c88-452b-9f10-30177d9c714a None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:26.050635 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-8483e863-0c88-452b-9f10-30177d9c714a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:26.059910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:26.063517 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:26.134571 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 115/238] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:26.161550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483e863-0c88-452b-9f10-30177d9c714a None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:26.196995 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e827b-fe40-49f6-b61c-56676ca7680d None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:26.286244 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483e863-0c88-452b-9f10-30177d9c714a None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:26.298381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751b331-96bf-44b0-be6b-6fa6904892c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:26.299929 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751b331-96bf-44b0-be6b-6fa6904892c8 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:26.300217 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751b331-96bf-44b0-be6b-6fa6904892c8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:26.300550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751b331-96bf-44b0-be6b-6fa6904892c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:26.301768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751b331-96bf-44b0-be6b-6fa6904892c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:26.303486 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4751b331-96bf-44b0-be6b-6fa6904892c8 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:26.311371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 124/239] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:49:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:26.313453 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751b331-96bf-44b0-be6b-6fa6904892c8 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:26.332035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679e22c-f41d-43ef-93cd-e23fe7291c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:26.333395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679e22c-f41d-43ef-93cd-e23fe7291c23 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:26.333927 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679e22c-f41d-43ef-93cd-e23fe7291c23 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:26.334408 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679e22c-f41d-43ef-93cd-e23fe7291c23 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:26.346368 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679e22c-f41d-43ef-93cd-e23fe7291c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-d679e22c-f41d-43ef-93cd-e23fe7291c23 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:26.355305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:26.356460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:26.357407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:26.360361 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 116/240] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:26.369333 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679e22c-f41d-43ef-93cd-e23fe7291c23 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:26.375442 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:26.504133 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679e22c-f41d-43ef-93cd-e23fe7291c23 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:26.543010 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f87b90-75a9-4eb1-a2a6-1239d38656bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:26.544237 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f87b90-75a9-4eb1-a2a6-1239d38656bc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:26.544747 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f87b90-75a9-4eb1-a2a6-1239d38656bc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:26.545307 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f87b90-75a9-4eb1-a2a6-1239d38656bc None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:26.546415 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f87b90-75a9-4eb1-a2a6-1239d38656bc None admin] RBAC: Adding query filter params (name=reader) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:26.546926 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f87b90-75a9-4eb1-a2a6-1239d38656bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:26.548584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f87b90-75a9-4eb1-a2a6-1239d38656bc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:26.559851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 117/241] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:26.566269 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f87b90-75a9-4eb1-a2a6-1239d38656bc None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:26.608203 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751b331-96bf-44b0-be6b-6fa6904892c8 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:26.612732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:26.692348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9ef56-ebfa-4787-83fe-03c811a5d4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:26.693192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9ef56-ebfa-4787-83fe-03c811a5d4f4 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:26.693646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9ef56-ebfa-4787-83fe-03c811a5d4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:26.694120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9ef56-ebfa-4787-83fe-03c811a5d4f4 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:26.695368 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9ef56-ebfa-4787-83fe-03c811a5d4f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:26.697571 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9ef56-ebfa-4787-83fe-03c811a5d4f4 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:26.708707 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:26.749181 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f87b90-75a9-4eb1-a2a6-1239d38656bc None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:26.785348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db4abe-8278-42d4-b83a-d06256c6ba73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:26.786251 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db4abe-8278-42d4-b83a-d06256c6ba73 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:26.786548 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db4abe-8278-42d4-b83a-d06256c6ba73 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:26.786833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db4abe-8278-42d4-b83a-d06256c6ba73 None admin] PATH_INFO: `/v3/users/761a698fbb914335be3509381a8d46b9` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:26.802276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db4abe-8278-42d4-b83a-d06256c6ba73 None admin] RBAC: Authorizing `identity:get_user(user_id=761a698fbb914335be3509381a8d46b9)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:26.803748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db4abe-8278-42d4-b83a-d06256c6ba73 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:26.805714 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 118/242] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:26 2022] GET /identity/v3/users/761a698fbb914335be3509381a8d46b9 => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:26.806755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a6015-6ea6-439b-ba11-ad952cc292be None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:26.807048 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a6015-6ea6-439b-ba11-ad952cc292be None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:26.807333 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a6015-6ea6-439b-ba11-ad952cc292be None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:26.809045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 119/243] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:26.811018 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a6015-6ea6-439b-ba11-ad952cc292be None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:26.852954 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 125/244] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:49:26 2022] POST /identity/v3/users => generated 293 bytes in 540 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:49:26.855068 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291665e-85eb-45b2-9a12-4b8cb2dfc4ae None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:26.855579 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291665e-85eb-45b2-9a12-4b8cb2dfc4ae None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:26.856123 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291665e-85eb-45b2-9a12-4b8cb2dfc4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:26.885967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-c291665e-85eb-45b2-9a12-4b8cb2dfc4ae None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:26.888932 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-c291665e-85eb-45b2-9a12-4b8cb2dfc4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:26.996212 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 126/245] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:27.001978 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a6015-6ea6-439b-ba11-ad952cc292be None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:27.014547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291665e-85eb-45b2-9a12-4b8cb2dfc4ae None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:27.036686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:27.062546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb4ca6-cc7b-41eb-9ff9-0ad88051ee00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:27.063305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb4ca6-cc7b-41eb-9ff9-0ad88051ee00 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:27.063688 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb4ca6-cc7b-41eb-9ff9-0ad88051ee00 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:27.063819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb4ca6-cc7b-41eb-9ff9-0ad88051ee00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:27.146339 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:27.151501 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb4ca6-cc7b-41eb-9ff9-0ad88051ee00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:27.153037 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb4ca6-cc7b-41eb-9ff9-0ad88051ee00 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:27.165028 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291665e-85eb-45b2-9a12-4b8cb2dfc4ae None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:27.204104 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022f328-3e20-49b5-a7bd-924921db8b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:27.206042 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022f328-3e20-49b5-a7bd-924921db8b62 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:27.211551 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022f328-3e20-49b5-a7bd-924921db8b62 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:27.211551 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022f328-3e20-49b5-a7bd-924921db8b62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:27.218962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022f328-3e20-49b5-a7bd-924921db8b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-3022f328-3e20-49b5-a7bd-924921db8b62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:27.228664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:27.229864 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:27.230798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:27.233844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 127/246] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:27.239976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022f328-3e20-49b5-a7bd-924921db8b62 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:27.265613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 120/247] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:49:27.271740 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb4ca6-cc7b-41eb-9ff9-0ad88051ee00 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:27.302905 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:27.374092 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022f328-3e20-49b5-a7bd-924921db8b62 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:27.401864 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb4ca6-cc7b-41eb-9ff9-0ad88051ee00 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:27.417561 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1a50a-2bf0-4222-bd06-6dd397657175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:27.418719 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1a50a-2bf0-4222-bd06-6dd397657175 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:27.419229 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1a50a-2bf0-4222-bd06-6dd397657175 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:27.419759 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1a50a-2bf0-4222-bd06-6dd397657175 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:27.420887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1a50a-2bf0-4222-bd06-6dd397657175 None admin] RBAC: Adding query filter params (name=member) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:27.421575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1a50a-2bf0-4222-bd06-6dd397657175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:27.423461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1a50a-2bf0-4222-bd06-6dd397657175 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:27.434406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 128/248] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:27.440771 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290e46b-cd42-4d84-88c0-ccc44a700011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:27.441300 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1a50a-2bf0-4222-bd06-6dd397657175 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:27.441653 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290e46b-cd42-4d84-88c0-ccc44a700011 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:27.441950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290e46b-cd42-4d84-88c0-ccc44a700011 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:27.442234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290e46b-cd42-4d84-88c0-ccc44a700011 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:27.443453 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290e46b-cd42-4d84-88c0-ccc44a700011 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:27.445104 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290e46b-cd42-4d84-88c0-ccc44a700011 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:27.447464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 121/249] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:27 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:27.455832 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290e46b-cd42-4d84-88c0-ccc44a700011 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:27.464645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:27.574005 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:27.584255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290e46b-cd42-4d84-88c0-ccc44a700011 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:27.598789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1a50a-2bf0-4222-bd06-6dd397657175 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:27.627570 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282bba97-9430-412f-be56-82fd905b6881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:27.628360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bba97-9430-412f-be56-82fd905b6881 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:27.628581 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bba97-9430-412f-be56-82fd905b6881 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:27.628851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282bba97-9430-412f-be56-82fd905b6881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:27.629691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bba97-9430-412f-be56-82fd905b6881 None admin] RBAC: Adding query filter params (role.id=83321c42fce7438aa5e29c1acb4f07af, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2, user.id=761a698fbb914335be3509381a8d46b9) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:27.630001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bba97-9430-412f-be56-82fd905b6881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:27.631437 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282bba97-9430-412f-be56-82fd905b6881 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:27.638710 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 122/250] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:27 2022] GET /identity/v3/role_assignments?user.id=761a698fbb914335be3509381a8d46b9&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=83321c42fce7438aa5e29c1acb4f07af => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:27.640045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428bccb-ab71-4d37-b5be-6570dbc3d578 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:27.640357 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428bccb-ab71-4d37-b5be-6570dbc3d578 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:27.640626 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6428bccb-ab71-4d37-b5be-6570dbc3d578 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:27.642096 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 123/251] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:27.646830 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c2cc-b563-425a-a395-8fc4f8e22829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:27.647730 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74054a-1c70-4219-9e00-3d90097661c9 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:27.648026 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74054a-1c70-4219-9e00-3d90097661c9 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:27.648310 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74054a-1c70-4219-9e00-3d90097661c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:27.648951 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c2cc-b563-425a-a395-8fc4f8e22829 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:27.649473 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c2cc-b563-425a-a395-8fc4f8e22829 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:27.649994 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0c2cc-b563-425a-a395-8fc4f8e22829 None admin] PATH_INFO: `/v3/users/c41b01ba1dda49d989972952eeb81619` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:27.651785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0c2cc-b563-425a-a395-8fc4f8e22829 None admin] RBAC: Authorizing `identity:get_user(user_id=c41b01ba1dda49d989972952eeb81619)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:27.654087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0c2cc-b563-425a-a395-8fc4f8e22829 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:27.661924 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 129/252] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:27 2022] GET /identity/v3/users/c41b01ba1dda49d989972952eeb81619 => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:27.665476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c2cc-b563-425a-a395-8fc4f8e22829 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:27.668275 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-6c74054a-1c70-4219-9e00-3d90097661c9 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:27.671129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-6c74054a-1c70-4219-9e00-3d90097661c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:27.778601 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 124/253] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:27.790778 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:27.818034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0c2cc-b563-425a-a395-8fc4f8e22829 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:27.869999 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9362ad9-e2a0-45a5-856b-8e6b9a5cbca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:27.870835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9362ad9-e2a0-45a5-856b-8e6b9a5cbca6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:27.871121 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9362ad9-e2a0-45a5-856b-8e6b9a5cbca6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:27.871401 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9362ad9-e2a0-45a5-856b-8e6b9a5cbca6 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:27.872696 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9362ad9-e2a0-45a5-856b-8e6b9a5cbca6 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:27.874392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9362ad9-e2a0-45a5-856b-8e6b9a5cbca6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:27.876533 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 130/254] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:27 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:27.881188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74054a-1c70-4219-9e00-3d90097661c9 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:27.900869 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9362ad9-e2a0-45a5-856b-8e6b9a5cbca6 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:28.080611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:28.106321 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74054a-1c70-4219-9e00-3d90097661c9 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:28.110642 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9362ad9-e2a0-45a5-856b-8e6b9a5cbca6 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:28.174930 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a96804-e287-4a4d-8366-02f3438d9b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:28.175767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96804-e287-4a4d-8366-02f3438d9b9e None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:28.176057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96804-e287-4a4d-8366-02f3438d9b9e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:28.176363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a96804-e287-4a4d-8366-02f3438d9b9e None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/users/c41b01ba1dda49d989972952eeb81619/roles/2e3b001c16d24c279be5c68aa3847ca7` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:28.179287 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a96804-e287-4a4d-8366-02f3438d9b9e None admin] RBAC: Authorizing `identity:create_grant(project_id=7e20e05ff6d243f3ab00b511081c5bf2, user_id=c41b01ba1dda49d989972952eeb81619, role_id=2e3b001c16d24c279be5c68aa3847ca7)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:28.180842 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a96804-e287-4a4d-8366-02f3438d9b9e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:28.194045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c30d68-5795-4a25-ab19-d94ba9a4addd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:28.194943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c30d68-5795-4a25-ab19-d94ba9a4addd None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:28.195236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c30d68-5795-4a25-ab19-d94ba9a4addd None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:28.195571 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c30d68-5795-4a25-ab19-d94ba9a4addd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:28.214847 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c30d68-5795-4a25-ab19-d94ba9a4addd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-a5c30d68-5795-4a25-ab19-d94ba9a4addd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:28.222801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:28.228822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 125/255] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:49:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:28.231125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c30d68-5795-4a25-ab19-d94ba9a4addd None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:28.278981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 131/256] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:49:27 2022] PUT /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/users/c41b01ba1dda49d989972952eeb81619/roles/2e3b001c16d24c279be5c68aa3847ca7 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:49:28.403730 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c30d68-5795-4a25-ab19-d94ba9a4addd None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:28.414392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990de8-2650-40e2-849a-70c9cb48538a None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:28.414709 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990de8-2650-40e2-849a-70c9cb48538a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:28.414987 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b990de8-2650-40e2-849a-70c9cb48538a None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:28.416534 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 132/257] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:49:28.426725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b790d-bf3f-4c33-94e6-47bcf98d6f3c None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:28.427341 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b790d-bf3f-4c33-94e6-47bcf98d6f3c None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:28.427837 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b790d-bf3f-4c33-94e6-47bcf98d6f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:28.436060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:28.448619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-529b790d-bf3f-4c33-94e6-47bcf98d6f3c None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:28.450933 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-529b790d-bf3f-4c33-94e6-47bcf98d6f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:28.469315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0260fab-fa77-4b0b-bd92-c089927a70f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:28.470375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0260fab-fa77-4b0b-bd92-c089927a70f2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:28.471422 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0260fab-fa77-4b0b-bd92-c089927a70f2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:28.471949 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0260fab-fa77-4b0b-bd92-c089927a70f2 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:28.472983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0260fab-fa77-4b0b-bd92-c089927a70f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:28.475324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0260fab-fa77-4b0b-bd92-c089927a70f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:28.477126 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0260fab-fa77-4b0b-bd92-c089927a70f2 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:28.500362 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 126/258] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:28.509197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0260fab-fa77-4b0b-bd92-c089927a70f2 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:28.532780 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:28.599011 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 133/259] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:28.662316 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b790d-bf3f-4c33-94e6-47bcf98d6f3c None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:28.715281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0260fab-fa77-4b0b-bd92-c089927a70f2 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:28.773116 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e203bc3-35e4-4d41-9271-57a1412737b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:28.774492 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e203bc3-35e4-4d41-9271-57a1412737b8 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:28.774942 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e203bc3-35e4-4d41-9271-57a1412737b8 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:28.775232 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e203bc3-35e4-4d41-9271-57a1412737b8 None admin] PATH_INFO: `/v3/users/da341dea30f049cf832df018785674b8` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:28.776643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e203bc3-35e4-4d41-9271-57a1412737b8 None admin] RBAC: Authorizing `identity:get_user(user_id=da341dea30f049cf832df018785674b8)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:28.778463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e203bc3-35e4-4d41-9271-57a1412737b8 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:28.780701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 127/260] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:28 2022] GET /identity/v3/users/da341dea30f049cf832df018785674b8 => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:28.795367 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e203bc3-35e4-4d41-9271-57a1412737b8 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:28.923343 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b790d-bf3f-4c33-94e6-47bcf98d6f3c None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:28.944123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:29.006566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ebcc8-fc02-4fbe-a3a4-f64de9292c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:29.006909 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e203bc3-35e4-4d41-9271-57a1412737b8 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:29.007701 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ebcc8-fc02-4fbe-a3a4-f64de9292c44 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:29.008122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ebcc8-fc02-4fbe-a3a4-f64de9292c44 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:29.008556 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ebcc8-fc02-4fbe-a3a4-f64de9292c44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:29.014616 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:29.031368 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46ebcc8-fc02-4fbe-a3a4-f64de9292c44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-e46ebcc8-fc02-4fbe-a3a4-f64de9292c44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:29.040240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:29.041461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:29.042981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:29.050453 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 134/261] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:49:29.056146 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ebcc8-fc02-4fbe-a3a4-f64de9292c44 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:29.117091 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0c949-5a48-4c6e-96ec-4a6edea492b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:29.117768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0c949-5a48-4c6e-96ec-4a6edea492b2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:29.118064 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0c949-5a48-4c6e-96ec-4a6edea492b2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:29.118351 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0c949-5a48-4c6e-96ec-4a6edea492b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:29.197020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ebcc8-fc02-4fbe-a3a4-f64de9292c44 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:29.232600 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23801b-2f61-4ef7-a1e8-84862578ecb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:29.233282 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23801b-2f61-4ef7-a1e8-84862578ecb8 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:29.233570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23801b-2f61-4ef7-a1e8-84862578ecb8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:29.233849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23801b-2f61-4ef7-a1e8-84862578ecb8 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:29.234619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23801b-2f61-4ef7-a1e8-84862578ecb8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:29.234965 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23801b-2f61-4ef7-a1e8-84862578ecb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:29.236448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23801b-2f61-4ef7-a1e8-84862578ecb8 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:29.245863 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 135/262] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:29.251519 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23801b-2f61-4ef7-a1e8-84862578ecb8 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:29.317563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0c949-5a48-4c6e-96ec-4a6edea492b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:29.319305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0c949-5a48-4c6e-96ec-4a6edea492b2 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:29.321633 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:29.362113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23801b-2f61-4ef7-a1e8-84862578ecb8 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:29.421872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da56f3-db42-484d-8dc5-4e102d8c28ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:29.422984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da56f3-db42-484d-8dc5-4e102d8c28ed None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:29.424613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da56f3-db42-484d-8dc5-4e102d8c28ed None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:29.425065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da56f3-db42-484d-8dc5-4e102d8c28ed None admin] PATH_INFO: `/v3/users/da341dea30f049cf832df018785674b8` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:29.426689 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da56f3-db42-484d-8dc5-4e102d8c28ed None admin] RBAC: Authorizing `identity:get_user(user_id=da341dea30f049cf832df018785674b8)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:29.428185 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da56f3-db42-484d-8dc5-4e102d8c28ed None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:29.438782 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 136/263] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:29 2022] GET /identity/v3/users/da341dea30f049cf832df018785674b8 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:29.442371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da56f3-db42-484d-8dc5-4e102d8c28ed None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:29.488737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:29.530626 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 128/264] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:29.541702 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0c949-5a48-4c6e-96ec-4a6edea492b2 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:29.573479 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da56f3-db42-484d-8dc5-4e102d8c28ed None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:29.628114 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:29.646326 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0f3a1-2b14-4457-8c93-5601c8e023c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:29.648850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0f3a1-2b14-4457-8c93-5601c8e023c3 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:29.648850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0f3a1-2b14-4457-8c93-5601c8e023c3 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:29.648850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0f3a1-2b14-4457-8c93-5601c8e023c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:29.700143 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0c949-5a48-4c6e-96ec-4a6edea492b2 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:29.769008 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50b2f6-ed24-4ad3-a750-8544bd162713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:29.769895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b2f6-ed24-4ad3-a750-8544bd162713 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:29.770182 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b2f6-ed24-4ad3-a750-8544bd162713 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:29.770454 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd50b2f6-ed24-4ad3-a750-8544bd162713 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:29.772058 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50b2f6-ed24-4ad3-a750-8544bd162713 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:29.773535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd50b2f6-ed24-4ad3-a750-8544bd162713 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:29.775262 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 129/265] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:29 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:29.785862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50b2f6-ed24-4ad3-a750-8544bd162713 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:29.793192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0f3a1-2b14-4457-8c93-5601c8e023c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:29.794705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0f3a1-2b14-4457-8c93-5601c8e023c3 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:29.864870 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:29.911038 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd50b2f6-ed24-4ad3-a750-8544bd162713 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:29.951480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562beeee-abc8-4035-b120-51ca3eed5160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:29.952160 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562beeee-abc8-4035-b120-51ca3eed5160 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:29.952476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562beeee-abc8-4035-b120-51ca3eed5160 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:29.952884 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562beeee-abc8-4035-b120-51ca3eed5160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:29.954219 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562beeee-abc8-4035-b120-51ca3eed5160 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2, user.id=da341dea30f049cf832df018785674b8) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:29.954532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562beeee-abc8-4035-b120-51ca3eed5160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:29.956030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562beeee-abc8-4035-b120-51ca3eed5160 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:29.967172 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 130/266] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:29 2022] GET /identity/v3/role_assignments?user.id=da341dea30f049cf832df018785674b8&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:30.043949 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 137/267] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:30.050441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562beeee-abc8-4035-b120-51ca3eed5160 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:30.190025 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562beeee-abc8-4035-b120-51ca3eed5160 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:30.199779 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:30.227119 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c13c8c-4b30-4b5b-9c26-e6c14a2b8df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:30.228002 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c13c8c-4b30-4b5b-9c26-e6c14a2b8df3 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:30.228286 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c13c8c-4b30-4b5b-9c26-e6c14a2b8df3 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:30.228592 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c13c8c-4b30-4b5b-9c26-e6c14a2b8df3 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:30.229905 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c13c8c-4b30-4b5b-9c26-e6c14a2b8df3 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:30.231358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c13c8c-4b30-4b5b-9c26-e6c14a2b8df3 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:30.233479 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 131/268] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:30 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:30.239048 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0f3a1-2b14-4457-8c93-5601c8e023c3 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:30.285871 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:30.397136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0f3a1-2b14-4457-8c93-5601c8e023c3 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:30.452471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf002a-a032-4e0b-8bb9-6d48cd6a43e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:30.453065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf002a-a032-4e0b-8bb9-6d48cd6a43e2 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:30.453353 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf002a-a032-4e0b-8bb9-6d48cd6a43e2 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:30.453640 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf002a-a032-4e0b-8bb9-6d48cd6a43e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:30.454500 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf002a-a032-4e0b-8bb9-6d48cd6a43e2 None admin] RBAC: Adding query filter params (role.id=4313b1c9df5345778a249b04355b90a0, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2, user.id=da341dea30f049cf832df018785674b8) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:30.454821 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf002a-a032-4e0b-8bb9-6d48cd6a43e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:30.456921 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf002a-a032-4e0b-8bb9-6d48cd6a43e2 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:30.467953 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 138/269] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:30 2022] GET /identity/v3/role_assignments?user.id=da341dea30f049cf832df018785674b8&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=4313b1c9df5345778a249b04355b90a0 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:30.577877 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e63de-6ad0-48cf-8b6a-358ec3455c27 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:30.578550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e63de-6ad0-48cf-8b6a-358ec3455c27 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:30.579058 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e63de-6ad0-48cf-8b6a-358ec3455c27 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:30.581650 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 132/270] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:30.587039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7fc21-f4e4-4fb5-b12c-64ff0a277951 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:30.587348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7fc21-f4e4-4fb5-b12c-64ff0a277951 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:30.587615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7fc21-f4e4-4fb5-b12c-64ff0a277951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:30.605661 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-c6e7fc21-f4e4-4fb5-b12c-64ff0a277951 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:30.606894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-c6e7fc21-f4e4-4fb5-b12c-64ff0a277951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:30.701161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 139/271] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:30.745212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e63de-6ad0-48cf-8b6a-358ec3455c27 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:30.867958 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fdaf7-0b9e-4983-8b88-29e52ce3ca6e None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:30.868199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e63de-6ad0-48cf-8b6a-358ec3455c27 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:30.868300 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fdaf7-0b9e-4983-8b88-29e52ce3ca6e None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:30.868622 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0fdaf7-0b9e-4983-8b88-29e52ce3ca6e None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:30.870256 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 140/272] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:30.876036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ada029-b926-46ba-90c2-40daa12da359 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:30.876328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ada029-b926-46ba-90c2-40daa12da359 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:30.876644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ada029-b926-46ba-90c2-40daa12da359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:30.896049 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-d3ada029-b926-46ba-90c2-40daa12da359 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:30.897401 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-d3ada029-b926-46ba-90c2-40daa12da359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:30.905681 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccb3ff-6f28-4d03-80e2-663e1471f754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:30.906586 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccb3ff-6f28-4d03-80e2-663e1471f754 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:30.906882 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccb3ff-6f28-4d03-80e2-663e1471f754 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:30.907165 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ccb3ff-6f28-4d03-80e2-663e1471f754 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:30.922565 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ccb3ff-6f28-4d03-80e2-663e1471f754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:30.924536 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-f5ccb3ff-6f28-4d03-80e2-663e1471f754 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:30.945030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:30.946425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 133/273] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:30.947487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccb3ff-6f28-4d03-80e2-663e1471f754 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:31.024478 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 141/274] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:31.045679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ada029-b926-46ba-90c2-40daa12da359 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:31.050545 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:31.123729 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ccb3ff-6f28-4d03-80e2-663e1471f754 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:31.161860 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2884a1-1a23-46e7-b729-b384d4a13bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:31.162551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2884a1-1a23-46e7-b729-b384d4a13bf2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.162839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2884a1-1a23-46e7-b729-b384d4a13bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.163161 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2884a1-1a23-46e7-b729-b384d4a13bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.163882 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2884a1-1a23-46e7-b729-b384d4a13bf2 None admin] RBAC: Adding query filter params (name=member) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:31.164196 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2884a1-1a23-46e7-b729-b384d4a13bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:31.165844 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2884a1-1a23-46e7-b729-b384d4a13bf2 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:31.176356 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 134/275] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:31.182129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2884a1-1a23-46e7-b729-b384d4a13bf2 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:31.237138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ada029-b926-46ba-90c2-40daa12da359 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:31.286486 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa560362-be15-44b0-9294-6c5a345a5126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:31.288286 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa560362-be15-44b0-9294-6c5a345a5126 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.289182 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa560362-be15-44b0-9294-6c5a345a5126 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.289469 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa560362-be15-44b0-9294-6c5a345a5126 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.303087 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:31.307752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2884a1-1a23-46e7-b729-b384d4a13bf2 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:31.311251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa560362-be15-44b0-9294-6c5a345a5126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-fa560362-be15-44b0-9294-6c5a345a5126 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:31.319524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:31.320659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:31.321636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:31.326098 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 142/276] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:31.327460 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b798e6a-9704-4813-8c29-bd3774b5be2a None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.327757 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b798e6a-9704-4813-8c29-bd3774b5be2a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.328102 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b798e6a-9704-4813-8c29-bd3774b5be2a None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.332469 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 143/277] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:31.332469 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b798e6a-9704-4813-8c29-bd3774b5be2a None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:31.348958 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:31.360097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d348d-41f6-4019-aa30-aef527ae43df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:31.362480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d348d-41f6-4019-aa30-aef527ae43df None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.362480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d348d-41f6-4019-aa30-aef527ae43df None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.362480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d348d-41f6-4019-aa30-aef527ae43df None admin] PATH_INFO: `/v3/users/0a1778648de841b9aac1dd4d339a7601` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.383582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d348d-41f6-4019-aa30-aef527ae43df None admin] RBAC: Authorizing `identity:get_user(user_id=0a1778648de841b9aac1dd4d339a7601)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:31.385628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d348d-41f6-4019-aa30-aef527ae43df None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:31.392821 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 135/278] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:31 2022] GET /identity/v3/users/0a1778648de841b9aac1dd4d339a7601 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:31.392821 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6346ce-03f4-474e-93bf-6da782fb73f3 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.392821 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6346ce-03f4-474e-93bf-6da782fb73f3 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.392821 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6346ce-03f4-474e-93bf-6da782fb73f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.413178 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-ce6346ce-03f4-474e-93bf-6da782fb73f3 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:31.418074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-ce6346ce-03f4-474e-93bf-6da782fb73f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:31.476757 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b798e6a-9704-4813-8c29-bd3774b5be2a None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:31.512593 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 136/279] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:31.512593 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6346ce-03f4-474e-93bf-6da782fb73f3 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:31.528891 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc148956-cca2-4078-90fa-018e65211df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:31.529755 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc148956-cca2-4078-90fa-018e65211df5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.530146 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc148956-cca2-4078-90fa-018e65211df5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.530589 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc148956-cca2-4078-90fa-018e65211df5 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.531600 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc148956-cca2-4078-90fa-018e65211df5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:31.532016 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc148956-cca2-4078-90fa-018e65211df5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:31.534131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc148956-cca2-4078-90fa-018e65211df5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:31.544255 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 144/280] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:31.545284 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc148956-cca2-4078-90fa-018e65211df5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:31.623690 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:31.630788 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:31.638881 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6346ce-03f4-474e-93bf-6da782fb73f3 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:31.663785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc148956-cca2-4078-90fa-018e65211df5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:31.675234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9c3a0-3ba8-4f95-8f9c-ea3c467f46a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:31.675923 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9c3a0-3ba8-4f95-8f9c-ea3c467f46a7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.676194 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9c3a0-3ba8-4f95-8f9c-ea3c467f46a7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.676516 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9c3a0-3ba8-4f95-8f9c-ea3c467f46a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.677361 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9c3a0-3ba8-4f95-8f9c-ea3c467f46a7 None admin] RBAC: Adding query filter params (role.id=2e3b001c16d24c279be5c68aa3847ca7, user.id=0a1778648de841b9aac1dd4d339a7601, scope.system=all) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:31.677681 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9c3a0-3ba8-4f95-8f9c-ea3c467f46a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:31.679130 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a9c3a0-3ba8-4f95-8f9c-ea3c467f46a7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:31.689730 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 137/281] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:49:31 2022] GET /identity/v3/role_assignments?user.id=0a1778648de841b9aac1dd4d339a7601&scope.system=all&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:31.689730 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9c3a0-3ba8-4f95-8f9c-ea3c467f46a7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:31.705900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406395ed-b460-4f00-b689-c896a4683d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:31.707376 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406395ed-b460-4f00-b689-c896a4683d01 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.708040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406395ed-b460-4f00-b689-c896a4683d01 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.708359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406395ed-b460-4f00-b689-c896a4683d01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.709886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406395ed-b460-4f00-b689-c896a4683d01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:31.711488 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406395ed-b460-4f00-b689-c896a4683d01 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:31.714153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 145/282] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:49:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:31.715806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e1ffc-a9ba-4ce2-b37c-46981d89c8ab None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.716170 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e1ffc-a9ba-4ce2-b37c-46981d89c8ab None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.716547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e1ffc-a9ba-4ce2-b37c-46981d89c8ab None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.720418 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 146/283] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:31.721914 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e1ffc-a9ba-4ce2-b37c-46981d89c8ab None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:31.819155 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9c3a0-3ba8-4f95-8f9c-ea3c467f46a7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:31.848564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e1ffc-a9ba-4ce2-b37c-46981d89c8ab None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:31.862345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d123a-3b1b-4999-86a8-61ca34531a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:31.863611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d123a-3b1b-4999-86a8-61ca34531a43 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.864114 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d123a-3b1b-4999-86a8-61ca34531a43 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.864625 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d123a-3b1b-4999-86a8-61ca34531a43 None admin] PATH_INFO: `/v3/users/761a698fbb914335be3509381a8d46b9` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.866183 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d123a-3b1b-4999-86a8-61ca34531a43 None admin] RBAC: Authorizing `identity:get_user(user_id=761a698fbb914335be3509381a8d46b9)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:31.867953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d123a-3b1b-4999-86a8-61ca34531a43 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:31.869877 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 138/284] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:31 2022] GET /identity/v3/users/761a698fbb914335be3509381a8d46b9 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:31.871400 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df67424-1306-494b-a1f2-54205c581afb None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.872255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df67424-1306-494b-a1f2-54205c581afb None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.872255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df67424-1306-494b-a1f2-54205c581afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.890251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c3e26-84b3-4d2b-a80a-05b72a0b2b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:31.890508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-9df67424-1306-494b-a1f2-54205c581afb None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:31.890931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c3e26-84b3-4d2b-a80a-05b72a0b2b1a None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:31.891217 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c3e26-84b3-4d2b-a80a-05b72a0b2b1a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:31.891493 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c3e26-84b3-4d2b-a80a-05b72a0b2b1a None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:31.891603 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-9df67424-1306-494b-a1f2-54205c581afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:31.892209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c3e26-84b3-4d2b-a80a-05b72a0b2b1a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:31.894293 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c3e26-84b3-4d2b-a80a-05b72a0b2b1a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:31.904298 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:31.908801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:31.965973 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 139/285] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:31.966865 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df67424-1306-494b-a1f2-54205c581afb None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:32.049666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 147/286] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:49:31 2022] POST /identity/v3/users => generated 293 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:49:32.051259 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c3e26-84b3-4d2b-a80a-05b72a0b2b1a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:32.083298 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df67424-1306-494b-a1f2-54205c581afb None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:32.135598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc666800-7354-41b1-bdbf-638f0d0991be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:32.136701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc666800-7354-41b1-bdbf-638f0d0991be None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:32.136986 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc666800-7354-41b1-bdbf-638f0d0991be None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:32.137263 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc666800-7354-41b1-bdbf-638f0d0991be None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:32.139229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc666800-7354-41b1-bdbf-638f0d0991be None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:32.141283 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc666800-7354-41b1-bdbf-638f0d0991be None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:32.144636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 140/287] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:31 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:32.150776 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc666800-7354-41b1-bdbf-638f0d0991be None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:32.175644 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:32.193247 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c3e26-84b3-4d2b-a80a-05b72a0b2b1a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:32.236030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999082f8-3560-4a17-9e32-0ac8d9830f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:32.237631 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999082f8-3560-4a17-9e32-0ac8d9830f66 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:32.238168 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999082f8-3560-4a17-9e32-0ac8d9830f66 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:32.238652 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999082f8-3560-4a17-9e32-0ac8d9830f66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:32.253164 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999082f8-3560-4a17-9e32-0ac8d9830f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-999082f8-3560-4a17-9e32-0ac8d9830f66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:32.263570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:32.264727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:32.265662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:32.269032 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 148/288] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:32.275283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999082f8-3560-4a17-9e32-0ac8d9830f66 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:32.287454 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc666800-7354-41b1-bdbf-638f0d0991be None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:32.335905 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f58bbc-e7bf-4514-ad90-094dfd2ee052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:32.336821 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f58bbc-e7bf-4514-ad90-094dfd2ee052 None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:32.337071 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f58bbc-e7bf-4514-ad90-094dfd2ee052 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:32.337348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f58bbc-e7bf-4514-ad90-094dfd2ee052 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/users/761a698fbb914335be3509381a8d46b9/roles/83321c42fce7438aa5e29c1acb4f07af` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:32.341457 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f58bbc-e7bf-4514-ad90-094dfd2ee052 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e20e05ff6d243f3ab00b511081c5bf2, user_id=761a698fbb914335be3509381a8d46b9, role_id=83321c42fce7438aa5e29c1acb4f07af)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:32.342935 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f58bbc-e7bf-4514-ad90-094dfd2ee052 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:32.358459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:32.411417 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999082f8-3560-4a17-9e32-0ac8d9830f66 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:32.445725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 141/289] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:49:32 2022] PUT /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/users/761a698fbb914335be3509381a8d46b9/roles/83321c42fce7438aa5e29c1acb4f07af => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:49:32.450866 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58266583-a877-4387-ba8c-2fe70ed5b13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:32.451568 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58266583-a877-4387-ba8c-2fe70ed5b13a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:32.451860 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58266583-a877-4387-ba8c-2fe70ed5b13a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:32.452141 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58266583-a877-4387-ba8c-2fe70ed5b13a None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:32.453648 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58266583-a877-4387-ba8c-2fe70ed5b13a None admin] RBAC: Adding query filter params (name=member) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:32.453648 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58266583-a877-4387-ba8c-2fe70ed5b13a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:32.454758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58266583-a877-4387-ba8c-2fe70ed5b13a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:32.465473 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 149/290] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:32.471302 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f58bbc-e7bf-4514-ad90-094dfd2ee052 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:32.563112 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:32.590964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f58bbc-e7bf-4514-ad90-094dfd2ee052 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:32.640961 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c019e71-ddd0-420c-aebc-395ca301387b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:32.641857 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c019e71-ddd0-420c-aebc-395ca301387b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:32.642143 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c019e71-ddd0-420c-aebc-395ca301387b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:32.643086 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c019e71-ddd0-420c-aebc-395ca301387b None admin] PATH_INFO: `/v3/users/c41b01ba1dda49d989972952eeb81619` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:32.649391 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c019e71-ddd0-420c-aebc-395ca301387b None admin] RBAC: Authorizing `identity:get_user(user_id=c41b01ba1dda49d989972952eeb81619)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:32.650956 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c019e71-ddd0-420c-aebc-395ca301387b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:32.653001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 142/291] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:32 2022] GET /identity/v3/users/c41b01ba1dda49d989972952eeb81619 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:32.659562 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58266583-a877-4387-ba8c-2fe70ed5b13a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:32.787957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58266583-a877-4387-ba8c-2fe70ed5b13a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:32.805687 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:32.825880 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94e5ad-1ed8-4b60-9070-2236bbbd4909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:32.826608 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94e5ad-1ed8-4b60-9070-2236bbbd4909 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:32.826911 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94e5ad-1ed8-4b60-9070-2236bbbd4909 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:32.827194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94e5ad-1ed8-4b60-9070-2236bbbd4909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:32.909629 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94e5ad-1ed8-4b60-9070-2236bbbd4909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:32.911139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94e5ad-1ed8-4b60-9070-2236bbbd4909 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:33.024397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 150/292] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:33.030426 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c019e71-ddd0-420c-aebc-395ca301387b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:33.161640 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c019e71-ddd0-420c-aebc-395ca301387b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:33.203720 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57d4a6-98bb-4a7a-baf8-9f95f96e5b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:33.204618 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57d4a6-98bb-4a7a-baf8-9f95f96e5b08 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:33.204932 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57d4a6-98bb-4a7a-baf8-9f95f96e5b08 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:33.205210 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57d4a6-98bb-4a7a-baf8-9f95f96e5b08 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:33.206429 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57d4a6-98bb-4a7a-baf8-9f95f96e5b08 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:33.207882 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57d4a6-98bb-4a7a-baf8-9f95f96e5b08 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:33.209862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 143/293] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:33 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:33.218728 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94e5ad-1ed8-4b60-9070-2236bbbd4909 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:33.251444 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:33.374932 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94e5ad-1ed8-4b60-9070-2236bbbd4909 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:33.414349 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f41af-15fa-4bee-9dc9-3b6c1a51066c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:33.414933 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f41af-15fa-4bee-9dc9-3b6c1a51066c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:33.415222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f41af-15fa-4bee-9dc9-3b6c1a51066c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:33.415500 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f41af-15fa-4bee-9dc9-3b6c1a51066c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:33.416370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f41af-15fa-4bee-9dc9-3b6c1a51066c None admin] RBAC: Adding query filter params (role.id=2e3b001c16d24c279be5c68aa3847ca7, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2, user.id=c41b01ba1dda49d989972952eeb81619) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:33.416694 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f41af-15fa-4bee-9dc9-3b6c1a51066c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:33.418245 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f41af-15fa-4bee-9dc9-3b6c1a51066c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:33.429587 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 151/294] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:33 2022] GET /identity/v3/role_assignments?user.id=c41b01ba1dda49d989972952eeb81619&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:34.261358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c6448-15d4-4337-944c-cedc5bab9a06 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:34.262391 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c6448-15d4-4337-944c-cedc5bab9a06 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:34.262900 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c6448-15d4-4337-944c-cedc5bab9a06 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:34.264862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 144/295] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:34.270966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7751a-5535-460b-8163-06c1258e4bf5 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:34.271867 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7751a-5535-460b-8163-06c1258e4bf5 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:34.272385 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7751a-5535-460b-8163-06c1258e4bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:34.290813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-82f7751a-5535-460b-8163-06c1258e4bf5 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:34.292405 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-82f7751a-5535-460b-8163-06c1258e4bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:34.372455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 152/296] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:34.389306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c6448-15d4-4337-944c-cedc5bab9a06 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:34.405492 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:34.508683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c6448-15d4-4337-944c-cedc5bab9a06 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:34.545238 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804e4e9-280c-44f0-a317-35a01f7c1c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:34.546044 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804e4e9-280c-44f0-a317-35a01f7c1c31 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:34.546333 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804e4e9-280c-44f0-a317-35a01f7c1c31 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:34.546614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804e4e9-280c-44f0-a317-35a01f7c1c31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:34.558524 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804e4e9-280c-44f0-a317-35a01f7c1c31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-3804e4e9-280c-44f0-a317-35a01f7c1c31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:34.566375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:34.567496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:34.568989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 145/297] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:34.576012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7751a-5535-460b-8163-06c1258e4bf5 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:34.591851 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:34.685584 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7751a-5535-460b-8163-06c1258e4bf5 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:34.729347 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93adc3-5176-43d6-bd04-94abad2fe85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:34.730459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93adc3-5176-43d6-bd04-94abad2fe85d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:34.730994 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93adc3-5176-43d6-bd04-94abad2fe85d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:34.731495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93adc3-5176-43d6-bd04-94abad2fe85d None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:34.732539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93adc3-5176-43d6-bd04-94abad2fe85d None admin] RBAC: Adding query filter params (name=member) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:34.733092 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93adc3-5176-43d6-bd04-94abad2fe85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:34.734931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93adc3-5176-43d6-bd04-94abad2fe85d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:34.750105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 153/298] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:34.752064 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804e4e9-280c-44f0-a317-35a01f7c1c31 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:34.836000 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:34.868905 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804e4e9-280c-44f0-a317-35a01f7c1c31 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:34.909093 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab136c3a-a298-4513-806b-62f62283883c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:34.910013 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab136c3a-a298-4513-806b-62f62283883c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:34.910299 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab136c3a-a298-4513-806b-62f62283883c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:34.910582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab136c3a-a298-4513-806b-62f62283883c None admin] PATH_INFO: `/v3/users/0a1778648de841b9aac1dd4d339a7601` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:34.911979 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab136c3a-a298-4513-806b-62f62283883c None admin] RBAC: Authorizing `identity:get_user(user_id=0a1778648de841b9aac1dd4d339a7601)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:34.913529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab136c3a-a298-4513-806b-62f62283883c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:34.915694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 146/299] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:34 2022] GET /identity/v3/users/0a1778648de841b9aac1dd4d339a7601 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:34.921739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93adc3-5176-43d6-bd04-94abad2fe85d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:34.944628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583d326-b451-442f-9eef-dca8d207f108 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:34.944877 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583d326-b451-442f-9eef-dca8d207f108 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:34.945158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583d326-b451-442f-9eef-dca8d207f108 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:34.946885 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 147/300] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:34.949822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e62e73-efe9-4fb0-b790-3eaa72ac4728 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:34.950112 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e62e73-efe9-4fb0-b790-3eaa72ac4728 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:34.950399 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e62e73-efe9-4fb0-b790-3eaa72ac4728 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:34.951767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 148/301] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:34.954044 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08316-2635-411e-af73-2baa2acd513a None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:34.954044 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08316-2635-411e-af73-2baa2acd513a None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:34.954288 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c08316-2635-411e-af73-2baa2acd513a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:34.974655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-49c08316-2635-411e-af73-2baa2acd513a None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:34.975940 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-49c08316-2635-411e-af73-2baa2acd513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:35.031816 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:35.075225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 149/302] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:35.076458 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93adc3-5176-43d6-bd04-94abad2fe85d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:35.076598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939024-9ada-43d7-a05f-6b5e2b00a8cf None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:35.076895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939024-9ada-43d7-a05f-6b5e2b00a8cf None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:35.077141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939024-9ada-43d7-a05f-6b5e2b00a8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:35.096648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-dc939024-9ada-43d7-a05f-6b5e2b00a8cf None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:35.097797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-dc939024-9ada-43d7-a05f-6b5e2b00a8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:35.128981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf264a-4463-409d-ada2-921574c68d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:35.129856 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf264a-4463-409d-ada2-921574c68d72 None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:35.130142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf264a-4463-409d-ada2-921574c68d72 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:35.130426 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf264a-4463-409d-ada2-921574c68d72 None admin] PATH_INFO: `/v3/system/users/0a1778648de841b9aac1dd4d339a7601/roles/2e3b001c16d24c279be5c68aa3847ca7` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:35.132531 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf264a-4463-409d-ada2-921574c68d72 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0a1778648de841b9aac1dd4d339a7601, role_id=2e3b001c16d24c279be5c68aa3847ca7)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:35.133462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf264a-4463-409d-ada2-921574c68d72 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:35.168347 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:35.173297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 150/303] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:35.177298 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc939024-9ada-43d7-a05f-6b5e2b00a8cf None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:35.233303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 154/304] 10.222.0.88 () {58 vars in 1334 bytes} [Thu May 19 20:49:34 2022] PUT /identity/v3/system/users/0a1778648de841b9aac1dd4d339a7601/roles/2e3b001c16d24c279be5c68aa3847ca7 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:35.234188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf264a-4463-409d-ada2-921574c68d72 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:35.288864 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc939024-9ada-43d7-a05f-6b5e2b00a8cf None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:35.332269 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25757faf-f02e-4efe-bb0c-c302e504f7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:35.333106 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25757faf-f02e-4efe-bb0c-c302e504f7fe None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:35.333386 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25757faf-f02e-4efe-bb0c-c302e504f7fe None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:35.333655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25757faf-f02e-4efe-bb0c-c302e504f7fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:35.348034 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25757faf-f02e-4efe-bb0c-c302e504f7fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-25757faf-f02e-4efe-bb0c-c302e504f7fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:35.356203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:35.357456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:35.358636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:35.359368 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf264a-4463-409d-ada2-921574c68d72 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:35.359808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 151/305] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:35.359808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498bf0-01b6-4a58-b700-ab3bec1b013d None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:35.359808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498bf0-01b6-4a58-b700-ab3bec1b013d None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:35.359808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498bf0-01b6-4a58-b700-ab3bec1b013d None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:35.365955 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 152/306] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:35.365955 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16498bf0-01b6-4a58-b700-ab3bec1b013d None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:35.415937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458f870-adf2-4981-9e2d-e5b61b3d0f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:35.416917 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458f870-adf2-4981-9e2d-e5b61b3d0f21 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:35.417333 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458f870-adf2-4981-9e2d-e5b61b3d0f21 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:35.417615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458f870-adf2-4981-9e2d-e5b61b3d0f21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:35.419100 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458f870-adf2-4981-9e2d-e5b61b3d0f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:35.421940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458f870-adf2-4981-9e2d-e5b61b3d0f21 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:35.424176 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 155/307] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:49:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:35.425539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e879372-9c6a-44ac-86a2-cec4aecefe1a None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:35.425828 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e879372-9c6a-44ac-86a2-cec4aecefe1a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:35.426111 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e879372-9c6a-44ac-86a2-cec4aecefe1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:35.436558 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:35.449975 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-0e879372-9c6a-44ac-86a2-cec4aecefe1a None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:35.451390 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-0e879372-9c6a-44ac-86a2-cec4aecefe1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:35.456240 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:35.507789 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16498bf0-01b6-4a58-b700-ab3bec1b013d None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:35.530946 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 156/308] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:35.532118 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e879372-9c6a-44ac-86a2-cec4aecefe1a None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:35.547503 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444af162-123b-42af-8358-f3ff45fca65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:35.548357 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444af162-123b-42af-8358-f3ff45fca65c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:35.548754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444af162-123b-42af-8358-f3ff45fca65c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:35.549141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444af162-123b-42af-8358-f3ff45fca65c None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:35.550034 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444af162-123b-42af-8358-f3ff45fca65c None admin] RBAC: Adding query filter params (name=reader) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:35.550417 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444af162-123b-42af-8358-f3ff45fca65c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:35.552075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444af162-123b-42af-8358-f3ff45fca65c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:35.562016 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 153/309] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:35.563828 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444af162-123b-42af-8358-f3ff45fca65c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:35.667141 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e879372-9c6a-44ac-86a2-cec4aecefe1a None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:35.680687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444af162-123b-42af-8358-f3ff45fca65c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:35.705922 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:35.709224 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051fdc2-c2ec-425e-8d86-3e0f6f12394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:35.709955 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051fdc2-c2ec-425e-8d86-3e0f6f12394a None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:35.710198 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051fdc2-c2ec-425e-8d86-3e0f6f12394a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:35.710470 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051fdc2-c2ec-425e-8d86-3e0f6f12394a None admin] PATH_INFO: `/v3/groups` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:35.711180 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051fdc2-c2ec-425e-8d86-3e0f6f12394a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:35.712715 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051fdc2-c2ec-425e-8d86-3e0f6f12394a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:35.716137 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:35.720629 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ac4ef-1fb4-464e-9cb9-da72a3e39cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:35.721448 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac4ef-1fb4-464e-9cb9-da72a3e39cc5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:35.721735 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac4ef-1fb4-464e-9cb9-da72a3e39cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:35.722000 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ac4ef-1fb4-464e-9cb9-da72a3e39cc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:35.744432 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ac4ef-1fb4-464e-9cb9-da72a3e39cc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-df6ac4ef-1fb4-464e-9cb9-da72a3e39cc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:35.752775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:35.754040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:35.755272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:35.756478 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 154/310] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:35.757339 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ac4ef-1fb4-464e-9cb9-da72a3e39cc5 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:35.808385 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 157/311] 10.222.0.88 () {60 vars in 1134 bytes} [Thu May 19 20:49:35 2022] POST /identity/v3/groups => generated 229 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:49:35.808385 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051fdc2-c2ec-425e-8d86-3e0f6f12394a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:35.905974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ac4ef-1fb4-464e-9cb9-da72a3e39cc5 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:35.931970 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051fdc2-c2ec-425e-8d86-3e0f6f12394a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:35.957864 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332071ed-0c88-41c5-b8e1-2032e212b656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:35.959379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332071ed-0c88-41c5-b8e1-2032e212b656 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:35.959854 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332071ed-0c88-41c5-b8e1-2032e212b656 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:35.960300 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332071ed-0c88-41c5-b8e1-2032e212b656 None admin] PATH_INFO: `/v3/users/f23c81a968714bd682c43c7dd9b03ec9` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:35.974890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587920d-0f38-4809-8827-ef0edd85f942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:35.975713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587920d-0f38-4809-8827-ef0edd85f942 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:35.976105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587920d-0f38-4809-8827-ef0edd85f942 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:35.976542 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587920d-0f38-4809-8827-ef0edd85f942 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:35.977508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587920d-0f38-4809-8827-ef0edd85f942 None admin] RBAC: Adding query filter params (name=reader) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:35.977937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587920d-0f38-4809-8827-ef0edd85f942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:35.979565 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587920d-0f38-4809-8827-ef0edd85f942 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:35.982036 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332071ed-0c88-41c5-b8e1-2032e212b656 None admin] RBAC: Authorizing `identity:get_user(user_id=f23c81a968714bd682c43c7dd9b03ec9)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:35.984197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332071ed-0c88-41c5-b8e1-2032e212b656 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:35.987202 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 155/312] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:35 2022] GET /identity/v3/users/f23c81a968714bd682c43c7dd9b03ec9 => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:35.992392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 158/313] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:35.993962 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332071ed-0c88-41c5-b8e1-2032e212b656 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:35.997568 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587920d-0f38-4809-8827-ef0edd85f942 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:36.061496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:36.080129 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:36.134744 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587920d-0f38-4809-8827-ef0edd85f942 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:36.167515 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332071ed-0c88-41c5-b8e1-2032e212b656 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:36.197389 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e032979-76ed-4170-a2ba-fe822405babc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:36.199003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e032979-76ed-4170-a2ba-fe822405babc None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:36.199529 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e032979-76ed-4170-a2ba-fe822405babc None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:36.200018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e032979-76ed-4170-a2ba-fe822405babc None admin] PATH_INFO: `/v3/users/761a698fbb914335be3509381a8d46b9` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:36.202471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e032979-76ed-4170-a2ba-fe822405babc None admin] RBAC: Authorizing `identity:get_user(user_id=761a698fbb914335be3509381a8d46b9)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:36.205352 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e032979-76ed-4170-a2ba-fe822405babc None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:36.207528 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bad9b6-34ce-4dad-bc90-e523064d4f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:36.208440 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bad9b6-34ce-4dad-bc90-e523064d4f7c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:36.208889 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bad9b6-34ce-4dad-bc90-e523064d4f7c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:36.209329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bad9b6-34ce-4dad-bc90-e523064d4f7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:36.210431 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bad9b6-34ce-4dad-bc90-e523064d4f7c None admin] RBAC: Adding query filter params (role.id=83321c42fce7438aa5e29c1acb4f07af, user.id=f23c81a968714bd682c43c7dd9b03ec9, scope.system=all) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:36.210918 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bad9b6-34ce-4dad-bc90-e523064d4f7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:36.212579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bad9b6-34ce-4dad-bc90-e523064d4f7c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:36.214701 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 159/314] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:35 2022] GET /identity/v3/users/761a698fbb914335be3509381a8d46b9 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:36.217664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e032979-76ed-4170-a2ba-fe822405babc None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:36.219994 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 156/315] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:49:35 2022] GET /identity/v3/role_assignments?user.id=f23c81a968714bd682c43c7dd9b03ec9&scope.system=all&role.id=83321c42fce7438aa5e29c1acb4f07af => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:36.420804 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e032979-76ed-4170-a2ba-fe822405babc None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:36.464967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:36.483334 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9e17c-9115-4c72-b998-aad33c0dca44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:36.485142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9e17c-9115-4c72-b998-aad33c0dca44 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:36.485142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9e17c-9115-4c72-b998-aad33c0dca44 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:36.485467 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9e17c-9115-4c72-b998-aad33c0dca44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:36.611961 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9e17c-9115-4c72-b998-aad33c0dca44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:36.614794 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9e17c-9115-4c72-b998-aad33c0dca44 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:36.757422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 160/316] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:49:36.763989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bad9b6-34ce-4dad-bc90-e523064d4f7c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:36.876185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bad9b6-34ce-4dad-bc90-e523064d4f7c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:36.879103 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:36.914529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4d6bc-169d-4531-8226-58d362c0b179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:36.915473 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4d6bc-169d-4531-8226-58d362c0b179 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:36.915763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4d6bc-169d-4531-8226-58d362c0b179 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:36.916042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4d6bc-169d-4531-8226-58d362c0b179 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:36.917331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4d6bc-169d-4531-8226-58d362c0b179 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:36.918793 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4d6bc-169d-4531-8226-58d362c0b179 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:36.920915 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 157/317] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:36 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:36.934396 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9e17c-9115-4c72-b998-aad33c0dca44 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:36.962896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:37.050102 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9e17c-9115-4c72-b998-aad33c0dca44 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:37.091918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48515570-a27e-4231-b2de-501029f396d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:37.092628 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48515570-a27e-4231-b2de-501029f396d6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:37.092921 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48515570-a27e-4231-b2de-501029f396d6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:37.093205 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48515570-a27e-4231-b2de-501029f396d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:37.094103 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48515570-a27e-4231-b2de-501029f396d6 None admin] RBAC: Adding query filter params (role.id=83321c42fce7438aa5e29c1acb4f07af, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2, user.id=761a698fbb914335be3509381a8d46b9) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:37.094393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48515570-a27e-4231-b2de-501029f396d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:37.096523 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48515570-a27e-4231-b2de-501029f396d6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:37.104109 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 161/318] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:49:36 2022] GET /identity/v3/role_assignments?user.id=761a698fbb914335be3509381a8d46b9&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=83321c42fce7438aa5e29c1acb4f07af => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:37.561233 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55a766-61a8-43af-9046-067b7d51e6ee None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:37.561668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55a766-61a8-43af-9046-067b7d51e6ee None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:37.561789 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55a766-61a8-43af-9046-067b7d51e6ee None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:37.563190 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 158/319] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:37.569587 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b5e32-b368-4041-a1ad-953f17793250 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:37.570148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b5e32-b368-4041-a1ad-953f17793250 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:37.570148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b5e32-b368-4041-a1ad-953f17793250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:37.587397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-7f2b5e32-b368-4041-a1ad-953f17793250 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:37.588668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-7f2b5e32-b368-4041-a1ad-953f17793250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:37.666072 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 162/320] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:37.690437 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55a766-61a8-43af-9046-067b7d51e6ee None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:37.803051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55a766-61a8-43af-9046-067b7d51e6ee None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:37.846320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e2df3-6866-4638-bf9d-f496e5886f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:37.847081 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e2df3-6866-4638-bf9d-f496e5886f7f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:37.847404 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e2df3-6866-4638-bf9d-f496e5886f7f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:37.847683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e2df3-6866-4638-bf9d-f496e5886f7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:37.859332 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e2df3-6866-4638-bf9d-f496e5886f7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-9f5e2df3-6866-4638-bf9d-f496e5886f7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:37.867243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:37.869022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:37.870309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:37.872712 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 159/321] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:37.874445 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b5e32-b368-4041-a1ad-953f17793250 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:37.906386 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:38.009000 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b5e32-b368-4041-a1ad-953f17793250 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:38.055209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6110949-c7d6-4069-91b8-3bdf9edb8457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:38.055927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6110949-c7d6-4069-91b8-3bdf9edb8457 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:38.056216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6110949-c7d6-4069-91b8-3bdf9edb8457 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:38.056533 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6110949-c7d6-4069-91b8-3bdf9edb8457 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:38.057385 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6110949-c7d6-4069-91b8-3bdf9edb8457 None admin] RBAC: Adding query filter params (name=member) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:38.058072 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6110949-c7d6-4069-91b8-3bdf9edb8457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:38.059438 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6110949-c7d6-4069-91b8-3bdf9edb8457 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:38.070743 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 163/322] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:38.076382 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e2df3-6866-4638-bf9d-f496e5886f7f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:38.116097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:38.201267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e2df3-6866-4638-bf9d-f496e5886f7f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:38.246715 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feacb5b-53c6-48cd-b0ba-444df30af6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:38.247641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feacb5b-53c6-48cd-b0ba-444df30af6fe None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:38.247940 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feacb5b-53c6-48cd-b0ba-444df30af6fe None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:38.248197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feacb5b-53c6-48cd-b0ba-444df30af6fe None admin] PATH_INFO: `/v3/users/0a1778648de841b9aac1dd4d339a7601` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:38.249497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feacb5b-53c6-48cd-b0ba-444df30af6fe None admin] RBAC: Authorizing `identity:get_user(user_id=0a1778648de841b9aac1dd4d339a7601)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:38.250950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feacb5b-53c6-48cd-b0ba-444df30af6fe None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:38.253356 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 160/323] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:38 2022] GET /identity/v3/users/0a1778648de841b9aac1dd4d339a7601 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:38.259027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6110949-c7d6-4069-91b8-3bdf9edb8457 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:38.378747 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:38.393141 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6110949-c7d6-4069-91b8-3bdf9edb8457 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:38.429404 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375b64b-1c33-4e98-9486-b7e33a50ba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:38.430093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375b64b-1c33-4e98-9486-b7e33a50ba94 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:38.430387 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375b64b-1c33-4e98-9486-b7e33a50ba94 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:38.430673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375b64b-1c33-4e98-9486-b7e33a50ba94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:38.431598 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375b64b-1c33-4e98-9486-b7e33a50ba94 None admin] RBAC: Adding query filter params (role.id=2e3b001c16d24c279be5c68aa3847ca7, user.id=0a1778648de841b9aac1dd4d339a7601, scope.system=all) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:38.432531 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375b64b-1c33-4e98-9486-b7e33a50ba94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:38.436380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375b64b-1c33-4e98-9486-b7e33a50ba94 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:38.456389 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 164/324] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:49:38 2022] GET /identity/v3/role_assignments?user.id=0a1778648de841b9aac1dd4d339a7601&scope.system=all&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:38.513238 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2244b-a02f-4743-a84b-e5aa658b8655 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:38.513517 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2244b-a02f-4743-a84b-e5aa658b8655 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:38.513780 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2244b-a02f-4743-a84b-e5aa658b8655 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:38.515438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 161/325] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:38.521342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f033f-1da8-4097-8890-455324dbbd54 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:38.521619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f033f-1da8-4097-8890-455324dbbd54 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:38.521907 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f033f-1da8-4097-8890-455324dbbd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:38.540194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-7a6f033f-1da8-4097-8890-455324dbbd54 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:38.541497 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-7a6f033f-1da8-4097-8890-455324dbbd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:38.625363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 165/326] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:38.632575 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef96200-b5c2-4d99-bbaa-7a1289b06031 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:38.632805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef96200-b5c2-4d99-bbaa-7a1289b06031 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:38.633061 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef96200-b5c2-4d99-bbaa-7a1289b06031 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:38.634661 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 162/327] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:38.640479 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a26b1-62df-47fa-93f9-25d26b8057e9 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:38.640829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a26b1-62df-47fa-93f9-25d26b8057e9 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:38.641055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a26b1-62df-47fa-93f9-25d26b8057e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:38.649119 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef96200-b5c2-4d99-bbaa-7a1289b06031 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:38.661036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-7b3a26b1-62df-47fa-93f9-25d26b8057e9 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:38.662666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-7b3a26b1-62df-47fa-93f9-25d26b8057e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:38.734528 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:38.740815 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 166/328] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:38.765261 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:38.769254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef96200-b5c2-4d99-bbaa-7a1289b06031 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:38.771242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a26b1-62df-47fa-93f9-25d26b8057e9 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:38.806821 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80423ae-7f4c-468b-ab70-3befa5ad6202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:38.807872 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80423ae-7f4c-468b-ab70-3befa5ad6202 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:38.808166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80423ae-7f4c-468b-ab70-3befa5ad6202 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:38.808478 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80423ae-7f4c-468b-ab70-3befa5ad6202 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:38.809667 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80423ae-7f4c-468b-ab70-3befa5ad6202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:38.811097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80423ae-7f4c-468b-ab70-3befa5ad6202 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:38.812766 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 163/329] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:49:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:38.818393 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80423ae-7f4c-468b-ab70-3befa5ad6202 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:38.881340 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a26b1-62df-47fa-93f9-25d26b8057e9 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:38.918209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b022409-c784-4ead-b5b3-0117837a2cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:38.918926 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b022409-c784-4ead-b5b3-0117837a2cbb None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:38.919231 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b022409-c784-4ead-b5b3-0117837a2cbb None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:38.919488 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b022409-c784-4ead-b5b3-0117837a2cbb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:38.924686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80423ae-7f4c-468b-ab70-3befa5ad6202 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:38.934577 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b022409-c784-4ead-b5b3-0117837a2cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-0b022409-c784-4ead-b5b3-0117837a2cbb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:38.942427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:38.943977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:38.945290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 167/330] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:49:38.949883 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b022409-c784-4ead-b5b3-0117837a2cbb None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:38.963413 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9eda60-bf2a-49cb-9a64-628f22dd880c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:38.964133 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eda60-bf2a-49cb-9a64-628f22dd880c None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:38.964408 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eda60-bf2a-49cb-9a64-628f22dd880c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:38.964692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9eda60-bf2a-49cb-9a64-628f22dd880c None admin] PATH_INFO: `/v3/groups` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:38.965412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9eda60-bf2a-49cb-9a64-628f22dd880c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:38.966858 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9eda60-bf2a-49cb-9a64-628f22dd880c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:39.010494 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:39.050508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 164/331] 10.222.0.88 () {60 vars in 1134 bytes} [Thu May 19 20:49:38 2022] POST /identity/v3/groups => generated 226 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:49:39.054887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b022409-c784-4ead-b5b3-0117837a2cbb None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:39.095307 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319bc59-d05c-4580-a344-008f42bf3d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:39.096893 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319bc59-d05c-4580-a344-008f42bf3d2d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:39.097182 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319bc59-d05c-4580-a344-008f42bf3d2d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:39.097461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319bc59-d05c-4580-a344-008f42bf3d2d None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:39.098241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319bc59-d05c-4580-a344-008f42bf3d2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:39.098595 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319bc59-d05c-4580-a344-008f42bf3d2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:39.100167 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319bc59-d05c-4580-a344-008f42bf3d2d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:39.110262 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 168/332] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:39.116493 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9eda60-bf2a-49cb-9a64-628f22dd880c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:39.148810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:39.230948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9eda60-bf2a-49cb-9a64-628f22dd880c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:39.271975 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77771cb0-d760-432f-962e-1c87b730798b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:39.272889 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771cb0-d760-432f-962e-1c87b730798b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:39.273231 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771cb0-d760-432f-962e-1c87b730798b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:39.273531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771cb0-d760-432f-962e-1c87b730798b None admin] PATH_INFO: `/v3/users/f23c81a968714bd682c43c7dd9b03ec9` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:39.274768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77771cb0-d760-432f-962e-1c87b730798b None admin] RBAC: Authorizing `identity:get_user(user_id=f23c81a968714bd682c43c7dd9b03ec9)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:39.276226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77771cb0-d760-432f-962e-1c87b730798b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:39.277975 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 165/333] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:39 2022] GET /identity/v3/users/f23c81a968714bd682c43c7dd9b03ec9 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:39.284431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319bc59-d05c-4580-a344-008f42bf3d2d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:39.395349 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319bc59-d05c-4580-a344-008f42bf3d2d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:39.419886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:39.433954 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e4ab2-ae42-4b35-9c77-324abdeb639e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:39.434866 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e4ab2-ae42-4b35-9c77-324abdeb639e None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:39.435151 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e4ab2-ae42-4b35-9c77-324abdeb639e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:39.435430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e4ab2-ae42-4b35-9c77-324abdeb639e None admin] PATH_INFO: `/v3/system/users/f23c81a968714bd682c43c7dd9b03ec9/roles/83321c42fce7438aa5e29c1acb4f07af` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:39.437055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e4ab2-ae42-4b35-9c77-324abdeb639e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f23c81a968714bd682c43c7dd9b03ec9, role_id=83321c42fce7438aa5e29c1acb4f07af)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:39.440063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858e4ab2-ae42-4b35-9c77-324abdeb639e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:39.523858 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 169/334] 10.222.0.88 () {58 vars in 1334 bytes} [Thu May 19 20:49:39 2022] PUT /identity/v3/system/users/f23c81a968714bd682c43c7dd9b03ec9/roles/83321c42fce7438aa5e29c1acb4f07af => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:41.882906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9159336-e979-4f35-b97e-0cc51704407e None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:41.883617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9159336-e979-4f35-b97e-0cc51704407e None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:41.883617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9159336-e979-4f35-b97e-0cc51704407e None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:41.885178 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 166/335] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:41.897352 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f779ad-bce6-415d-9dec-5d05aedd4372 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:41.898039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f779ad-bce6-415d-9dec-5d05aedd4372 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:41.898039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f779ad-bce6-415d-9dec-5d05aedd4372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:41.921423 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-e7f779ad-bce6-415d-9dec-5d05aedd4372 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:41.922658 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-e7f779ad-bce6-415d-9dec-5d05aedd4372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:42.027413 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 170/336] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:42.068006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9159336-e979-4f35-b97e-0cc51704407e None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:42.197458 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:42.204834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9159336-e979-4f35-b97e-0cc51704407e None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:42.247878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e801-e706-4015-bfc8-6b78daee332f None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:42.248597 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e801-e706-4015-bfc8-6b78daee332f None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:42.249101 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34e801-e706-4015-bfc8-6b78daee332f None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:42.250758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 171/337] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:49:42.263938 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373a96c-f9b8-4796-b6be-8c1a80a39454 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:42.264542 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373a96c-f9b8-4796-b6be-8c1a80a39454 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:42.265062 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373a96c-f9b8-4796-b6be-8c1a80a39454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:42.273126 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8d7d9-0105-4e56-a048-ab9107fd794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:42.273956 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8d7d9-0105-4e56-a048-ab9107fd794f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:42.274245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8d7d9-0105-4e56-a048-ab9107fd794f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:42.274823 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8d7d9-0105-4e56-a048-ab9107fd794f None admin] PATH_INFO: `/v3/roles/member` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:42.301334 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-b373a96c-f9b8-4796-b6be-8c1a80a39454 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:42.301551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8d7d9-0105-4e56-a048-ab9107fd794f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:42.302583 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-b373a96c-f9b8-4796-b6be-8c1a80a39454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-abb8d7d9-0105-4e56-a048-ab9107fd794f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:42.316943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:42.318108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:42.319142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 167/338] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:42.326389 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8d7d9-0105-4e56-a048-ab9107fd794f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:42.502075 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 172/339] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:42.514900 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8d7d9-0105-4e56-a048-ab9107fd794f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:42.588049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7415e63-af31-4dbd-b916-05642136f9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:42.589177 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7415e63-af31-4dbd-b916-05642136f9ef None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:42.589671 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7415e63-af31-4dbd-b916-05642136f9ef None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:42.590155 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7415e63-af31-4dbd-b916-05642136f9ef None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:42.591213 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7415e63-af31-4dbd-b916-05642136f9ef None admin] RBAC: Adding query filter params (name=member) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:42.591755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7415e63-af31-4dbd-b916-05642136f9ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:42.593470 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7415e63-af31-4dbd-b916-05642136f9ef None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:42.610040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 168/340] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:42.614254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373a96c-f9b8-4796-b6be-8c1a80a39454 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:42.620375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7415e63-af31-4dbd-b916-05642136f9ef None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:42.660072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:42.704792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:42.778089 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373a96c-f9b8-4796-b6be-8c1a80a39454 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:42.786769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7415e63-af31-4dbd-b916-05642136f9ef None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:42.837528 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facab937-315b-41a0-8a74-459d8668592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:42.839554 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facab937-315b-41a0-8a74-459d8668592b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:42.840550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facab937-315b-41a0-8a74-459d8668592b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:42.841177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facab937-315b-41a0-8a74-459d8668592b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:42.846484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b718-af49-4580-af2a-30c4dd211c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:42.847566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b718-af49-4580-af2a-30c4dd211c34 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:42.847872 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b718-af49-4580-af2a-30c4dd211c34 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:42.848368 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b718-af49-4580-af2a-30c4dd211c34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:42.866783 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facab937-315b-41a0-8a74-459d8668592b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-facab937-315b-41a0-8a74-459d8668592b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:42.880190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:42.881660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:42.882644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:42.886112 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 173/341] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:42.888603 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73f898-6779-41c4-8089-a547bd4f05b3 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:42.889465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73f898-6779-41c4-8089-a547bd4f05b3 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:42.889896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73f898-6779-41c4-8089-a547bd4f05b3 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:42.891778 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 174/342] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:42.898539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa922e-5324-4b48-9e6a-a8a08d9c2af6 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:42.899634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa922e-5324-4b48-9e6a-a8a08d9c2af6 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:42.900208 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fa922e-5324-4b48-9e6a-a8a08d9c2af6 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:42.902936 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 175/343] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:42.906713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa922e-5324-4b48-9e6a-a8a08d9c2af6 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:42.933589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961b718-af49-4580-af2a-30c4dd211c34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:42.936174 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6961b718-af49-4580-af2a-30c4dd211c34 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:42.972576 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:43.061477 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fa922e-5324-4b48-9e6a-a8a08d9c2af6 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:43.064473 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:43.103328 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 169/344] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:43.104859 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a184c9-2d33-40c4-85d7-80360cc44e18 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:43.105152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a184c9-2d33-40c4-85d7-80360cc44e18 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:43.105434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a184c9-2d33-40c4-85d7-80360cc44e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:43.107853 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28cc8a-10d4-4044-b8e2-30ff39f18668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:43.109204 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28cc8a-10d4-4044-b8e2-30ff39f18668 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:43.109726 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28cc8a-10d4-4044-b8e2-30ff39f18668 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:43.110323 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28cc8a-10d4-4044-b8e2-30ff39f18668 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:43.111541 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28cc8a-10d4-4044-b8e2-30ff39f18668 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:43.112098 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28cc8a-10d4-4044-b8e2-30ff39f18668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:43.113799 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28cc8a-10d4-4044-b8e2-30ff39f18668 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:43.123575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 176/345] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:43.125378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb28bcd-d8cc-41fb-96a8-333d66ba4bb6 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:43.125638 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-56a184c9-2d33-40c4-85d7-80360cc44e18 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:43.126033 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb28bcd-d8cc-41fb-96a8-333d66ba4bb6 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:43.126599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb28bcd-d8cc-41fb-96a8-333d66ba4bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:43.127488 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-56a184c9-2d33-40c4-85d7-80360cc44e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:43.152425 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-2cb28bcd-d8cc-41fb-96a8-333d66ba4bb6 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:43.153934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-2cb28bcd-d8cc-41fb-96a8-333d66ba4bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:43.236565 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 170/346] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:43.236565 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a184c9-2d33-40c4-85d7-80360cc44e18 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:43.276010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 177/347] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:43.279061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb28bcd-d8cc-41fb-96a8-333d66ba4bb6 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:43.325926 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:43.397289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a184c9-2d33-40c4-85d7-80360cc44e18 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:43.434653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:43.443230 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadf8f7-da86-4dfa-a3ff-1f7975ca1b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:43.443653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb28bcd-d8cc-41fb-96a8-333d66ba4bb6 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:43.446705 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadf8f7-da86-4dfa-a3ff-1f7975ca1b84 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:43.447378 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadf8f7-da86-4dfa-a3ff-1f7975ca1b84 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:43.452302 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadf8f7-da86-4dfa-a3ff-1f7975ca1b84 None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:43.452302 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadf8f7-da86-4dfa-a3ff-1f7975ca1b84 None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:43.452302 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadf8f7-da86-4dfa-a3ff-1f7975ca1b84 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:43.453061 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 171/348] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:43 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:43.453933 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadf8f7-da86-4dfa-a3ff-1f7975ca1b84 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:43.491739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6e5ae-f9b5-44d6-8d1a-196d55742903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:43.492543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6e5ae-f9b5-44d6-8d1a-196d55742903 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:43.492839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6e5ae-f9b5-44d6-8d1a-196d55742903 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:43.493126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6e5ae-f9b5-44d6-8d1a-196d55742903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:43.594068 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6e5ae-f9b5-44d6-8d1a-196d55742903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:43.596117 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6e5ae-f9b5-44d6-8d1a-196d55742903 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:43.601460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadf8f7-da86-4dfa-a3ff-1f7975ca1b84 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:43.624020 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:43.648644 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd86780-997d-4664-af86-1ecf620dd670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:43.649469 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86780-997d-4664-af86-1ecf620dd670 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:43.649756 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86780-997d-4664-af86-1ecf620dd670 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:43.650643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86780-997d-4664-af86-1ecf620dd670 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:43.667382 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd86780-997d-4664-af86-1ecf620dd670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-9fd86780-997d-4664-af86-1ecf620dd670 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:43.677521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:43.678616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:43.679711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:43.683760 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 172/349] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:43.684747 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd86780-997d-4664-af86-1ecf620dd670 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:43.723559 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:43.732531 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 178/350] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:43.733425 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6e5ae-f9b5-44d6-8d1a-196d55742903 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:43.827824 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd86780-997d-4664-af86-1ecf620dd670 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:43.866467 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6e5ae-f9b5-44d6-8d1a-196d55742903 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:43.879390 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d490f-7946-4431-a6e6-6883ce4067bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:43.879390 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d490f-7946-4431-a6e6-6883ce4067bb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:43.879390 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d490f-7946-4431-a6e6-6883ce4067bb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:43.879780 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d490f-7946-4431-a6e6-6883ce4067bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:43.895065 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266d490f-7946-4431-a6e6-6883ce4067bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-266d490f-7946-4431-a6e6-6883ce4067bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:43.903221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:43.904543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 19 20:49:43.905741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:43.909838 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 173/351] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:43.910876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d490f-7946-4431-a6e6-6883ce4067bb None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:43.919967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741dbdd-1d83-47f1-bdeb-133609cf7807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:43.921310 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741dbdd-1d83-47f1-bdeb-133609cf7807 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:43.921310 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741dbdd-1d83-47f1-bdeb-133609cf7807 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:43.921565 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741dbdd-1d83-47f1-bdeb-133609cf7807 None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:43.929291 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741dbdd-1d83-47f1-bdeb-133609cf7807 None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:43.930821 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741dbdd-1d83-47f1-bdeb-133609cf7807 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:43.935287 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 179/352] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:43 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:43.936627 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741dbdd-1d83-47f1-bdeb-133609cf7807 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:43.963396 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:44.027950 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:44.062462 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266d490f-7946-4431-a6e6-6883ce4067bb None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:44.102614 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741dbdd-1d83-47f1-bdeb-133609cf7807 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:44.122602 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af2c56-1b13-4040-a86c-9d4e5fc19261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:44.123729 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af2c56-1b13-4040-a86c-9d4e5fc19261 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:44.124230 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af2c56-1b13-4040-a86c-9d4e5fc19261 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:44.124688 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af2c56-1b13-4040-a86c-9d4e5fc19261 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:44.125682 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af2c56-1b13-4040-a86c-9d4e5fc19261 None admin] RBAC: Adding query filter params (name=member) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:44.126172 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af2c56-1b13-4040-a86c-9d4e5fc19261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:44.128283 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af2c56-1b13-4040-a86c-9d4e5fc19261 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:44.138330 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 174/353] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:44.139715 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc47a9-af51-4584-8ae4-6f4d88a8d56e None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:44.140002 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc47a9-af51-4584-8ae4-6f4d88a8d56e None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:44.140531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc47a9-af51-4584-8ae4-6f4d88a8d56e None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:44.142887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 175/354] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:44.143980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc47a9-af51-4584-8ae4-6f4d88a8d56e None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:44.158087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada33fc-599f-48e0-bcfd-c5903b4e139a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:44.159335 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada33fc-599f-48e0-bcfd-c5903b4e139a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:44.159785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada33fc-599f-48e0-bcfd-c5903b4e139a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:44.160065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada33fc-599f-48e0-bcfd-c5903b4e139a None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:44.161934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada33fc-599f-48e0-bcfd-c5903b4e139a None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:44.163530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada33fc-599f-48e0-bcfd-c5903b4e139a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:44.165479 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 180/355] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:43 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:44.166378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada33fc-599f-48e0-bcfd-c5903b4e139a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:44.263630 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:44.271297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc47a9-af51-4584-8ae4-6f4d88a8d56e None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:44.294115 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada33fc-599f-48e0-bcfd-c5903b4e139a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:44.320408 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ae893-8d7a-4cda-859a-7e9225d8583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:44.321578 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ae893-8d7a-4cda-859a-7e9225d8583d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:44.321578 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ae893-8d7a-4cda-859a-7e9225d8583d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:44.321828 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ae893-8d7a-4cda-859a-7e9225d8583d None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:44.322428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ae893-8d7a-4cda-859a-7e9225d8583d None admin] RBAC: Adding query filter params (name=reader) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:44.322920 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ae893-8d7a-4cda-859a-7e9225d8583d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:44.324489 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ae893-8d7a-4cda-859a-7e9225d8583d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:44.326728 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:44.335061 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 176/356] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:44.338669 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ae893-8d7a-4cda-859a-7e9225d8583d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:44.341658 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e51c8-7d61-4122-b6d6-7bf39e0d4927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:44.343494 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e51c8-7d61-4122-b6d6-7bf39e0d4927 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:44.343494 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e51c8-7d61-4122-b6d6-7bf39e0d4927 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:44.343494 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e51c8-7d61-4122-b6d6-7bf39e0d4927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:44.343781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e51c8-7d61-4122-b6d6-7bf39e0d4927 None admin] RBAC: Adding query filter params (group.id=1e903affae424374a9cfedc0a6ff8a4b, role.id=2e3b001c16d24c279be5c68aa3847ca7, scope.project.id=834a091f1d39471ba13d3156d015bf3d) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:44.344054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e51c8-7d61-4122-b6d6-7bf39e0d4927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:44.346798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e51c8-7d61-4122-b6d6-7bf39e0d4927 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:44.354677 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 181/357] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:49:44 2022] GET /identity/v3/role_assignments?group.id=1e903affae424374a9cfedc0a6ff8a4b&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:44.355945 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122e713-2a20-40ad-8cf5-4f3465950dc1 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:44.356242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122e713-2a20-40ad-8cf5-4f3465950dc1 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:44.356595 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122e713-2a20-40ad-8cf5-4f3465950dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:44.378067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-5122e713-2a20-40ad-8cf5-4f3465950dc1 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:44.379225 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-5122e713-2a20-40ad-8cf5-4f3465950dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:44.475846 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ae893-8d7a-4cda-859a-7e9225d8583d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:44.494490 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 182/358] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:44.495648 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122e713-2a20-40ad-8cf5-4f3465950dc1 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:44.533759 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee57d2-5e75-4d5d-8975-bc3a86ce20dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:44.534878 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee57d2-5e75-4d5d-8975-bc3a86ce20dc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:44.535298 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee57d2-5e75-4d5d-8975-bc3a86ce20dc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:44.535715 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee57d2-5e75-4d5d-8975-bc3a86ce20dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:44.582655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:44.637051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee57d2-5e75-4d5d-8975-bc3a86ce20dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:44.638577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee57d2-5e75-4d5d-8975-bc3a86ce20dc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:44.655771 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122e713-2a20-40ad-8cf5-4f3465950dc1 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:44.688672 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:44.712057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba7b85-853f-4ee7-a839-6d30150b4098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:44.713698 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba7b85-853f-4ee7-a839-6d30150b4098 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:44.714177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba7b85-853f-4ee7-a839-6d30150b4098 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:44.714719 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba7b85-853f-4ee7-a839-6d30150b4098 None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:44.716769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba7b85-853f-4ee7-a839-6d30150b4098 None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:44.718268 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba7b85-853f-4ee7-a839-6d30150b4098 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:44.719968 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 183/359] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:44 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:44.721314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2645c0e-b54f-459c-bfae-621434e5af15 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:44.721603 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2645c0e-b54f-459c-bfae-621434e5af15 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:44.721885 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2645c0e-b54f-459c-bfae-621434e5af15 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:44.723278 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 184/360] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:44.724056 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2645c0e-b54f-459c-bfae-621434e5af15 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:44.774041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 177/361] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:44.774738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee57d2-5e75-4d5d-8975-bc3a86ce20dc None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:44.856195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:44.907378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2645c0e-b54f-459c-bfae-621434e5af15 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:44.913433 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee57d2-5e75-4d5d-8975-bc3a86ce20dc None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:44.961425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98d06c-0869-4ff9-a93d-799397a52267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:44.962703 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98d06c-0869-4ff9-a93d-799397a52267 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:44.963222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98d06c-0869-4ff9-a93d-799397a52267 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:44.963451 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c89234-a2c3-462d-9f50-f17e79e03b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:44.964373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98d06c-0869-4ff9-a93d-799397a52267 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:44.964824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c89234-a2c3-462d-9f50-f17e79e03b1d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:44.964824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c89234-a2c3-462d-9f50-f17e79e03b1d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:44.965017 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c89234-a2c3-462d-9f50-f17e79e03b1d None admin] PATH_INFO: `/v3/users/f23c81a968714bd682c43c7dd9b03ec9` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:44.967760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c89234-a2c3-462d-9f50-f17e79e03b1d None admin] RBAC: Authorizing `identity:get_user(user_id=f23c81a968714bd682c43c7dd9b03ec9)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:44.970008 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c89234-a2c3-462d-9f50-f17e79e03b1d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:44.972428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 185/362] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:49:44 2022] GET /identity/v3/users/f23c81a968714bd682c43c7dd9b03ec9 => generated 293 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:44.973279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c89234-a2c3-462d-9f50-f17e79e03b1d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:44.985952 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea98d06c-0869-4ff9-a93d-799397a52267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-ea98d06c-0869-4ff9-a93d-799397a52267 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:44.995607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:44.996964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:44.997913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 178/363] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:45.004475 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41151217-0cbb-4787-89c8-f48ec161ae8c None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:45.004475 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41151217-0cbb-4787-89c8-f48ec161ae8c None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:45.004475 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41151217-0cbb-4787-89c8-f48ec161ae8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:45.028912 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-41151217-0cbb-4787-89c8-f48ec161ae8c None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:45.030120 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-41151217-0cbb-4787-89c8-f48ec161ae8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:45.067605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:45.127494 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c89234-a2c3-462d-9f50-f17e79e03b1d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:45.143149 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 179/364] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:45.146201 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41151217-0cbb-4787-89c8-f48ec161ae8c None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:45.175130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30329c0b-7b10-473d-bfa6-b8efc6af6cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:45.175707 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30329c0b-7b10-473d-bfa6-b8efc6af6cd0 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:45.176000 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30329c0b-7b10-473d-bfa6-b8efc6af6cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:45.176283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30329c0b-7b10-473d-bfa6-b8efc6af6cd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:45.178394 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30329c0b-7b10-473d-bfa6-b8efc6af6cd0 None admin] RBAC: Adding query filter params (group.id=1e903affae424374a9cfedc0a6ff8a4b, role.id=4313b1c9df5345778a249b04355b90a0, scope.project.id=834a091f1d39471ba13d3156d015bf3d) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:45.178724 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30329c0b-7b10-473d-bfa6-b8efc6af6cd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:45.180234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30329c0b-7b10-473d-bfa6-b8efc6af6cd0 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:45.187615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 186/365] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:49:44 2022] GET /identity/v3/role_assignments?group.id=1e903affae424374a9cfedc0a6ff8a4b&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=4313b1c9df5345778a249b04355b90a0 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:45.188840 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30329c0b-7b10-473d-bfa6-b8efc6af6cd0 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:45.224593 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:45.302093 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41151217-0cbb-4787-89c8-f48ec161ae8c None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:45.318250 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30329c0b-7b10-473d-bfa6-b8efc6af6cd0 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:45.350547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:45.355035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9e228-a473-45cc-839c-eef8078b5a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:45.356434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9e228-a473-45cc-839c-eef8078b5a71 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:45.356763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9e228-a473-45cc-839c-eef8078b5a71 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:45.357153 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9e228-a473-45cc-839c-eef8078b5a71 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:45.359449 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9e228-a473-45cc-839c-eef8078b5a71 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:45.361536 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed9e228-a473-45cc-839c-eef8078b5a71 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:45.364389 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 180/366] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:45 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:45.365359 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9e228-a473-45cc-839c-eef8078b5a71 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:45.371281 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb1a75-bf64-4edf-9463-99a4fc3f4978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:45.372433 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb1a75-bf64-4edf-9463-99a4fc3f4978 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:45.372945 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb1a75-bf64-4edf-9463-99a4fc3f4978 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:45.373413 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb1a75-bf64-4edf-9463-99a4fc3f4978 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:45.374461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb1a75-bf64-4edf-9463-99a4fc3f4978 None admin] RBAC: Adding query filter params (role.id=83321c42fce7438aa5e29c1acb4f07af, user.id=f23c81a968714bd682c43c7dd9b03ec9, scope.system=all) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:45.374983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb1a75-bf64-4edf-9463-99a4fc3f4978 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:45.376835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb1a75-bf64-4edf-9463-99a4fc3f4978 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:45.402226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 187/367] 10.222.0.88 () {56 vars in 1308 bytes} [Thu May 19 20:49:45 2022] GET /identity/v3/role_assignments?user.id=f23c81a968714bd682c43c7dd9b03ec9&scope.system=all&role.id=83321c42fce7438aa5e29c1acb4f07af => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:45.408416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb1a75-bf64-4edf-9463-99a4fc3f4978 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:45.529541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9e228-a473-45cc-839c-eef8078b5a71 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:45.552675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:45.574289 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb1a75-bf64-4edf-9463-99a4fc3f4978 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:45.576362 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae1959-d32c-493d-a2e4-5da99d72e7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:45.577226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae1959-d32c-493d-a2e4-5da99d72e7b0 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:45.577522 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae1959-d32c-493d-a2e4-5da99d72e7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:45.577833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ae1959-d32c-493d-a2e4-5da99d72e7b0 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:45.578608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae1959-d32c-493d-a2e4-5da99d72e7b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:45.578949 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae1959-d32c-493d-a2e4-5da99d72e7b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:45.581228 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ae1959-d32c-493d-a2e4-5da99d72e7b0 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:45.593799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 181/368] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:45.600302 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae1959-d32c-493d-a2e4-5da99d72e7b0 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:45.626088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9fd0d-24e3-4516-a76c-19d2e79577a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:45.627947 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9fd0d-24e3-4516-a76c-19d2e79577a9 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:45.628242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9fd0d-24e3-4516-a76c-19d2e79577a9 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:45.628566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b9fd0d-24e3-4516-a76c-19d2e79577a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:45.646190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b9fd0d-24e3-4516-a76c-19d2e79577a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-37b9fd0d-24e3-4516-a76c-19d2e79577a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:45.656785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:45.657983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:45.658956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:45.660212 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 188/369] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:49:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:45.665206 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9fd0d-24e3-4516-a76c-19d2e79577a9 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:45.725775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:45.769570 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ae1959-d32c-493d-a2e4-5da99d72e7b0 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:45.812178 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0bedc-8a31-4b73-aa17-d3e594e68b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:45.813026 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0bedc-8a31-4b73-aa17-d3e594e68b4d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:45.813299 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0bedc-8a31-4b73-aa17-d3e594e68b4d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:45.813583 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0bedc-8a31-4b73-aa17-d3e594e68b4d None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:45.815459 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0bedc-8a31-4b73-aa17-d3e594e68b4d None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:45.817163 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0bedc-8a31-4b73-aa17-d3e594e68b4d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:45.818810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 182/370] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:45 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:45.819597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0bedc-8a31-4b73-aa17-d3e594e68b4d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:45.838813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b9fd0d-24e3-4516-a76c-19d2e79577a9 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:45.877824 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:45.894632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a470ff9-7ed4-4dac-990e-b2a63deebb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:45.895823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470ff9-7ed4-4dac-990e-b2a63deebb49 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:45.896351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470ff9-7ed4-4dac-990e-b2a63deebb49 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:45.896848 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470ff9-7ed4-4dac-990e-b2a63deebb49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:45.937732 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0bedc-8a31-4b73-aa17-d3e594e68b4d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:45.979532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a8060-f0ab-480b-92cf-90e9c7c89ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:45.980273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a8060-f0ab-480b-92cf-90e9c7c89ca5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:45.980697 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a8060-f0ab-480b-92cf-90e9c7c89ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:45.980991 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a8060-f0ab-480b-92cf-90e9c7c89ca5 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:45.981753 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a8060-f0ab-480b-92cf-90e9c7c89ca5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:45.982063 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a8060-f0ab-480b-92cf-90e9c7c89ca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:45.983572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a8060-f0ab-480b-92cf-90e9c7c89ca5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:46.000086 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 183/371] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:46.002307 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a8060-f0ab-480b-92cf-90e9c7c89ca5 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:46.008358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a470ff9-7ed4-4dac-990e-b2a63deebb49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:46.009915 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a470ff9-7ed4-4dac-990e-b2a63deebb49 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:46.069623 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:46.149241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a8060-f0ab-480b-92cf-90e9c7c89ca5 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:46.161566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:46.195526 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 189/372] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:46.196161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a470ff9-7ed4-4dac-990e-b2a63deebb49 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:46.208274 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fec0c8-2e4b-4aec-bab9-ff513f8cde38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:46.209406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fec0c8-2e4b-4aec-bab9-ff513f8cde38 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:46.209910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fec0c8-2e4b-4aec-bab9-ff513f8cde38 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:46.210421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fec0c8-2e4b-4aec-bab9-ff513f8cde38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:46.212216 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fec0c8-2e4b-4aec-bab9-ff513f8cde38 None admin] RBAC: Adding query filter params (group.id=1e903affae424374a9cfedc0a6ff8a4b, role.id=2e3b001c16d24c279be5c68aa3847ca7, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:46.212890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fec0c8-2e4b-4aec-bab9-ff513f8cde38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:46.215413 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fec0c8-2e4b-4aec-bab9-ff513f8cde38 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:46.224386 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 184/373] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:49:46 2022] GET /identity/v3/role_assignments?group.id=1e903affae424374a9cfedc0a6ff8a4b&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:46.225627 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fec0c8-2e4b-4aec-bab9-ff513f8cde38 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:46.394503 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a470ff9-7ed4-4dac-990e-b2a63deebb49 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:46.450262 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce9ac-11b8-45d3-989b-dee76097a938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:46.451174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce9ac-11b8-45d3-989b-dee76097a938 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:46.451468 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce9ac-11b8-45d3-989b-dee76097a938 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:46.451750 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ce9ac-11b8-45d3-989b-dee76097a938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:46.467089 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:46.497435 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fec0c8-2e4b-4aec-bab9-ff513f8cde38 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:46.558586 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ce9ac-11b8-45d3-989b-dee76097a938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:46.559837 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca786c0-951b-451e-a361-5a06ae9a2dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:46.560951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca786c0-951b-451e-a361-5a06ae9a2dfa None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:46.561248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca786c0-951b-451e-a361-5a06ae9a2dfa None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:46.561460 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ce9ac-11b8-45d3-989b-dee76097a938 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:46.561568 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca786c0-951b-451e-a361-5a06ae9a2dfa None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:46.562847 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca786c0-951b-451e-a361-5a06ae9a2dfa None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:46.567238 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca786c0-951b-451e-a361-5a06ae9a2dfa None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:46.569173 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 185/374] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:46 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:46.574847 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca786c0-951b-451e-a361-5a06ae9a2dfa None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:46.636875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:46.678082 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 190/375] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:49:46.686971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce9ac-11b8-45d3-989b-dee76097a938 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:46.750766 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca786c0-951b-451e-a361-5a06ae9a2dfa None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:46.755125 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:46.842961 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791e508-7756-499c-8377-0a3d1f4378bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:46.844313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791e508-7756-499c-8377-0a3d1f4378bf None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:46.844867 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791e508-7756-499c-8377-0a3d1f4378bf None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:46.845927 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791e508-7756-499c-8377-0a3d1f4378bf None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:46.850739 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791e508-7756-499c-8377-0a3d1f4378bf None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:46.852653 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791e508-7756-499c-8377-0a3d1f4378bf None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:46.855314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 186/376] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:46 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:46.861761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791e508-7756-499c-8377-0a3d1f4378bf None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:46.864668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ce9ac-11b8-45d3-989b-dee76097a938 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:46.948114 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d133c-fbe0-4809-ba55-6c655d438503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:46.949040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d133c-fbe0-4809-ba55-6c655d438503 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:46.949333 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d133c-fbe0-4809-ba55-6c655d438503 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:46.949615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d133c-fbe0-4809-ba55-6c655d438503 None admin] PATH_INFO: `/v3/projects/06a03e4df28e472baf1019b7ccaefbc6` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:46.951002 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d133c-fbe0-4809-ba55-6c655d438503 None admin] RBAC: Authorizing `identity:get_project(project_id=06a03e4df28e472baf1019b7ccaefbc6)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:46.952555 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d133c-fbe0-4809-ba55-6c655d438503 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:46.955241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 191/377] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:46 2022] GET /identity/v3/projects/06a03e4df28e472baf1019b7ccaefbc6 => generated 344 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:46.961357 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d133c-fbe0-4809-ba55-6c655d438503 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:46.999331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791e508-7756-499c-8377-0a3d1f4378bf None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:47.013852 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:47.043928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e6194-3a19-4d04-a782-80baae246475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:47.045026 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6194-3a19-4d04-a782-80baae246475 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:47.045800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6194-3a19-4d04-a782-80baae246475 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:47.046883 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e6194-3a19-4d04-a782-80baae246475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:47.048118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e6194-3a19-4d04-a782-80baae246475 None admin] RBAC: Adding query filter params (group.id=1e903affae424374a9cfedc0a6ff8a4b, role.id=4313b1c9df5345778a249b04355b90a0, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:47.048807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e6194-3a19-4d04-a782-80baae246475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:47.051631 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e6194-3a19-4d04-a782-80baae246475 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:47.060198 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 187/378] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:49:46 2022] GET /identity/v3/role_assignments?group.id=1e903affae424374a9cfedc0a6ff8a4b&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=4313b1c9df5345778a249b04355b90a0 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:47.097670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217474b-580f-455b-ab4e-298b68b9892a None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:47.098176 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217474b-580f-455b-ab4e-298b68b9892a None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:47.098679 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217474b-580f-455b-ab4e-298b68b9892a None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:47.109611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 188/379] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:47 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:47.112151 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d186b-73a4-4512-8bb9-5b9f57392fba None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:47.112690 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d186b-73a4-4512-8bb9-5b9f57392fba None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:47.113190 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d186b-73a4-4512-8bb9-5b9f57392fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:47.116840 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d133c-fbe0-4809-ba55-6c655d438503 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:47.131254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:47.161729 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-ec9d186b-73a4-4512-8bb9-5b9f57392fba None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:47.164148 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-ec9d186b-73a4-4512-8bb9-5b9f57392fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:47.188491 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06763ce-f6b4-4450-9a67-d6e5f70a92d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:47.188491 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06763ce-f6b4-4450-9a67-d6e5f70a92d4 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:47.188491 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06763ce-f6b4-4450-9a67-d6e5f70a92d4 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:47.188491 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06763ce-f6b4-4450-9a67-d6e5f70a92d4 None admin] PATH_INFO: `/v3/groups/dd885ee3d0284f3db3c719ca82039a9e` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:47.199230 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06763ce-f6b4-4450-9a67-d6e5f70a92d4 None admin] RBAC: Authorizing `identity:get_group(group_id=dd885ee3d0284f3db3c719ca82039a9e)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:47.200788 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06763ce-f6b4-4450-9a67-d6e5f70a92d4 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:47.211148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 192/380] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:46 2022] GET /identity/v3/groups/dd885ee3d0284f3db3c719ca82039a9e => generated 229 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:47.212609 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06763ce-f6b4-4450-9a67-d6e5f70a92d4 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:47.306276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 189/381] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:47.350705 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d186b-73a4-4512-8bb9-5b9f57392fba None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:47.403259 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06763ce-f6b4-4450-9a67-d6e5f70a92d4 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:47.452294 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24780d-d75f-4337-9db8-4c7767cf18a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:47.452920 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24780d-d75f-4337-9db8-4c7767cf18a5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:47.453210 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24780d-d75f-4337-9db8-4c7767cf18a5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:47.453490 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24780d-d75f-4337-9db8-4c7767cf18a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:47.454346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24780d-d75f-4337-9db8-4c7767cf18a5 None admin] RBAC: Adding query filter params (group.id=dd885ee3d0284f3db3c719ca82039a9e, role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=06a03e4df28e472baf1019b7ccaefbc6) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:47.455202 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24780d-d75f-4337-9db8-4c7767cf18a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:47.456729 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a24780d-d75f-4337-9db8-4c7767cf18a5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:47.468495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 193/382] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:49:47 2022] GET /identity/v3/role_assignments?group.id=dd885ee3d0284f3db3c719ca82039a9e&scope.project.id=06a03e4df28e472baf1019b7ccaefbc6&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:47.499667 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d186b-73a4-4512-8bb9-5b9f57392fba None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:47.502706 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:47.548368 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a9b4a-f2b9-497a-bbfb-eab061b16746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:47.549501 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a9b4a-f2b9-497a-bbfb-eab061b16746 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:47.549955 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a9b4a-f2b9-497a-bbfb-eab061b16746 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:47.550435 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a9b4a-f2b9-497a-bbfb-eab061b16746 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:47.574826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a9b4a-f2b9-497a-bbfb-eab061b16746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-774a9b4a-f2b9-497a-bbfb-eab061b16746 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:47.583698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:47.589072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:47.590089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:47.592698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 190/383] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:47.601526 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24780d-d75f-4337-9db8-4c7767cf18a5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:47.651321 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:47.735574 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a24780d-d75f-4337-9db8-4c7767cf18a5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:47.776289 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3a0fd-4a39-4a52-9e20-c89616543f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:47.777019 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3a0fd-4a39-4a52-9e20-c89616543f60 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:47.777308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3a0fd-4a39-4a52-9e20-c89616543f60 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:47.777590 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3a0fd-4a39-4a52-9e20-c89616543f60 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:47.778377 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3a0fd-4a39-4a52-9e20-c89616543f60 None admin] RBAC: Adding query filter params (name=member) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:47.778691 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3a0fd-4a39-4a52-9e20-c89616543f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:47.780148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3a0fd-4a39-4a52-9e20-c89616543f60 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:47.795276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 194/384] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:47.796766 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a9b4a-f2b9-497a-bbfb-eab061b16746 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:47.979001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a9b4a-f2b9-497a-bbfb-eab061b16746 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:48.037045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3e4a8-9c42-4cad-8587-36e97abdf9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:48.037995 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3e4a8-9c42-4cad-8587-36e97abdf9bb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:48.038370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3e4a8-9c42-4cad-8587-36e97abdf9bb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:48.038551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3e4a8-9c42-4cad-8587-36e97abdf9bb None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:48.045307 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3e4a8-9c42-4cad-8587-36e97abdf9bb None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:48.046814 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3e4a8-9c42-4cad-8587-36e97abdf9bb None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:48.049185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 191/385] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:47 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:48.057568 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3a0fd-4a39-4a52-9e20-c89616543f60 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:48.179157 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:48.183896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3a0fd-4a39-4a52-9e20-c89616543f60 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:48.228368 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bb354-a13e-4696-ad51-cc85cde4f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:48.229274 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bb354-a13e-4696-ad51-cc85cde4f90c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:48.229562 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bb354-a13e-4696-ad51-cc85cde4f90c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:48.229841 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bb354-a13e-4696-ad51-cc85cde4f90c None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:48.231507 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bb354-a13e-4696-ad51-cc85cde4f90c None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:48.232819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146bb354-a13e-4696-ad51-cc85cde4f90c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:48.236030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 195/386] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:48 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:48.240973 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3e4a8-9c42-4cad-8587-36e97abdf9bb None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:48.291041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:48.391652 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3e4a8-9c42-4cad-8587-36e97abdf9bb None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:48.441129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ea0a8e-ed66-4a32-a2f7-735e4f3c9f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:48.442268 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea0a8e-ed66-4a32-a2f7-735e4f3c9f35 None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:48.442645 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea0a8e-ed66-4a32-a2f7-735e4f3c9f35 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:48.442957 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea0a8e-ed66-4a32-a2f7-735e4f3c9f35 None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d/groups/1e903affae424374a9cfedc0a6ff8a4b/roles/2e3b001c16d24c279be5c68aa3847ca7` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:48.449920 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea0a8e-ed66-4a32-a2f7-735e4f3c9f35 None admin] RBAC: Authorizing `identity:create_grant(project_id=834a091f1d39471ba13d3156d015bf3d, group_id=1e903affae424374a9cfedc0a6ff8a4b, role_id=2e3b001c16d24c279be5c68aa3847ca7)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:48.451416 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ea0a8e-ed66-4a32-a2f7-735e4f3c9f35 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:48.534092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 192/387] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:49:48 2022] PUT /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d/groups/1e903affae424374a9cfedc0a6ff8a4b/roles/2e3b001c16d24c279be5c68aa3847ca7 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:48.600416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c20d34-8a8a-437f-9e50-39c3f50e6c8d None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:48.601055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c20d34-8a8a-437f-9e50-39c3f50e6c8d None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:48.601568 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c20d34-8a8a-437f-9e50-39c3f50e6c8d None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:48.603795 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 196/388] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:48.610224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9ce86-cbf3-450f-9c97-b71178440a26 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:48.610955 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9ce86-cbf3-450f-9c97-b71178440a26 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:48.611543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f9ce86-cbf3-450f-9c97-b71178440a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:48.637704 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-08f9ce86-cbf3-450f-9c97-b71178440a26 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:48.639521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-08f9ce86-cbf3-450f-9c97-b71178440a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:48.752671 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 193/389] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:48.794341 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c20d34-8a8a-437f-9e50-39c3f50e6c8d None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:48.945628 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c20d34-8a8a-437f-9e50-39c3f50e6c8d None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:48.993171 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e8dd1-e02a-4fe7-ae06-12a6f290a251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:48.993991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e8dd1-e02a-4fe7-ae06-12a6f290a251 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:48.994291 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e8dd1-e02a-4fe7-ae06-12a6f290a251 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:48.994544 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e8dd1-e02a-4fe7-ae06-12a6f290a251 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:49.008443 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e8dd1-e02a-4fe7-ae06-12a6f290a251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-a27e8dd1-e02a-4fe7-ae06-12a6f290a251 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:49.016916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:49.018030 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:49.019253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 197/390] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:49.024041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9ce86-cbf3-450f-9c97-b71178440a26 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:49.071156 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:49.184757 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f9ce86-cbf3-450f-9c97-b71178440a26 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:49.232436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5579a59-3ff6-422e-aff3-a9ae05023b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:49.233518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5579a59-3ff6-422e-aff3-a9ae05023b8c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:49.234011 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5579a59-3ff6-422e-aff3-a9ae05023b8c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:49.234509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5579a59-3ff6-422e-aff3-a9ae05023b8c None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:49.235592 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5579a59-3ff6-422e-aff3-a9ae05023b8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:49.236752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5579a59-3ff6-422e-aff3-a9ae05023b8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:49.238447 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5579a59-3ff6-422e-aff3-a9ae05023b8c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:49.251074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 194/391] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:49.256412 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e8dd1-e02a-4fe7-ae06-12a6f290a251 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:49.297260 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:49.399192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e8dd1-e02a-4fe7-ae06-12a6f290a251 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:49.443401 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee59e474-11e0-4973-a815-deb1c5ce373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:49.444194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59e474-11e0-4973-a815-deb1c5ce373f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:49.444583 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59e474-11e0-4973-a815-deb1c5ce373f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:49.444860 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee59e474-11e0-4973-a815-deb1c5ce373f None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:49.446084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59e474-11e0-4973-a815-deb1c5ce373f None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:49.447758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee59e474-11e0-4973-a815-deb1c5ce373f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:49.449791 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 198/392] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:49 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:49.455006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5579a59-3ff6-422e-aff3-a9ae05023b8c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:49.491063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431650c3-775e-42ed-9f37-959af902d90f None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:49.491385 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431650c3-775e-42ed-9f37-959af902d90f None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:49.491630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431650c3-775e-42ed-9f37-959af902d90f None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:49.493304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 199/393] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:49:49.514037 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eb34b-124a-4e74-a7d0-6e80bc020fee None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:49.514388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eb34b-124a-4e74-a7d0-6e80bc020fee None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:49.514596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eb34b-124a-4e74-a7d0-6e80bc020fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:49.541975 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-f10eb34b-124a-4e74-a7d0-6e80bc020fee None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:49.543179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-f10eb34b-124a-4e74-a7d0-6e80bc020fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:49.554960 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:49.587012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5579a59-3ff6-422e-aff3-a9ae05023b8c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:49.663655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf455c-ec60-4fbe-8f4b-4bbe4417b7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:49.664961 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf455c-ec60-4fbe-8f4b-4bbe4417b7dd None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:49.666423 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf455c-ec60-4fbe-8f4b-4bbe4417b7dd None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:49.666921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf455c-ec60-4fbe-8f4b-4bbe4417b7dd None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:49.670083 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf455c-ec60-4fbe-8f4b-4bbe4417b7dd None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:49.673144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf455c-ec60-4fbe-8f4b-4bbe4417b7dd None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:49.679334 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 195/394] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:49 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:49.684958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf455c-ec60-4fbe-8f4b-4bbe4417b7dd None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:49.687666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 200/395] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:49.734476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eb34b-124a-4e74-a7d0-6e80bc020fee None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:49.789800 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:49.809526 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf455c-ec60-4fbe-8f4b-4bbe4417b7dd None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:49.895995 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6bcbd-49d2-43a8-91fd-7d7bc4b6ef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:49.897379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6bcbd-49d2-43a8-91fd-7d7bc4b6ef22 None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:49.898580 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6bcbd-49d2-43a8-91fd-7d7bc4b6ef22 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:49.899091 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6bcbd-49d2-43a8-91fd-7d7bc4b6ef22 None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d/groups/1e903affae424374a9cfedc0a6ff8a4b/roles/4313b1c9df5345778a249b04355b90a0` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:49.901866 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6bcbd-49d2-43a8-91fd-7d7bc4b6ef22 None admin] RBAC: Authorizing `identity:create_grant(project_id=834a091f1d39471ba13d3156d015bf3d, group_id=1e903affae424374a9cfedc0a6ff8a4b, role_id=4313b1c9df5345778a249b04355b90a0)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:49.903617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6bcbd-49d2-43a8-91fd-7d7bc4b6ef22 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:49.944090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eb34b-124a-4e74-a7d0-6e80bc020fee None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:49.981007 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6cb93-4cbe-4341-81dd-d62a3697a9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:49.981872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6cb93-4cbe-4341-81dd-d62a3697a9d0 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:49.982161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6cb93-4cbe-4341-81dd-d62a3697a9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:49.982441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6cb93-4cbe-4341-81dd-d62a3697a9d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:49.993962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec6cb93-4cbe-4341-81dd-d62a3697a9d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:49.996394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 196/396] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:49:49 2022] PUT /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d/groups/1e903affae424374a9cfedc0a6ff8a4b/roles/4313b1c9df5345778a249b04355b90a0 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-aec6cb93-4cbe-4341-81dd-d62a3697a9d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:50.002823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:50.004022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:50.005079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:50.008046 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 201/397] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:49:50.010748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6bcbd-49d2-43a8-91fd-7d7bc4b6ef22 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:50.029619 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:50.059820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0424a2a-b978-4bb7-ba45-6b69aab38b9c None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:50.060178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0424a2a-b978-4bb7-ba45-6b69aab38b9c None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:50.060514 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0424a2a-b978-4bb7-ba45-6b69aab38b9c None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:50.062119 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 202/398] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:49:50.070037 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7064e31-9269-4c8e-86bb-273a4fe0b99d None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:50.070503 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7064e31-9269-4c8e-86bb-273a4fe0b99d None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:50.070993 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7064e31-9269-4c8e-86bb-273a4fe0b99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:50.103015 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-e7064e31-9269-4c8e-86bb-273a4fe0b99d None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:50.105129 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-e7064e31-9269-4c8e-86bb-273a4fe0b99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:50.230812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6bcbd-49d2-43a8-91fd-7d7bc4b6ef22 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:50.268361 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 203/399] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:50.286437 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5495d-b1b6-461d-97a8-80f53cc6b442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:50.287141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5495d-b1b6-461d-97a8-80f53cc6b442 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:50.287425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5495d-b1b6-461d-97a8-80f53cc6b442 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:50.287704 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5495d-b1b6-461d-97a8-80f53cc6b442 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:50.288593 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5495d-b1b6-461d-97a8-80f53cc6b442 None admin] RBAC: Adding query filter params (name=member) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:50.288907 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5495d-b1b6-461d-97a8-80f53cc6b442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:50.290348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5495d-b1b6-461d-97a8-80f53cc6b442 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:50.300953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 197/400] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:50.306756 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7064e31-9269-4c8e-86bb-273a4fe0b99d None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:50.318218 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5495d-b1b6-461d-97a8-80f53cc6b442 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:50.329645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:50.431236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7064e31-9269-4c8e-86bb-273a4fe0b99d None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:50.486769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e16f2-6f59-402c-b952-569de65a2fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:50.487644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e16f2-6f59-402c-b952-569de65a2fd1 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:50.487928 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e16f2-6f59-402c-b952-569de65a2fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:50.488280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e16f2-6f59-402c-b952-569de65a2fd1 None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:50.490331 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e16f2-6f59-402c-b952-569de65a2fd1 None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:50.491831 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e16f2-6f59-402c-b952-569de65a2fd1 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:50.497430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 204/401] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:50 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:50.505660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e16f2-6f59-402c-b952-569de65a2fd1 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:50.521306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:50.573443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5495d-b1b6-461d-97a8-80f53cc6b442 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:50.621994 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c48a39-23e5-4ed5-9778-e21ef6a8a31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:50.623387 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48a39-23e5-4ed5-9778-e21ef6a8a31d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:50.623786 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48a39-23e5-4ed5-9778-e21ef6a8a31d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:50.624170 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48a39-23e5-4ed5-9778-e21ef6a8a31d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:50.630043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:50.640893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c48a39-23e5-4ed5-9778-e21ef6a8a31d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:50.650103 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e16f2-6f59-402c-b952-569de65a2fd1 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-c0c48a39-23e5-4ed5-9778-e21ef6a8a31d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:50.651529 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:50.652608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:50.654255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 198/402] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:50.655045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b24a3-6ea3-4d23-bb34-b42e05e75a61 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:50.655045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b24a3-6ea3-4d23-bb34-b42e05e75a61 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:50.655192 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b24a3-6ea3-4d23-bb34-b42e05e75a61 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:50.656731 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 199/403] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:50.662253 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b24a3-6ea3-4d23-bb34-b42e05e75a61 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:50.699476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9de88-bbc2-4130-8b49-fe186d51f588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:50.700976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9de88-bbc2-4130-8b49-fe186d51f588 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:50.701530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9de88-bbc2-4130-8b49-fe186d51f588 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:50.702063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9de88-bbc2-4130-8b49-fe186d51f588 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:50.703766 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9de88-bbc2-4130-8b49-fe186d51f588 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:50.706174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9de88-bbc2-4130-8b49-fe186d51f588 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:50.708833 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 205/404] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:50 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:50.710639 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a620667-40ca-4c06-a6fe-7e64299242f1 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:50.711204 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a620667-40ca-4c06-a6fe-7e64299242f1 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:50.711805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a620667-40ca-4c06-a6fe-7e64299242f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:50.732941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-4a620667-40ca-4c06-a6fe-7e64299242f1 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:50.734589 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-4a620667-40ca-4c06-a6fe-7e64299242f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:50.793653 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b24a3-6ea3-4d23-bb34-b42e05e75a61 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:50.813002 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 206/405] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:50.815526 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a620667-40ca-4c06-a6fe-7e64299242f1 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:50.852072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac7176-d52d-44b7-840d-bbba61d1446a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:50.852876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac7176-d52d-44b7-840d-bbba61d1446a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:50.853165 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac7176-d52d-44b7-840d-bbba61d1446a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:50.853447 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac7176-d52d-44b7-840d-bbba61d1446a None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:50.854346 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac7176-d52d-44b7-840d-bbba61d1446a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:50.854669 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac7176-d52d-44b7-840d-bbba61d1446a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:50.856492 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac7176-d52d-44b7-840d-bbba61d1446a None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:50.858518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:50.872213 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 200/406] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:50.872213 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac7176-d52d-44b7-840d-bbba61d1446a None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:50.954815 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:50.985671 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac7176-d52d-44b7-840d-bbba61d1446a None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:50.993489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a620667-40ca-4c06-a6fe-7e64299242f1 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:51.026298 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09eea3-db49-4c5a-b286-9d72bc50f3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:51.027128 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09eea3-db49-4c5a-b286-9d72bc50f3a7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:51.027514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09eea3-db49-4c5a-b286-9d72bc50f3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:51.027804 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09eea3-db49-4c5a-b286-9d72bc50f3a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:51.034544 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76afb57-c9dc-4898-8c58-8bca2d45d9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:51.035733 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76afb57-c9dc-4898-8c58-8bca2d45d9ed None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:51.036290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76afb57-c9dc-4898-8c58-8bca2d45d9ed None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:51.036769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76afb57-c9dc-4898-8c58-8bca2d45d9ed None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/groups/1e903affae424374a9cfedc0a6ff8a4b/roles/2e3b001c16d24c279be5c68aa3847ca7` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:51.039223 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76afb57-c9dc-4898-8c58-8bca2d45d9ed None admin] RBAC: Authorizing `identity:create_grant(project_id=7e20e05ff6d243f3ab00b511081c5bf2, group_id=1e903affae424374a9cfedc0a6ff8a4b, role_id=2e3b001c16d24c279be5c68aa3847ca7)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:51.040493 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09eea3-db49-4c5a-b286-9d72bc50f3a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:51.041228 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76afb57-c9dc-4898-8c58-8bca2d45d9ed None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-ec09eea3-db49-4c5a-b286-9d72bc50f3a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:51.049643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:51.050819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:51.051925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:51.056022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 201/407] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:49:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:51.056969 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09eea3-db49-4c5a-b286-9d72bc50f3a7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:51.131169 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:51.162274 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 207/408] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:49:50 2022] PUT /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/groups/1e903affae424374a9cfedc0a6ff8a4b/roles/2e3b001c16d24c279be5c68aa3847ca7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:51.169923 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76afb57-c9dc-4898-8c58-8bca2d45d9ed None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:51.222026 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09eea3-db49-4c5a-b286-9d72bc50f3a7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:51.312579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9c2af-38c9-4357-927c-939581d3ea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:51.314111 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9c2af-38c9-4357-927c-939581d3ea59 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:51.314625 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9c2af-38c9-4357-927c-939581d3ea59 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:51.315140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9c2af-38c9-4357-927c-939581d3ea59 None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:51.321062 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9c2af-38c9-4357-927c-939581d3ea59 None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:51.327754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9c2af-38c9-4357-927c-939581d3ea59 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:51.341363 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 202/409] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:51 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:51.352459 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9c2af-38c9-4357-927c-939581d3ea59 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:51.459952 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76afb57-c9dc-4898-8c58-8bca2d45d9ed None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:51.514214 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f977c-195e-4bc0-8e5d-ab35939c4c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:51.514913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f977c-195e-4bc0-8e5d-ab35939c4c0c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:51.515370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f977c-195e-4bc0-8e5d-ab35939c4c0c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:51.515654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f977c-195e-4bc0-8e5d-ab35939c4c0c None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:51.516868 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f977c-195e-4bc0-8e5d-ab35939c4c0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:51.517785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f977c-195e-4bc0-8e5d-ab35939c4c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:51.519564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f977c-195e-4bc0-8e5d-ab35939c4c0c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:51.525327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9c2af-38c9-4357-927c-939581d3ea59 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:51.531816 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 208/410] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:51.538437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183a00b-798d-4f10-9dda-3892a4d4b756 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:51.538437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183a00b-798d-4f10-9dda-3892a4d4b756 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:51.538437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183a00b-798d-4f10-9dda-3892a4d4b756 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:51.538437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 209/411] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:51.542649 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183a00b-798d-4f10-9dda-3892a4d4b756 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:51.546200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:51.571066 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:51.584107 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a2cd2-449e-414f-aaf6-8f500766e479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:51.585189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a2cd2-449e-414f-aaf6-8f500766e479 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:51.585581 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a2cd2-449e-414f-aaf6-8f500766e479 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:51.585967 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a2cd2-449e-414f-aaf6-8f500766e479 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:51.587426 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a2cd2-449e-414f-aaf6-8f500766e479 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:51.589122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a2cd2-449e-414f-aaf6-8f500766e479 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:51.591397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 203/412] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:51 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:51.592794 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3dcee-a258-42b3-b81f-a679bdb7bfa3 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:51.593187 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3dcee-a258-42b3-b81f-a679bdb7bfa3 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:51.593570 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a3dcee-a258-42b3-b81f-a679bdb7bfa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:51.615441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-13a3dcee-a258-42b3-b81f-a679bdb7bfa3 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:51.617284 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-13a3dcee-a258-42b3-b81f-a679bdb7bfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:51.709181 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183a00b-798d-4f10-9dda-3892a4d4b756 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:51.733036 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 204/413] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:51.733567 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3dcee-a258-42b3-b81f-a679bdb7bfa3 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:51.772054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe4ca3-039b-4b67-b270-ad47865e3a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:51.773087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe4ca3-039b-4b67-b270-ad47865e3a1e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:51.773775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe4ca3-039b-4b67-b270-ad47865e3a1e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:51.773775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe4ca3-039b-4b67-b270-ad47865e3a1e None admin] PATH_INFO: `/v3/groups/dd885ee3d0284f3db3c719ca82039a9e` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:51.774979 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe4ca3-039b-4b67-b270-ad47865e3a1e None admin] RBAC: Authorizing `identity:get_group(group_id=dd885ee3d0284f3db3c719ca82039a9e)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:51.776501 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe4ca3-039b-4b67-b270-ad47865e3a1e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:51.778522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 210/414] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:51 2022] GET /identity/v3/groups/dd885ee3d0284f3db3c719ca82039a9e => generated 229 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:51.779316 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe4ca3-039b-4b67-b270-ad47865e3a1e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:51.865992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:51.868617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a3dcee-a258-42b3-b81f-a679bdb7bfa3 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:51.895315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:51.900965 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe4ca3-039b-4b67-b270-ad47865e3a1e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:51.908424 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52c456-66f7-45a7-aea3-44b1d4fa51b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:51.909335 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52c456-66f7-45a7-aea3-44b1d4fa51b3 None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:51.909924 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52c456-66f7-45a7-aea3-44b1d4fa51b3 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:51.910212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52c456-66f7-45a7-aea3-44b1d4fa51b3 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/groups/1e903affae424374a9cfedc0a6ff8a4b/roles/4313b1c9df5345778a249b04355b90a0` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:51.912921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52c456-66f7-45a7-aea3-44b1d4fa51b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e20e05ff6d243f3ab00b511081c5bf2, group_id=1e903affae424374a9cfedc0a6ff8a4b, role_id=4313b1c9df5345778a249b04355b90a0)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:51.915066 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52c456-66f7-45a7-aea3-44b1d4fa51b3 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:51.951941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf80991-77cc-4daa-81ee-4fe99673c373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:51.953070 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf80991-77cc-4daa-81ee-4fe99673c373 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:51.953538 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf80991-77cc-4daa-81ee-4fe99673c373 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:51.953985 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf80991-77cc-4daa-81ee-4fe99673c373 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:51.966820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf80991-77cc-4daa-81ee-4fe99673c373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-bbf80991-77cc-4daa-81ee-4fe99673c373 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:51.975328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:51.976549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:51.977712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:51.980964 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 211/415] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:51.982131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf80991-77cc-4daa-81ee-4fe99673c373 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:52.028601 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 205/416] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:49:51 2022] PUT /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2/groups/1e903affae424374a9cfedc0a6ff8a4b/roles/4313b1c9df5345778a249b04355b90a0 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:52.029951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52c456-66f7-45a7-aea3-44b1d4fa51b3 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:52.128007 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf80991-77cc-4daa-81ee-4fe99673c373 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:52.172176 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:52.180490 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6e6b3-6af8-45d7-b2a5-d14c804241d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:52.181662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6e6b3-6af8-45d7-b2a5-d14c804241d1 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:52.182123 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6e6b3-6af8-45d7-b2a5-d14c804241d1 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:52.182564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a6e6b3-6af8-45d7-b2a5-d14c804241d1 None admin] PATH_INFO: `/v3/projects/06a03e4df28e472baf1019b7ccaefbc6` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:52.185067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6e6b3-6af8-45d7-b2a5-d14c804241d1 None admin] RBAC: Authorizing `identity:get_project(project_id=06a03e4df28e472baf1019b7ccaefbc6)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:52.187051 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a6e6b3-6af8-45d7-b2a5-d14c804241d1 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:52.190425 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 212/417] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:51 2022] GET /identity/v3/projects/06a03e4df28e472baf1019b7ccaefbc6 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:52.198237 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6e6b3-6af8-45d7-b2a5-d14c804241d1 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:52.281823 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52c456-66f7-45a7-aea3-44b1d4fa51b3 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:52.348322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfaa136-a330-49d8-a67b-da10893c7f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:52.350040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfaa136-a330-49d8-a67b-da10893c7f43 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:52.350040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfaa136-a330-49d8-a67b-da10893c7f43 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:52.351900 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfaa136-a330-49d8-a67b-da10893c7f43 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:52.353467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfaa136-a330-49d8-a67b-da10893c7f43 None admin] RBAC: Adding query filter params (name=member) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:52.354045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfaa136-a330-49d8-a67b-da10893c7f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:52.356999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfaa136-a330-49d8-a67b-da10893c7f43 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:52.374914 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 206/418] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:52.386315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfaa136-a330-49d8-a67b-da10893c7f43 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:52.413714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a6e6b3-6af8-45d7-b2a5-d14c804241d1 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:52.461542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:52.487148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13da1b95-84f2-49d4-b075-6c2ce3df6500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:52.490618 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da1b95-84f2-49d4-b075-6c2ce3df6500 None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:52.491563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da1b95-84f2-49d4-b075-6c2ce3df6500 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:52.492415 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da1b95-84f2-49d4-b075-6c2ce3df6500 None admin] PATH_INFO: `/v3/projects/06a03e4df28e472baf1019b7ccaefbc6/groups/dd885ee3d0284f3db3c719ca82039a9e/roles/ed68c2195de147c0a30d9150b33e47cf` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:52.497318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da1b95-84f2-49d4-b075-6c2ce3df6500 None admin] RBAC: Authorizing `identity:create_grant(project_id=06a03e4df28e472baf1019b7ccaefbc6, group_id=dd885ee3d0284f3db3c719ca82039a9e, role_id=ed68c2195de147c0a30d9150b33e47cf)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:52.500387 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13da1b95-84f2-49d4-b075-6c2ce3df6500 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:52.554637 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfaa136-a330-49d8-a67b-da10893c7f43 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:52.592196 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829709af-c2e6-49d2-9466-771e20bdb43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:52.592983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829709af-c2e6-49d2-9466-771e20bdb43b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:52.593343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829709af-c2e6-49d2-9466-771e20bdb43b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:52.593628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829709af-c2e6-49d2-9466-771e20bdb43b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:52.613011 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 213/419] 10.222.0.88 () {58 vars in 1442 bytes} [Thu May 19 20:49:52 2022] PUT /identity/v3/projects/06a03e4df28e472baf1019b7ccaefbc6/groups/dd885ee3d0284f3db3c719ca82039a9e/roles/ed68c2195de147c0a30d9150b33e47cf => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:49:52.664784 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264e9e6-a4df-44e4-b3d9-d856d42c72c9 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:52.665703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264e9e6-a4df-44e4-b3d9-d856d42c72c9 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:52.666545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264e9e6-a4df-44e4-b3d9-d856d42c72c9 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:52.672403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 214/420] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:52.675049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829709af-c2e6-49d2-9466-771e20bdb43b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:52.676872 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829709af-c2e6-49d2-9466-771e20bdb43b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:52.678032 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552089f-3656-48e8-9149-590d5e76e0bd None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:52.678329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552089f-3656-48e8-9149-590d5e76e0bd None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:52.678618 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552089f-3656-48e8-9149-590d5e76e0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:52.723141 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-9552089f-3656-48e8-9149-590d5e76e0bd None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:52.728679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-9552089f-3656-48e8-9149-590d5e76e0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:52.786450 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:52.861490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 207/421] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:49:52.871300 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829709af-c2e6-49d2-9466-771e20bdb43b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:52.878801 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:52.880976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 215/422] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:52.910063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552089f-3656-48e8-9149-590d5e76e0bd None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:52.993297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829709af-c2e6-49d2-9466-771e20bdb43b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:53.029823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552089f-3656-48e8-9149-590d5e76e0bd None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:53.048671 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a42df-7185-4540-bb9c-e6595888fa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:53.050579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a42df-7185-4540-bb9c-e6595888fa2a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:53.051281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a42df-7185-4540-bb9c-e6595888fa2a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:53.052029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a42df-7185-4540-bb9c-e6595888fa2a None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:53.054390 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a42df-7185-4540-bb9c-e6595888fa2a None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:53.057535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a42df-7185-4540-bb9c-e6595888fa2a None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:53.061768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 208/423] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:52 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:53.070034 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a42df-7185-4540-bb9c-e6595888fa2a None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:53.070563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c3c4b-582a-4d0b-8022-260d40c5c50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:53.076379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c3c4b-582a-4d0b-8022-260d40c5c50c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:53.076379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c3c4b-582a-4d0b-8022-260d40c5c50c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:53.076379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c3c4b-582a-4d0b-8022-260d40c5c50c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:53.083808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c3c4b-582a-4d0b-8022-260d40c5c50c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-b84c3c4b-582a-4d0b-8022-260d40c5c50c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:53.092552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:53.094615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:53.096489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 216/424] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:53.101029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c3c4b-582a-4d0b-8022-260d40c5c50c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:53.113542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:53.156749 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:53.213111 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c3c4b-582a-4d0b-8022-260d40c5c50c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:53.250319 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a798e-5d03-44f1-bd32-a4423e89ad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:53.251009 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a798e-5d03-44f1-bd32-a4423e89ad26 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:53.251870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a798e-5d03-44f1-bd32-a4423e89ad26 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:53.252164 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a798e-5d03-44f1-bd32-a4423e89ad26 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:53.252965 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a798e-5d03-44f1-bd32-a4423e89ad26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:53.253276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a798e-5d03-44f1-bd32-a4423e89ad26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:53.254687 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a798e-5d03-44f1-bd32-a4423e89ad26 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:53.264453 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 217/425] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:53.270986 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a798e-5d03-44f1-bd32-a4423e89ad26 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:53.367450 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a42df-7185-4540-bb9c-e6595888fa2a None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:53.433883 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a798e-5d03-44f1-bd32-a4423e89ad26 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:53.438868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6956a-cfeb-4f82-824f-8f2ab75e6238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:53.440652 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6956a-cfeb-4f82-824f-8f2ab75e6238 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:53.441201 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6956a-cfeb-4f82-824f-8f2ab75e6238 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:53.441692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6956a-cfeb-4f82-824f-8f2ab75e6238 None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:53.449218 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6956a-cfeb-4f82-824f-8f2ab75e6238 None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:53.450758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6956a-cfeb-4f82-824f-8f2ab75e6238 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:53.455923 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 209/426] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:53 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:53.458729 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6956a-cfeb-4f82-824f-8f2ab75e6238 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:53.465302 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:53.513459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950a87b-f877-4a2b-a44c-56859ea2117e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:53.514116 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a87b-f877-4a2b-a44c-56859ea2117e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:53.514388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a87b-f877-4a2b-a44c-56859ea2117e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:53.514664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a87b-f877-4a2b-a44c-56859ea2117e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:53.636730 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:53.683784 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6956a-cfeb-4f82-824f-8f2ab75e6238 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:53.687685 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950a87b-f877-4a2b-a44c-56859ea2117e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:53.691079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950a87b-f877-4a2b-a44c-56859ea2117e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:53.761698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e86f7c-5f4f-4153-a5a7-26388421bb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:53.762677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e86f7c-5f4f-4153-a5a7-26388421bb97 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:53.763110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e86f7c-5f4f-4153-a5a7-26388421bb97 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:53.763627 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e86f7c-5f4f-4153-a5a7-26388421bb97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:53.766074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e86f7c-5f4f-4153-a5a7-26388421bb97 None admin] RBAC: Adding query filter params (group.id=1e903affae424374a9cfedc0a6ff8a4b, role.id=2e3b001c16d24c279be5c68aa3847ca7, scope.project.id=834a091f1d39471ba13d3156d015bf3d) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:53.766870 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e86f7c-5f4f-4153-a5a7-26388421bb97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:53.770217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e86f7c-5f4f-4153-a5a7-26388421bb97 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:53.783675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 210/427] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:49:53 2022] GET /identity/v3/role_assignments?group.id=1e903affae424374a9cfedc0a6ff8a4b&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 611 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:53.854646 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6877f-c58b-43e6-abdb-7e84198e607f None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:53.855355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6877f-c58b-43e6-abdb-7e84198e607f None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:53.855856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6877f-c58b-43e6-abdb-7e84198e607f None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:53.857975 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 211/428] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:49:53.864193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41916a5-fdeb-404e-bac3-1957d515ef02 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:53.864977 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41916a5-fdeb-404e-bac3-1957d515ef02 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:53.865507 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41916a5-fdeb-404e-bac3-1957d515ef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:53.884521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-e41916a5-fdeb-404e-bac3-1957d515ef02 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:53.886186 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-e41916a5-fdeb-404e-bac3-1957d515ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:53.924552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 218/429] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:49:53.933958 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950a87b-f877-4a2b-a44c-56859ea2117e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:53.962528 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:53.996306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 212/430] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:54.037681 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41916a5-fdeb-404e-bac3-1957d515ef02 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:54.115825 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950a87b-f877-4a2b-a44c-56859ea2117e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:54.146733 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:54.158339 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c06ae2-4d71-4d11-8a9e-30b9ffc0619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:54.163953 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c06ae2-4d71-4d11-8a9e-30b9ffc0619c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:54.163953 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c06ae2-4d71-4d11-8a9e-30b9ffc0619c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:54.163953 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c06ae2-4d71-4d11-8a9e-30b9ffc0619c None admin] PATH_INFO: `/v3/projects/834a091f1d39471ba13d3156d015bf3d` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:54.163953 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c06ae2-4d71-4d11-8a9e-30b9ffc0619c None admin] RBAC: Authorizing `identity:get_project(project_id=834a091f1d39471ba13d3156d015bf3d)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:54.163953 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c06ae2-4d71-4d11-8a9e-30b9ffc0619c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:54.166610 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 219/431] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:53 2022] GET /identity/v3/projects/834a091f1d39471ba13d3156d015bf3d => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:54.173010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c06ae2-4d71-4d11-8a9e-30b9ffc0619c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:54.186681 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41916a5-fdeb-404e-bac3-1957d515ef02 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:54.246610 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028e4b0-c445-4f74-9d58-ac8e4ff468a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:54.248463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028e4b0-c445-4f74-9d58-ac8e4ff468a3 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:54.249121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028e4b0-c445-4f74-9d58-ac8e4ff468a3 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:54.249599 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028e4b0-c445-4f74-9d58-ac8e4ff468a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:54.264637 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028e4b0-c445-4f74-9d58-ac8e4ff468a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-7028e4b0-c445-4f74-9d58-ac8e4ff468a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:54.275297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:54.276564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 19 20:49:54.277582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:54.280584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 213/432] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:54.282901 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:54.286332 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028e4b0-c445-4f74-9d58-ac8e4ff468a3 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:54.309222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c06ae2-4d71-4d11-8a9e-30b9ffc0619c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:54.363299 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7508e-2803-48ea-a46b-44e4d1e023cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:54.364173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7508e-2803-48ea-a46b-44e4d1e023cf None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:54.364499 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7508e-2803-48ea-a46b-44e4d1e023cf None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:54.364775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7508e-2803-48ea-a46b-44e4d1e023cf None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:54.369156 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7508e-2803-48ea-a46b-44e4d1e023cf None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:54.370646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7508e-2803-48ea-a46b-44e4d1e023cf None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:54.375721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 220/433] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:54 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:54.380961 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7508e-2803-48ea-a46b-44e4d1e023cf None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:54.438609 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028e4b0-c445-4f74-9d58-ac8e4ff468a3 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:54.487958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb432f-bbb3-4c66-b25c-c413873a41bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:54.488827 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb432f-bbb3-4c66-b25c-c413873a41bf None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:54.489117 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb432f-bbb3-4c66-b25c-c413873a41bf None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:54.489398 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb432f-bbb3-4c66-b25c-c413873a41bf None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:54.490164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb432f-bbb3-4c66-b25c-c413873a41bf None admin] RBAC: Adding query filter params (name=member) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:54.490588 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb432f-bbb3-4c66-b25c-c413873a41bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:54.492113 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb432f-bbb3-4c66-b25c-c413873a41bf None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:54.497421 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7508e-2803-48ea-a46b-44e4d1e023cf None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:54.500123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:54.502810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 214/434] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:49:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:54.508844 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb432f-bbb3-4c66-b25c-c413873a41bf None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:54.541260 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7d452-4f0c-47d0-a2c9-613c6c0f2a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:54.541965 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7d452-4f0c-47d0-a2c9-613c6c0f2a42 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:54.542208 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7d452-4f0c-47d0-a2c9-613c6c0f2a42 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:54.542486 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7d452-4f0c-47d0-a2c9-613c6c0f2a42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:54.543403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7d452-4f0c-47d0-a2c9-613c6c0f2a42 None admin] RBAC: Adding query filter params (group.id=1e903affae424374a9cfedc0a6ff8a4b, role.id=4313b1c9df5345778a249b04355b90a0, scope.project.id=834a091f1d39471ba13d3156d015bf3d) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:54.543717 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7d452-4f0c-47d0-a2c9-613c6c0f2a42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:54.545284 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7d452-4f0c-47d0-a2c9-613c6c0f2a42 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:54.554114 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 221/435] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:49:54 2022] GET /identity/v3/role_assignments?group.id=1e903affae424374a9cfedc0a6ff8a4b&scope.project.id=834a091f1d39471ba13d3156d015bf3d&role.id=4313b1c9df5345778a249b04355b90a0 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:54.635675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb432f-bbb3-4c66-b25c-c413873a41bf None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:54.679401 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff27e4-0bc6-4bd3-9230-8bb1e05a15ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:54.680469 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff27e4-0bc6-4bd3-9230-8bb1e05a15ec None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:54.680941 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff27e4-0bc6-4bd3-9230-8bb1e05a15ec None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:54.681387 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff27e4-0bc6-4bd3-9230-8bb1e05a15ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:54.762850 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:54.773173 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff27e4-0bc6-4bd3-9230-8bb1e05a15ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:54.774934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff27e4-0bc6-4bd3-9230-8bb1e05a15ec None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:54.904803 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 215/436] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:49:54.911325 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7d452-4f0c-47d0-a2c9-613c6c0f2a42 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:54.932581 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:55.042323 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7d452-4f0c-47d0-a2c9-613c6c0f2a42 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:55.071942 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f9049-735b-4aa7-bce8-72852dc8eb84 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.072314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f9049-735b-4aa7-bce8-72852dc8eb84 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.072561 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f9049-735b-4aa7-bce8-72852dc8eb84 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.074099 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 216/437] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:55.080563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8abd3-bcfd-4e27-911f-a0f4bf4b5d63 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.081069 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8abd3-bcfd-4e27-911f-a0f4bf4b5d63 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.081281 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e92eb1-921d-4431-b571-0e8e54e1de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:55.081399 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8abd3-bcfd-4e27-911f-a0f4bf4b5d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.082084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e92eb1-921d-4431-b571-0e8e54e1de20 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.082363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e92eb1-921d-4431-b571-0e8e54e1de20 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.082691 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e92eb1-921d-4431-b571-0e8e54e1de20 None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.083941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e92eb1-921d-4431-b571-0e8e54e1de20 None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:55.085466 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e92eb1-921d-4431-b571-0e8e54e1de20 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:55.088318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 222/438] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:54 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:55.092734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e92eb1-921d-4431-b571-0e8e54e1de20 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:55.111061 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-3aa8abd3-bcfd-4e27-911f-a0f4bf4b5d63 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:55.112272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-3aa8abd3-bcfd-4e27-911f-a0f4bf4b5d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:55.200385 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:55.212531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 217/439] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:49:55.226096 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e92eb1-921d-4431-b571-0e8e54e1de20 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:55.229140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8abd3-bcfd-4e27-911f-a0f4bf4b5d63 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:55.257976 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:55.269339 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f5763-cf2d-4e39-8f81-cd2cb84c59f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:55.270701 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f5763-cf2d-4e39-8f81-cd2cb84c59f2 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.271331 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f5763-cf2d-4e39-8f81-cd2cb84c59f2 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.272288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f5763-cf2d-4e39-8f81-cd2cb84c59f2 None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.273757 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f5763-cf2d-4e39-8f81-cd2cb84c59f2 None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:55.275235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f5763-cf2d-4e39-8f81-cd2cb84c59f2 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:55.280514 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 223/440] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:55 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:55.282912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f5763-cf2d-4e39-8f81-cd2cb84c59f2 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:55.338289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8abd3-bcfd-4e27-911f-a0f4bf4b5d63 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:55.375789 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f27ce-bc3c-461a-ade9-c7aed7e6c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:55.377170 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f27ce-bc3c-461a-ade9-c7aed7e6c440 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.377459 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f27ce-bc3c-461a-ade9-c7aed7e6c440 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.377740 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f27ce-bc3c-461a-ade9-c7aed7e6c440 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.389506 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f27ce-bc3c-461a-ade9-c7aed7e6c440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:55.397748 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f5763-cf2d-4e39-8f81-cd2cb84c59f2 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-cc7f27ce-bc3c-461a-ade9-c7aed7e6c440 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:55.398403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:55.399409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:49:55.400430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 218/441] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:49:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:49:55.405523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f27ce-bc3c-461a-ade9-c7aed7e6c440 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:55.436120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fce92-2de6-4f8a-be48-9f487e511c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:55.436829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fce92-2de6-4f8a-be48-9f487e511c70 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.437114 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fce92-2de6-4f8a-be48-9f487e511c70 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.437394 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fce92-2de6-4f8a-be48-9f487e511c70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.438244 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fce92-2de6-4f8a-be48-9f487e511c70 None admin] RBAC: Adding query filter params (group.id=1e903affae424374a9cfedc0a6ff8a4b, role.id=2e3b001c16d24c279be5c68aa3847ca7, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:55.438567 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fce92-2de6-4f8a-be48-9f487e511c70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:55.440619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275fce92-2de6-4f8a-be48-9f487e511c70 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:55.446619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 224/442] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:49:55 2022] GET /identity/v3/role_assignments?group.id=1e903affae424374a9cfedc0a6ff8a4b&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=2e3b001c16d24c279be5c68aa3847ca7 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:55.447869 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe4404-23cd-4010-bd69-5aade8c6dfb9 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.448161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe4404-23cd-4010-bd69-5aade8c6dfb9 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.448479 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe4404-23cd-4010-bd69-5aade8c6dfb9 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.449774 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 225/443] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:55.462304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd495b7-78e9-4a19-b102-b0366cbe6225 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.462507 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd495b7-78e9-4a19-b102-b0366cbe6225 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.462774 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd495b7-78e9-4a19-b102-b0366cbe6225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.481878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-5dd495b7-78e9-4a19-b102-b0366cbe6225 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:55.483149 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-5dd495b7-78e9-4a19-b102-b0366cbe6225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:55.488750 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:55.506324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:55.524883 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f27ce-bc3c-461a-ade9-c7aed7e6c440 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:55.564944 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 226/444] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:55.574441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e01e80-a406-4dc7-b3fa-92638c5f51f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:55.575152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e01e80-a406-4dc7-b3fa-92638c5f51f6 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.575436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e01e80-a406-4dc7-b3fa-92638c5f51f6 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.575720 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e01e80-a406-4dc7-b3fa-92638c5f51f6 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.576523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e01e80-a406-4dc7-b3fa-92638c5f51f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:55.576833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e01e80-a406-4dc7-b3fa-92638c5f51f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:55.578259 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e01e80-a406-4dc7-b3fa-92638c5f51f6 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:55.588621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 219/445] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:49:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:55.591578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd495b7-78e9-4a19-b102-b0366cbe6225 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:55.594034 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e01e80-a406-4dc7-b3fa-92638c5f51f6 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:55.720150 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd495b7-78e9-4a19-b102-b0366cbe6225 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:55.732724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e01e80-a406-4dc7-b3fa-92638c5f51f6 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:55.764219 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933db63f-0496-47c5-9d58-c84df58cddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:55.765373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933db63f-0496-47c5-9d58-c84df58cddee None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.765656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933db63f-0496-47c5-9d58-c84df58cddee None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.765931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933db63f-0496-47c5-9d58-c84df58cddee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.777781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:55.780418 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f272c-d3ea-4e8b-ac5c-7b39b111dcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:55.781144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f272c-d3ea-4e8b-ac5c-7b39b111dcf4 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.781437 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f272c-d3ea-4e8b-ac5c-7b39b111dcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.781715 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f272c-d3ea-4e8b-ac5c-7b39b111dcf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.782210 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933db63f-0496-47c5-9d58-c84df58cddee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-933db63f-0496-47c5-9d58-c84df58cddee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:49:55.791138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:49:55.792271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:49:55.793887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 227/446] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:49:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:49:55.801355 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933db63f-0496-47c5-9d58-c84df58cddee None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:55.806296 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:55.871159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f272c-d3ea-4e8b-ac5c-7b39b111dcf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:55.872664 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f272c-d3ea-4e8b-ac5c-7b39b111dcf4 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:55.920026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933db63f-0496-47c5-9d58-c84df58cddee None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:55.956035 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93695a2e-328e-40e1-9de3-7553aacf44ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:55.956735 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93695a2e-328e-40e1-9de3-7553aacf44ff None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:55.957028 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93695a2e-328e-40e1-9de3-7553aacf44ff None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:55.957316 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93695a2e-328e-40e1-9de3-7553aacf44ff None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:55.958071 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93695a2e-328e-40e1-9de3-7553aacf44ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:55.958388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93695a2e-328e-40e1-9de3-7553aacf44ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:55.959791 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93695a2e-328e-40e1-9de3-7553aacf44ff None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:55.968119 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 228/447] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:49:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:55.973768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93695a2e-328e-40e1-9de3-7553aacf44ff None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:55.976863 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 220/448] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:49:55.982343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f272c-d3ea-4e8b-ac5c-7b39b111dcf4 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:56.034340 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:56.040022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:56.083403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93695a2e-328e-40e1-9de3-7553aacf44ff None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:56.091927 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f272c-d3ea-4e8b-ac5c-7b39b111dcf4 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:56.121480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18381e-710f-4fd2-85ed-92a778505559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:56.122139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18381e-710f-4fd2-85ed-92a778505559 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:56.122427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18381e-710f-4fd2-85ed-92a778505559 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:56.122717 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a18381e-710f-4fd2-85ed-92a778505559 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:56.129030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a79fc-79aa-4182-83b5-81fb2357823c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:56.129888 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a79fc-79aa-4182-83b5-81fb2357823c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:56.130172 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a79fc-79aa-4182-83b5-81fb2357823c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:56.130445 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a79fc-79aa-4182-83b5-81fb2357823c None admin] PATH_INFO: `/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:56.131651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a79fc-79aa-4182-83b5-81fb2357823c None admin] RBAC: Authorizing `identity:get_project(project_id=7e20e05ff6d243f3ab00b511081c5bf2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:56.133131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a79fc-79aa-4182-83b5-81fb2357823c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:56.134874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 221/449] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:55 2022] GET /identity/v3/projects/7e20e05ff6d243f3ab00b511081c5bf2 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:56.140314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a79fc-79aa-4182-83b5-81fb2357823c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:56.193381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18381e-710f-4fd2-85ed-92a778505559 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:56.195028 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a18381e-710f-4fd2-85ed-92a778505559 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:56.245012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a79fc-79aa-4182-83b5-81fb2357823c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:56.256794 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:56.261832 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:56.279640 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052d353-2bb8-4f68-b577-6507907989a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:56.280548 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052d353-2bb8-4f68-b577-6507907989a8 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:56.280892 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052d353-2bb8-4f68-b577-6507907989a8 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:56.281167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052d353-2bb8-4f68-b577-6507907989a8 None admin] PATH_INFO: `/v3/groups/1e903affae424374a9cfedc0a6ff8a4b` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:56.282381 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052d353-2bb8-4f68-b577-6507907989a8 None admin] RBAC: Authorizing `identity:get_group(group_id=1e903affae424374a9cfedc0a6ff8a4b)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:56.283854 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052d353-2bb8-4f68-b577-6507907989a8 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:56.285556 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 222/450] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:56 2022] GET /identity/v3/groups/1e903affae424374a9cfedc0a6ff8a4b => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:56.291883 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052d353-2bb8-4f68-b577-6507907989a8 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:56.298879 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 229/451] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:49:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 19 20:49:56.304676 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18381e-710f-4fd2-85ed-92a778505559 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:56.410236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a18381e-710f-4fd2-85ed-92a778505559 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:56.411950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052d353-2bb8-4f68-b577-6507907989a8 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:56.446590 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a748fd8-7cbe-493d-8143-b1e4c45f6ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:56.447465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a748fd8-7cbe-493d-8143-b1e4c45f6ed6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:56.447856 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a748fd8-7cbe-493d-8143-b1e4c45f6ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:56.448046 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a748fd8-7cbe-493d-8143-b1e4c45f6ed6 None admin] PATH_INFO: `/v3/projects/06a03e4df28e472baf1019b7ccaefbc6` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:56.449312 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a748fd8-7cbe-493d-8143-b1e4c45f6ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=06a03e4df28e472baf1019b7ccaefbc6)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:56.450788 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a748fd8-7cbe-493d-8143-b1e4c45f6ed6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:56.452545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 230/452] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:49:56 2022] GET /identity/v3/projects/06a03e4df28e472baf1019b7ccaefbc6 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:56.460901 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a748fd8-7cbe-493d-8143-b1e4c45f6ed6 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:56.461240 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487276-8c1d-4c71-a924-3f16496ba8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:56.462041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487276-8c1d-4c71-a924-3f16496ba8bc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:56.462382 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487276-8c1d-4c71-a924-3f16496ba8bc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:56.462874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b487276-8c1d-4c71-a924-3f16496ba8bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:56.463875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b487276-8c1d-4c71-a924-3f16496ba8bc None admin] RBAC: Adding query filter params (group.id=1e903affae424374a9cfedc0a6ff8a4b, role.id=4313b1c9df5345778a249b04355b90a0, scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:56.464247 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b487276-8c1d-4c71-a924-3f16496ba8bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:56.465922 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b487276-8c1d-4c71-a924-3f16496ba8bc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:56.473960 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 223/453] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:49:56 2022] GET /identity/v3/role_assignments?group.id=1e903affae424374a9cfedc0a6ff8a4b&scope.project.id=7e20e05ff6d243f3ab00b511081c5bf2&role.id=4313b1c9df5345778a249b04355b90a0 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:56.495242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:56.576820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a748fd8-7cbe-493d-8143-b1e4c45f6ed6 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:56.616406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9843bb-f45b-4347-aa7d-7905e6b62170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:56.617296 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9843bb-f45b-4347-aa7d-7905e6b62170 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:56.617586 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9843bb-f45b-4347-aa7d-7905e6b62170 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:56.617870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9843bb-f45b-4347-aa7d-7905e6b62170 None admin] PATH_INFO: `/v3/groups/dd885ee3d0284f3db3c719ca82039a9e` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:56.619251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9843bb-f45b-4347-aa7d-7905e6b62170 None admin] RBAC: Authorizing `identity:get_group(group_id=dd885ee3d0284f3db3c719ca82039a9e)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:56.620820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9843bb-f45b-4347-aa7d-7905e6b62170 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:56.622567 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 231/454] 10.222.0.88 () {56 vars in 1180 bytes} [Thu May 19 20:49:56 2022] GET /identity/v3/groups/dd885ee3d0284f3db3c719ca82039a9e => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:49:56.629411 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487276-8c1d-4c71-a924-3f16496ba8bc None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:56.688965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:56.746433 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b487276-8c1d-4c71-a924-3f16496ba8bc None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:56.788973 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e7079-89e1-40b8-98ac-b86830ed2579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:56.789651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7079-89e1-40b8-98ac-b86830ed2579 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:56.789939 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7079-89e1-40b8-98ac-b86830ed2579 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:56.790221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e7079-89e1-40b8-98ac-b86830ed2579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:56.791080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e7079-89e1-40b8-98ac-b86830ed2579 None admin] RBAC: Adding query filter params (group.id=dd885ee3d0284f3db3c719ca82039a9e, role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=06a03e4df28e472baf1019b7ccaefbc6) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:49:56.791391 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e7079-89e1-40b8-98ac-b86830ed2579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:56.792907 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e7079-89e1-40b8-98ac-b86830ed2579 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:49:56.800634 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 224/455] 10.222.0.88 () {56 vars in 1376 bytes} [Thu May 19 20:49:56 2022] GET /identity/v3/role_assignments?group.id=dd885ee3d0284f3db3c719ca82039a9e&scope.project.id=06a03e4df28e472baf1019b7ccaefbc6&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:49:59.542728 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43f783-7dfe-48ef-988f-e1c40103ac70 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:59.543279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43f783-7dfe-48ef-988f-e1c40103ac70 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:59.543409 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43f783-7dfe-48ef-988f-e1c40103ac70 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:59.545168 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 232/456] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:59.551785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0f434-ed77-4bd8-875b-e4580bd8ba3e None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:59.552771 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0f434-ed77-4bd8-875b-e4580bd8ba3e None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:59.553326 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0f434-ed77-4bd8-875b-e4580bd8ba3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:59.577467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-aaf0f434-ed77-4bd8-875b-e4580bd8ba3e None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:59.579141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-aaf0f434-ed77-4bd8-875b-e4580bd8ba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:59.670310 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 225/457] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:59.709201 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43f783-7dfe-48ef-988f-e1c40103ac70 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:49:59.736964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fa371-518f-41a2-b8de-25ef7b7e9cf7 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:59.737630 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fa371-518f-41a2-b8de-25ef7b7e9cf7 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:59.738143 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fa371-518f-41a2-b8de-25ef7b7e9cf7 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:59.740096 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 226/458] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:59.746762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c71fe7-fb13-4625-b76b-c5d9015402b8 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:59.747278 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c71fe7-fb13-4625-b76b-c5d9015402b8 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:59.747802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c71fe7-fb13-4625-b76b-c5d9015402b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:59.782602 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-a6c71fe7-fb13-4625-b76b-c5d9015402b8 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:59.788818 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-a6c71fe7-fb13-4625-b76b-c5d9015402b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:59.862050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:59.904611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 227/459] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:49:59.908251 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af83a37-18c8-4435-a890-02eae2974512 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:59.909022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af83a37-18c8-4435-a890-02eae2974512 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:59.909713 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af83a37-18c8-4435-a890-02eae2974512 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:59.912708 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:49:59.917952 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 228/460] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:49:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:49:59.919321 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a87f3-71ee-4487-9795-33a62c5f2132 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:59.919620 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a87f3-71ee-4487-9795-33a62c5f2132 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:59.919907 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a87f3-71ee-4487-9795-33a62c5f2132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:59.927276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43f783-7dfe-48ef-988f-e1c40103ac70 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:49:59.944789 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-816a87f3-71ee-4487-9795-33a62c5f2132 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:49:59.946416 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-816a87f3-71ee-4487-9795-33a62c5f2132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:49:59.976502 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318ae82-5d47-4120-b0ef-5a2fcc7922db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:49:59.978095 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318ae82-5d47-4120-b0ef-5a2fcc7922db None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:49:59.978571 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318ae82-5d47-4120-b0ef-5a2fcc7922db None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:49:59.979032 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318ae82-5d47-4120-b0ef-5a2fcc7922db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:49:59.991057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318ae82-5d47-4120-b0ef-5a2fcc7922db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:49:59.993063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318ae82-5d47-4120-b0ef-5a2fcc7922db None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-f318ae82-5d47-4120-b0ef-5a2fcc7922db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:00.016667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:00.017796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:00.019308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:00.019308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:00.019308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:00.019308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:00.022022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 233/461] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:49:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:00.023154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318ae82-5d47-4120-b0ef-5a2fcc7922db None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:00.030575 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 229/462] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:00.031685 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a87f3-71ee-4487-9795-33a62c5f2132 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:00.140543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318ae82-5d47-4120-b0ef-5a2fcc7922db None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:00.170779 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a87f3-71ee-4487-9795-33a62c5f2132 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:00.172738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:00.178086 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223cacf-429c-4c7d-a5fd-1d62de5320ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:00.179302 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223cacf-429c-4c7d-a5fd-1d62de5320ee None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:00.179781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223cacf-429c-4c7d-a5fd-1d62de5320ee None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:00.180276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223cacf-429c-4c7d-a5fd-1d62de5320ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:00.192530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223cacf-429c-4c7d-a5fd-1d62de5320ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:00.194401 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223cacf-429c-4c7d-a5fd-1d62de5320ee None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:00.211320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4b093-e213-4809-8836-aff6f03e678e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:00.212018 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4b093-e213-4809-8836-aff6f03e678e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:00.212306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4b093-e213-4809-8836-aff6f03e678e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:00.212624 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4b093-e213-4809-8836-aff6f03e678e None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:00.213371 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4b093-e213-4809-8836-aff6f03e678e None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:00.213686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4b093-e213-4809-8836-aff6f03e678e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:00.215328 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4b093-e213-4809-8836-aff6f03e678e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-5223cacf-429c-4c7d-a5fd-1d62de5320ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:00.216521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:00.218345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:00.220384 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:00.220384 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:00.220384 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:00.220384 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:00.229403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 234/463] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:00.230778 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223cacf-429c-4c7d-a5fd-1d62de5320ee None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:00.234011 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 230/464] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:00.234658 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:00.236271 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4b093-e213-4809-8836-aff6f03e678e None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:00.344235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223cacf-429c-4c7d-a5fd-1d62de5320ee None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:00.355566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4b093-e213-4809-8836-aff6f03e678e None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:00.383429 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e6c8c-df8e-42cc-b50d-eee012fc7625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:00.384584 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e6c8c-df8e-42cc-b50d-eee012fc7625 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:00.385087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e6c8c-df8e-42cc-b50d-eee012fc7625 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:00.385550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e6c8c-df8e-42cc-b50d-eee012fc7625 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:00.396967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e6c8c-df8e-42cc-b50d-eee012fc7625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:00.398826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e6c8c-df8e-42cc-b50d-eee012fc7625 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:00.404780 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f16e43-ec11-4fe9-a8ff-940e13662c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:00.405749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f16e43-ec11-4fe9-a8ff-940e13662c92 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:00.406229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f16e43-ec11-4fe9-a8ff-940e13662c92 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:00.406700 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f16e43-ec11-4fe9-a8ff-940e13662c92 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:00.408773 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f16e43-ec11-4fe9-a8ff-940e13662c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:00.409292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f16e43-ec11-4fe9-a8ff-940e13662c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:00.411730 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f16e43-ec11-4fe9-a8ff-940e13662c92 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-ef7e6c8c-df8e-42cc-b50d-eee012fc7625 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:00.420178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:00.421345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:00.422461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:00.422461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:00.422461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:00.422461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:00.422461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 235/465] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:00.422461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e6c8c-df8e-42cc-b50d-eee012fc7625 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:00.434503 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 231/466] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:00.435899 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f16e43-ec11-4fe9-a8ff-940e13662c92 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:00.481617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:00.499633 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:00.550175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e6c8c-df8e-42cc-b50d-eee012fc7625 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:00.574170 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f16e43-ec11-4fe9-a8ff-940e13662c92 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:00.587039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fd77a-b4df-4f1c-a5eb-9cfdf3d1953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:00.587762 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fd77a-b4df-4f1c-a5eb-9cfdf3d1953f None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:00.588103 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fd77a-b4df-4f1c-a5eb-9cfdf3d1953f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:00.588299 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fd77a-b4df-4f1c-a5eb-9cfdf3d1953f None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:00.589062 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fd77a-b4df-4f1c-a5eb-9cfdf3d1953f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:00.590811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fd77a-b4df-4f1c-a5eb-9cfdf3d1953f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:00.615398 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252d788-510d-496b-8031-51fa8710b25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:00.616074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252d788-510d-496b-8031-51fa8710b25b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:00.616390 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252d788-510d-496b-8031-51fa8710b25b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:00.616679 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252d788-510d-496b-8031-51fa8710b25b None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:00.617537 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252d788-510d-496b-8031-51fa8710b25b None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:00.617795 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252d788-510d-496b-8031-51fa8710b25b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:00.619195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252d788-510d-496b-8031-51fa8710b25b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:00.631079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 232/467] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:00.631895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252d788-510d-496b-8031-51fa8710b25b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:00.718854 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:00.738028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252d788-510d-496b-8031-51fa8710b25b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:00.741594 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 236/468] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:50:00 2022] POST /identity/v3/users => generated 246 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:00.741594 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b3265-5076-4c37-b3b0-10669ed984b9 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:00.741594 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b3265-5076-4c37-b3b0-10669ed984b9 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:00.742256 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b3265-5076-4c37-b3b0-10669ed984b9 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:00.749578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 237/469] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:00.750932 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b3265-5076-4c37-b3b0-10669ed984b9 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:00.784509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b28210-4972-4be7-a687-87055d51f73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:00.785202 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b28210-4972-4be7-a687-87055d51f73f None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:00.785486 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b28210-4972-4be7-a687-87055d51f73f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:00.785837 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b28210-4972-4be7-a687-87055d51f73f None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:00.786580 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b28210-4972-4be7-a687-87055d51f73f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:00.788206 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b28210-4972-4be7-a687-87055d51f73f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:00.884145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b3265-5076-4c37-b3b0-10669ed984b9 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:00.899148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:00.912369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 233/470] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:50:00 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:00.912369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed55bc-e61a-43bc-9c04-5181798daf98 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:00.912369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed55bc-e61a-43bc-9c04-5181798daf98 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:00.912369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed55bc-e61a-43bc-9c04-5181798daf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:00.949187 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-26ed55bc-e61a-43bc-9c04-5181798daf98 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:00.952660 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-26ed55bc-e61a-43bc-9c04-5181798daf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:00.952789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6d0d0-278b-4790-95dc-fe81bf6eb189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:00.952789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6d0d0-278b-4790-95dc-fe81bf6eb189 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:00.952789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6d0d0-278b-4790-95dc-fe81bf6eb189 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:00.952789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6d0d0-278b-4790-95dc-fe81bf6eb189 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:00.953323 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6d0d0-278b-4790-95dc-fe81bf6eb189 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:00.955437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb6d0d0-278b-4790-95dc-fe81bf6eb189 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:01.062903 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 234/471] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:01.089333 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed55bc-e61a-43bc-9c04-5181798daf98 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:01.098117 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 238/472] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:50:00 2022] POST /identity/v3/users => generated 248 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:01.179528 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:01.181599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1e885-04b2-47c0-8863-49bd9fe10c36 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:01.181914 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1e885-04b2-47c0-8863-49bd9fe10c36 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:01.182201 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1e885-04b2-47c0-8863-49bd9fe10c36 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:01.184356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 239/473] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:50:01.192063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77742479-7749-4de2-bff5-f151f569ba6d None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:01.194769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77742479-7749-4de2-bff5-f151f569ba6d None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:01.194769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77742479-7749-4de2-bff5-f151f569ba6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:01.217971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-77742479-7749-4de2-bff5-f151f569ba6d None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:01.219244 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-77742479-7749-4de2-bff5-f151f569ba6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:01.224579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed55bc-e61a-43bc-9c04-5181798daf98 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:01.271541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96a977-dc77-490b-9d70-0c65e9fcb66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:01.272444 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96a977-dc77-490b-9d70-0c65e9fcb66f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:01.272727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96a977-dc77-490b-9d70-0c65e9fcb66f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:01.273005 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96a977-dc77-490b-9d70-0c65e9fcb66f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:01.297930 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96a977-dc77-490b-9d70-0c65e9fcb66f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:01.299645 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96a977-dc77-490b-9d70-0c65e9fcb66f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:01.321067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 240/474] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-0c96a977-dc77-490b-9d70-0c65e9fcb66f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:01.322999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:01.324158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:01.325350 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:01.325350 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:01.325350 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:01.325350 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:01.325350 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 235/475] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:01.330985 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77742479-7749-4de2-bff5-f151f569ba6d None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:01.353410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96a977-dc77-490b-9d70-0c65e9fcb66f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:01.482044 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:01.498981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77742479-7749-4de2-bff5-f151f569ba6d None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:01.549427 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96a977-dc77-490b-9d70-0c65e9fcb66f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:01.552463 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282437c8-219d-4bc3-b120-5545e826292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:01.555335 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282437c8-219d-4bc3-b120-5545e826292b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:01.555335 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282437c8-219d-4bc3-b120-5545e826292b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:01.555335 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282437c8-219d-4bc3-b120-5545e826292b None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:01.555740 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282437c8-219d-4bc3-b120-5545e826292b None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:01.555996 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282437c8-219d-4bc3-b120-5545e826292b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:01.557537 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282437c8-219d-4bc3-b120-5545e826292b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:01.580581 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:01.581542 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 241/476] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:01.589869 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282437c8-219d-4bc3-b120-5545e826292b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:01.609549 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b29dd-6bbe-45f7-8662-36a5c07de0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:01.610249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b29dd-6bbe-45f7-8662-36a5c07de0f0 None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:01.611047 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b29dd-6bbe-45f7-8662-36a5c07de0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:01.611345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b29dd-6bbe-45f7-8662-36a5c07de0f0 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:01.612112 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b29dd-6bbe-45f7-8662-36a5c07de0f0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:01.613586 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b29dd-6bbe-45f7-8662-36a5c07de0f0 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:01.649980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.sql.core [None req-902b29dd-6bbe-45f7-8662-36a5c07de0f0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:50:01.649980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:50:01.649980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [parameters: {'id': '954e51f685ac49bd8583e83245615395', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:50:01.649980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=278914) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-902b29dd-6bbe-45f7-8662-36a5c07de0f0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application conn.query(q) May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application result.read() May 19 20:50:01.688572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return ref.to_dict() May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:50:01.690250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application next(self.gen) May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application yield resource May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application next(self.gen) May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise self.value May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application session.commit() May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:50:01.691414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self.session.flush() May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application compat.raise_( May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:50:01.692608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:50:01.693693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application conn.query(q) May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application result.read() May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:50:01.694778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [parameters: {'id': '954e51f685ac49bd8583e83245615395', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:01.697485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 19 20:50:01.699826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:01.704925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 236/477] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:50:01 2022] POST /identity/v3/roles => generated 144 bytes in 352 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:01.712663 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b29dd-6bbe-45f7-8662-36a5c07de0f0 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:01.761417 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282437c8-219d-4bc3-b120-5545e826292b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:01.808363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db693ea-8632-4b8a-a4e2-6bb6a5d5da11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:01.808363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db693ea-8632-4b8a-a4e2-6bb6a5d5da11 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:01.808363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db693ea-8632-4b8a-a4e2-6bb6a5d5da11 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:01.808363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db693ea-8632-4b8a-a4e2-6bb6a5d5da11 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:01.808862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db693ea-8632-4b8a-a4e2-6bb6a5d5da11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:01.808862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db693ea-8632-4b8a-a4e2-6bb6a5d5da11 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:01.864597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b29dd-6bbe-45f7-8662-36a5c07de0f0 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:01.890389 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 242/478] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:50:01 2022] POST /identity/v3/users => generated 249 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:50:01.924374 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee63ed-dfeb-4f27-9d61-3141618c8192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:01.925328 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee63ed-dfeb-4f27-9d61-3141618c8192 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:01.925619 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee63ed-dfeb-4f27-9d61-3141618c8192 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:01.925903 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee63ed-dfeb-4f27-9d61-3141618c8192 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:01.946359 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee63ed-dfeb-4f27-9d61-3141618c8192 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-a9ee63ed-dfeb-4f27-9d61-3141618c8192 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:01.955472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:01.956762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 19 20:50:01.957805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:01.961369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 237/479] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:50:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:01.967353 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db693ea-8632-4b8a-a4e2-6bb6a5d5da11 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:01.992361 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:02.147647 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db693ea-8632-4b8a-a4e2-6bb6a5d5da11 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:02.191445 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb9e80-037c-4a5d-9b1d-190c22fd89ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:02.193229 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb9e80-037c-4a5d-9b1d-190c22fd89ab None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:02.193583 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb9e80-037c-4a5d-9b1d-190c22fd89ab None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:02.194057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb9e80-037c-4a5d-9b1d-190c22fd89ab None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:02.194900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb9e80-037c-4a5d-9b1d-190c22fd89ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:02.195219 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb9e80-037c-4a5d-9b1d-190c22fd89ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:02.196872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb9e80-037c-4a5d-9b1d-190c22fd89ab None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:02.207758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 243/480] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:03.656291 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de00f94-282e-42de-82c8-37c180595bb1 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:03.656914 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de00f94-282e-42de-82c8-37c180595bb1 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:03.656914 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de00f94-282e-42de-82c8-37c180595bb1 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:03.659719 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 238/481] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:03.665466 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74cb59-865c-45e0-aa74-4ef9e011300a None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:03.666106 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74cb59-865c-45e0-aa74-4ef9e011300a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:03.666106 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74cb59-865c-45e0-aa74-4ef9e011300a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:03.693130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-ce74cb59-865c-45e0-aa74-4ef9e011300a None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:03.694420 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-ce74cb59-865c-45e0-aa74-4ef9e011300a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:03.826988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 244/482] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:03.905695 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de00f94-282e-42de-82c8-37c180595bb1 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:03.972857 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:03.989833 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29ced-89cd-476e-9165-1c7af243cde9 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:03.989833 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29ced-89cd-476e-9165-1c7af243cde9 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:03.989833 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29ced-89cd-476e-9165-1c7af243cde9 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:03.991679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 245/483] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:03.996839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb917d-52c8-4c0f-a8c2-c5f3d45f2b76 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:03.997428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb917d-52c8-4c0f-a8c2-c5f3d45f2b76 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:03.997951 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb917d-52c8-4c0f-a8c2-c5f3d45f2b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:04.016446 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:04.039031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-03bb917d-52c8-4c0f-a8c2-c5f3d45f2b76 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:04.040388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-03bb917d-52c8-4c0f-a8c2-c5f3d45f2b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:04.049216 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de00f94-282e-42de-82c8-37c180595bb1 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:04.098212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c745360-67b0-4420-8821-e3e707869cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:04.098975 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c745360-67b0-4420-8821-e3e707869cd7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:04.099260 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c745360-67b0-4420-8821-e3e707869cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:04.099539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c745360-67b0-4420-8821-e3e707869cd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:04.125563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c745360-67b0-4420-8821-e3e707869cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-6c745360-67b0-4420-8821-e3e707869cd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:04.144421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:04.145584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:04.146585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 239/484] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:04.174724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c745360-67b0-4420-8821-e3e707869cd7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:04.176512 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 246/485] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:50:04.218057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb917d-52c8-4c0f-a8c2-c5f3d45f2b76 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:04.400751 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb917d-52c8-4c0f-a8c2-c5f3d45f2b76 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:04.443417 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ba3a8-b172-423e-b827-7a2ad0fd6120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:04.444721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba3a8-b172-423e-b827-7a2ad0fd6120 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:04.445262 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba3a8-b172-423e-b827-7a2ad0fd6120 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:04.445792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ba3a8-b172-423e-b827-7a2ad0fd6120 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:04.448159 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:04.456253 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c745360-67b0-4420-8821-e3e707869cd7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:04.459345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ba3a8-b172-423e-b827-7a2ad0fd6120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:04.461142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-bd9ba3a8-b172-423e-b827-7a2ad0fd6120 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:04.471983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:04.473136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 247/486] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:04.474288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b185241-6530-47c0-9028-7b224a4f71f0 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:04.475836 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b185241-6530-47c0-9028-7b224a4f71f0 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:04.476396 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b185241-6530-47c0-9028-7b224a4f71f0 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:04.482406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 248/487] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:04.484026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbf0c6-63bd-40f0-8484-54900d1dd8f7 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:04.484592 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbf0c6-63bd-40f0-8484-54900d1dd8f7 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:04.485116 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbf0c6-63bd-40f0-8484-54900d1dd8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:04.506147 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-56fbf0c6-63bd-40f0-8484-54900d1dd8f7 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:04.507919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-56fbf0c6-63bd-40f0-8484-54900d1dd8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:04.530993 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88fdd7-3b80-4a2c-9288-d88416275ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:04.531849 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fdd7-3b80-4a2c-9288-d88416275ea9 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:04.532138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fdd7-3b80-4a2c-9288-d88416275ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:04.532968 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88fdd7-3b80-4a2c-9288-d88416275ea9 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:04.533749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88fdd7-3b80-4a2c-9288-d88416275ea9 None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:04.534347 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88fdd7-3b80-4a2c-9288-d88416275ea9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:04.535742 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88fdd7-3b80-4a2c-9288-d88416275ea9 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:04.552104 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 240/488] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:04.553209 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88fdd7-3b80-4a2c-9288-d88416275ea9 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:04.600390 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 249/489] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:04.600390 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbf0c6-63bd-40f0-8484-54900d1dd8f7 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:04.704085 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88fdd7-3b80-4a2c-9288-d88416275ea9 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:04.803376 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbf0c6-63bd-40f0-8484-54900d1dd8f7 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:04.814625 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd98e5-f401-41e5-a67e-b932c9d1c2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:04.816065 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd98e5-f401-41e5-a67e-b932c9d1c2ae None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:04.816989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd98e5-f401-41e5-a67e-b932c9d1c2ae None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:04.817484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd98e5-f401-41e5-a67e-b932c9d1c2ae None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:04.818699 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd98e5-f401-41e5-a67e-b932c9d1c2ae None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:04.819196 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd98e5-f401-41e5-a67e-b932c9d1c2ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:04.821827 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd98e5-f401-41e5-a67e-b932c9d1c2ae None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:04.831517 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:04.836314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 241/490] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:04.837570 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd98e5-f401-41e5-a67e-b932c9d1c2ae None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:04.848799 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f51ea-6c93-482d-87ad-5ba8f3e14028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:04.849700 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f51ea-6c93-482d-87ad-5ba8f3e14028 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:04.850070 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f51ea-6c93-482d-87ad-5ba8f3e14028 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:04.850421 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f51ea-6c93-482d-87ad-5ba8f3e14028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:04.862541 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f51ea-6c93-482d-87ad-5ba8f3e14028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:04.864258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f51ea-6c93-482d-87ad-5ba8f3e14028 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:04.881586 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-bd9f51ea-6c93-482d-87ad-5ba8f3e14028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:04.894012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:04.898263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:04.899294 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:04.899294 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:04.899294 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:04.899294 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:04.900748 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 250/491] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:04.901792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f51ea-6c93-482d-87ad-5ba8f3e14028 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:04.987851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd98e5-f401-41e5-a67e-b932c9d1c2ae None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:05.028637 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cff8f-9527-401e-8805-737677b77a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:05.029679 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cff8f-9527-401e-8805-737677b77a21 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.029938 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cff8f-9527-401e-8805-737677b77a21 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.030300 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cff8f-9527-401e-8805-737677b77a21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.032283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f51ea-6c93-482d-87ad-5ba8f3e14028 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:05.043768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cff8f-9527-401e-8805-737677b77a21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-348cff8f-9527-401e-8805-737677b77a21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:05.052035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:05.053185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:05.054168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:05.058694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 242/492] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:05.059741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cff8f-9527-401e-8805-737677b77a21 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:05.078365 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5d465-3331-40ba-89fa-c407659f67ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:05.079196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d465-3331-40ba-89fa-c407659f67ec None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.080058 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d465-3331-40ba-89fa-c407659f67ec None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.080371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5d465-3331-40ba-89fa-c407659f67ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.091370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5d465-3331-40ba-89fa-c407659f67ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:05.092889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5d465-3331-40ba-89fa-c407659f67ec None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-c1f5d465-3331-40ba-89fa-c407659f67ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:05.111083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:05.112254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:05.113346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:05.113346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:05.113346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:05.113346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:05.113346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 251/493] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:05.113346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5d465-3331-40ba-89fa-c407659f67ec None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:05.149052 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:05.161228 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:05.173218 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cff8f-9527-401e-8805-737677b77a21 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:05.210155 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a4420-e38b-4c13-b468-a386f44f7caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:05.210840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a4420-e38b-4c13-b468-a386f44f7caf None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.211133 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a4420-e38b-4c13-b468-a386f44f7caf None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.211415 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a4420-e38b-4c13-b468-a386f44f7caf None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.212153 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a4420-e38b-4c13-b468-a386f44f7caf None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:05.212502 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a4420-e38b-4c13-b468-a386f44f7caf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:05.213892 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a4420-e38b-4c13-b468-a386f44f7caf None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:05.223315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5d465-3331-40ba-89fa-c407659f67ec None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:05.225659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 243/494] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:05.227116 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199738c8-e18a-4111-a2f4-d3656e8d28bb None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.227408 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199738c8-e18a-4111-a2f4-d3656e8d28bb None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.227689 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199738c8-e18a-4111-a2f4-d3656e8d28bb None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.229117 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 244/495] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:05.230514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128b2b6-f430-4896-8e91-892ace70c2d6 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.230806 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128b2b6-f430-4896-8e91-892ace70c2d6 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.231087 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128b2b6-f430-4896-8e91-892ace70c2d6 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.232352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 245/496] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:05.233126 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128b2b6-f430-4896-8e91-892ace70c2d6 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:05.264530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99b50e-d8fd-4268-b523-196ec5e82565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:05.265203 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99b50e-d8fd-4268-b523-196ec5e82565 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.265491 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99b50e-d8fd-4268-b523-196ec5e82565 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.265780 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99b50e-d8fd-4268-b523-196ec5e82565 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.266537 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99b50e-d8fd-4268-b523-196ec5e82565 None admin] RBAC: Adding query filter params (name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:05.267406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99b50e-d8fd-4268-b523-196ec5e82565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:05.268862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99b50e-d8fd-4268-b523-196ec5e82565 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:05.277266 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 252/497] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:05.278093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99b50e-d8fd-4268-b523-196ec5e82565 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:05.339464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a128b2b6-f430-4896-8e91-892ace70c2d6 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:05.374303 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2715e72a-816d-4fd9-834c-3fc2e5bbb889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:05.374974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715e72a-816d-4fd9-834c-3fc2e5bbb889 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.375129 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:05.375283 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715e72a-816d-4fd9-834c-3fc2e5bbb889 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.375543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2715e72a-816d-4fd9-834c-3fc2e5bbb889 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.376861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715e72a-816d-4fd9-834c-3fc2e5bbb889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:05.377178 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715e72a-816d-4fd9-834c-3fc2e5bbb889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:05.378565 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2715e72a-816d-4fd9-834c-3fc2e5bbb889 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:05.385697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99b50e-d8fd-4268-b523-196ec5e82565 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:05.390150 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 246/498] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:05.397296 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff939d1-4ede-4492-9e20-cb01061c1601 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.397296 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff939d1-4ede-4492-9e20-cb01061c1601 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.397296 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff939d1-4ede-4492-9e20-cb01061c1601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.418728 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-fff939d1-4ede-4492-9e20-cb01061c1601 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:05.420038 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-fff939d1-4ede-4492-9e20-cb01061c1601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:05.422277 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:05.423784 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245da0c-e834-4a8f-bcdf-026e46bfa335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:05.424635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245da0c-e834-4a8f-bcdf-026e46bfa335 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.424929 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245da0c-e834-4a8f-bcdf-026e46bfa335 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.425212 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245da0c-e834-4a8f-bcdf-026e46bfa335 None admin] PATH_INFO: `/v3/users/nova` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.434190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245da0c-e834-4a8f-bcdf-026e46bfa335 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:05.435720 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245da0c-e834-4a8f-bcdf-026e46bfa335 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-e245da0c-e834-4a8f-bcdf-026e46bfa335 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:05.452057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:05.453329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 253/499] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:50:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d5ad7-b6e7-4777-a172-13ea0bac85cf None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.455354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d5ad7-b6e7-4777-a172-13ea0bac85cf None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.456411 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d5ad7-b6e7-4777-a172-13ea0bac85cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.473207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-198d5ad7-b6e7-4777-a172-13ea0bac85cf None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:05.474422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-198d5ad7-b6e7-4777-a172-13ea0bac85cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:05.506335 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 247/500] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:05.507194 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff939d1-4ede-4492-9e20-cb01061c1601 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:05.544833 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 254/501] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:05.545704 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d5ad7-b6e7-4777-a172-13ea0bac85cf None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:05.621547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff939d1-4ede-4492-9e20-cb01061c1601 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:05.629854 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:05.650369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d5ad7-b6e7-4777-a172-13ea0bac85cf None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:05.656670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea819f1-5a33-4997-98b1-c21a81e4b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:05.658033 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea819f1-5a33-4997-98b1-c21a81e4b1eb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.658341 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea819f1-5a33-4997-98b1-c21a81e4b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.658624 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea819f1-5a33-4997-98b1-c21a81e4b1eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.665504 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:05.669490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea819f1-5a33-4997-98b1-c21a81e4b1eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:05.670981 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea819f1-5a33-4997-98b1-c21a81e4b1eb None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:05.686231 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c3d66-0bd0-4cc2-9c48-0f80ec176971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:05.687025 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c3d66-0bd0-4cc2-9c48-0f80ec176971 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.687307 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c3d66-0bd0-4cc2-9c48-0f80ec176971 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.687581 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c3d66-0bd0-4cc2-9c48-0f80ec176971 None admin] PATH_INFO: `/v3/users/glance` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-bea819f1-5a33-4997-98b1-c21a81e4b1eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:05.689783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:05.690921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:05.692140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:05.692140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:05.692140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:05.692140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:05.692140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 248/502] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:05.692140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea819f1-5a33-4997-98b1-c21a81e4b1eb None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:05.698981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c3d66-0bd0-4cc2-9c48-0f80ec176971 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:05.700688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c3d66-0bd0-4cc2-9c48-0f80ec176971 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-d82c3d66-0bd0-4cc2-9c48-0f80ec176971 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:05.716545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:05.717712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 255/503] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:05.718822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c3d66-0bd0-4cc2-9c48-0f80ec176971 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:05.803401 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea819f1-5a33-4997-98b1-c21a81e4b1eb None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:05.826181 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c3d66-0bd0-4cc2-9c48-0f80ec176971 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:05.848946 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6dd2b-8565-4f9e-aeeb-9cc268ea6060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:05.849618 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6dd2b-8565-4f9e-aeeb-9cc268ea6060 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.849906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6dd2b-8565-4f9e-aeeb-9cc268ea6060 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.850177 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e6dd2b-8565-4f9e-aeeb-9cc268ea6060 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.851015 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6dd2b-8565-4f9e-aeeb-9cc268ea6060 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:05.851320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6dd2b-8565-4f9e-aeeb-9cc268ea6060 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:05.852775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e6dd2b-8565-4f9e-aeeb-9cc268ea6060 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:05.860840 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d60f40-3bb0-4990-859d-44595b5e725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:05.861632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d60f40-3bb0-4990-859d-44595b5e725f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:05.861915 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d60f40-3bb0-4990-859d-44595b5e725f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:05.862190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d60f40-3bb0-4990-859d-44595b5e725f None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:05.870096 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:05.875231 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d60f40-3bb0-4990-859d-44595b5e725f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-a7d60f40-3bb0-4990-859d-44595b5e725f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:05.883674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:05.889990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 256/504] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:05.891026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d60f40-3bb0-4990-859d-44595b5e725f None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:05.897921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 249/505] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:50:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:05.897921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6dd2b-8565-4f9e-aeeb-9cc268ea6060 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:05.945630 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:05.995927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d60f40-3bb0-4990-859d-44595b5e725f None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:05.999586 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e6dd2b-8565-4f9e-aeeb-9cc268ea6060 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:06.037132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852652c-7dfb-463f-9be6-0bfb3bf3cd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:06.037425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc00211-4f70-4468-8eae-ed18913fd5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:06.037940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852652c-7dfb-463f-9be6-0bfb3bf3cd2f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:06.038064 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc00211-4f70-4468-8eae-ed18913fd5be None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:06.038227 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852652c-7dfb-463f-9be6-0bfb3bf3cd2f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:06.038392 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc00211-4f70-4468-8eae-ed18913fd5be None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:06.038505 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852652c-7dfb-463f-9be6-0bfb3bf3cd2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:06.038615 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc00211-4f70-4468-8eae-ed18913fd5be None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:06.039272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc00211-4f70-4468-8eae-ed18913fd5be None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:06.039589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc00211-4f70-4468-8eae-ed18913fd5be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:06.041047 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc00211-4f70-4468-8eae-ed18913fd5be None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:06.050312 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852652c-7dfb-463f-9be6-0bfb3bf3cd2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:06.051824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852652c-7dfb-463f-9be6-0bfb3bf3cd2f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:06.054626 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 250/506] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:06.055309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc00211-4f70-4468-8eae-ed18913fd5be None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-5852652c-7dfb-463f-9be6-0bfb3bf3cd2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:06.071419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:06.072653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:06.078779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:06.078779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:06.078779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:06.078779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:06.078779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 257/507] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:06.078779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852652c-7dfb-463f-9be6-0bfb3bf3cd2f None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:06.129585 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:06.157924 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc00211-4f70-4468-8eae-ed18913fd5be None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:06.168239 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:06.178452 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852652c-7dfb-463f-9be6-0bfb3bf3cd2f None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:06.198407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaac91f-7522-4575-acf6-53887b83cc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:06.199223 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaac91f-7522-4575-acf6-53887b83cc13 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:06.199620 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaac91f-7522-4575-acf6-53887b83cc13 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:06.200013 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaac91f-7522-4575-acf6-53887b83cc13 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:06.201122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaac91f-7522-4575-acf6-53887b83cc13 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:06.201655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaac91f-7522-4575-acf6-53887b83cc13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:06.203476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaac91f-7522-4575-acf6-53887b83cc13 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:06.217498 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ec503-a35d-477e-84a9-7a73ae46cfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:06.218214 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ec503-a35d-477e-84a9-7a73ae46cfb9 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:06.218459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ec503-a35d-477e-84a9-7a73ae46cfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:06.218736 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ec503-a35d-477e-84a9-7a73ae46cfb9 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:06.219579 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ec503-a35d-477e-84a9-7a73ae46cfb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:06.220583 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ec503-a35d-477e-84a9-7a73ae46cfb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:06.222075 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ec503-a35d-477e-84a9-7a73ae46cfb9 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:06.226263 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 251/508] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:50:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:06.227131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaac91f-7522-4575-acf6-53887b83cc13 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:06.231260 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 258/509] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:06.231729 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ec503-a35d-477e-84a9-7a73ae46cfb9 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:06.336569 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaac91f-7522-4575-acf6-53887b83cc13 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:06.337037 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ec503-a35d-477e-84a9-7a73ae46cfb9 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:06.352655 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:06.373852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225e7a4-1560-429e-8615-3e2bdf35f9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:06.374663 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225e7a4-1560-429e-8615-3e2bdf35f9a9 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:06.374946 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225e7a4-1560-429e-8615-3e2bdf35f9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:06.375156 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a68f9-40c6-46f7-9132-480748dabe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:06.375284 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2225e7a4-1560-429e-8615-3e2bdf35f9a9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:06.375816 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a68f9-40c6-46f7-9132-480748dabe4c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:06.376105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a68f9-40c6-46f7-9132-480748dabe4c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:06.376429 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a68f9-40c6-46f7-9132-480748dabe4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:06.384093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225e7a4-1560-429e-8615-3e2bdf35f9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:06.385550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2225e7a4-1560-429e-8615-3e2bdf35f9a9 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-2225e7a4-1560-429e-8615-3e2bdf35f9a9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:06.401465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:06.402829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 259/510] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:06.404026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225e7a4-1560-429e-8615-3e2bdf35f9a9 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:06.415224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:06.452405 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a68f9-40c6-46f7-9132-480748dabe4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:06.453892 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a68f9-40c6-46f7-9132-480748dabe4c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:06.518392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2225e7a4-1560-429e-8615-3e2bdf35f9a9 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:06.566552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079a490-4ed8-4059-8b30-3d9b54614131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:06.567224 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079a490-4ed8-4059-8b30-3d9b54614131 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:06.567506 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079a490-4ed8-4059-8b30-3d9b54614131 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:06.567787 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079a490-4ed8-4059-8b30-3d9b54614131 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:06.568545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079a490-4ed8-4059-8b30-3d9b54614131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:06.568858 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079a490-4ed8-4059-8b30-3d9b54614131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:06.570445 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079a490-4ed8-4059-8b30-3d9b54614131 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:06.572002 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 252/511] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:06.572898 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a68f9-40c6-46f7-9132-480748dabe4c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:06.583494 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 260/512] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:06.584356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079a490-4ed8-4059-8b30-3d9b54614131 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:06.619089 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:06.644492 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:06.679771 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a68f9-40c6-46f7-9132-480748dabe4c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:06.692416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079a490-4ed8-4059-8b30-3d9b54614131 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:06.719705 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03139a34-3d1e-4f03-be58-290d3f889922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:06.720384 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03139a34-3d1e-4f03-be58-290d3f889922 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:06.720674 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03139a34-3d1e-4f03-be58-290d3f889922 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:06.720951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03139a34-3d1e-4f03-be58-290d3f889922 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:06.733812 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afcc670-6b92-43e7-8938-d63ee2f8178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:06.734750 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcc670-6b92-43e7-8938-d63ee2f8178e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:06.735213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcc670-6b92-43e7-8938-d63ee2f8178e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:06.735535 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcc670-6b92-43e7-8938-d63ee2f8178e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:06.746942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afcc670-6b92-43e7-8938-d63ee2f8178e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:06.748508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afcc670-6b92-43e7-8938-d63ee2f8178e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-5afcc670-6b92-43e7-8938-d63ee2f8178e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:06.768849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:06.769974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:06.770990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:06.770990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:06.770990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:06.770990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:06.770990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 261/513] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:06.770990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afcc670-6b92-43e7-8938-d63ee2f8178e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:06.798454 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03139a34-3d1e-4f03-be58-290d3f889922 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:06.799937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03139a34-3d1e-4f03-be58-290d3f889922 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:06.873655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:06.882513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:06.899580 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afcc670-6b92-43e7-8938-d63ee2f8178e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:06.912388 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 253/514] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:06.913264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03139a34-3d1e-4f03-be58-290d3f889922 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:06.941313 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56451e54-30a5-4717-9c44-186e70609666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:06.945430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56451e54-30a5-4717-9c44-186e70609666 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:06.945430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56451e54-30a5-4717-9c44-186e70609666 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:06.945430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56451e54-30a5-4717-9c44-186e70609666 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:06.945430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56451e54-30a5-4717-9c44-186e70609666 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:06.945430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56451e54-30a5-4717-9c44-186e70609666 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:06.945430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56451e54-30a5-4717-9c44-186e70609666 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:06.983721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 262/515] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:50:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:06.984597 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56451e54-30a5-4717-9c44-186e70609666 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:07.020991 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03139a34-3d1e-4f03-be58-290d3f889922 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:07.057963 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640d30b-ef8e-4adf-b161-bd5ff608cfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:07.059010 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640d30b-ef8e-4adf-b161-bd5ff608cfe3 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:07.059285 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640d30b-ef8e-4adf-b161-bd5ff608cfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:07.059557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640d30b-ef8e-4adf-b161-bd5ff608cfe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:07.071496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640d30b-ef8e-4adf-b161-bd5ff608cfe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:07.073004 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640d30b-ef8e-4adf-b161-bd5ff608cfe3 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-e640d30b-ef8e-4adf-b161-bd5ff608cfe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:07.091293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:07.092476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:07.093752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:07.093752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:07.093752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:07.093752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:07.096497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 254/516] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:07.097904 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640d30b-ef8e-4adf-b161-bd5ff608cfe3 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:07.101972 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56451e54-30a5-4717-9c44-186e70609666 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:07.119761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:07.141569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db3b89-dac1-434a-823a-09780dfd115c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:07.143455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db3b89-dac1-434a-823a-09780dfd115c None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:07.143455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db3b89-dac1-434a-823a-09780dfd115c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:07.152839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db3b89-dac1-434a-823a-09780dfd115c None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:07.153581 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db3b89-dac1-434a-823a-09780dfd115c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:07.155077 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db3b89-dac1-434a-823a-09780dfd115c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:07.208563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640d30b-ef8e-4adf-b161-bd5ff608cfe3 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:07.239874 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:07.244503 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 263/517] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:50:06 2022] POST /identity/v3/users => generated 247 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:07.253521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454230b-daca-44e5-9e02-4ab59a6aebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:07.253521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454230b-daca-44e5-9e02-4ab59a6aebf9 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:07.253822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db3b89-dac1-434a-823a-09780dfd115c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:07.253937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454230b-daca-44e5-9e02-4ab59a6aebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:07.253937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454230b-daca-44e5-9e02-4ab59a6aebf9 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:07.253937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454230b-daca-44e5-9e02-4ab59a6aebf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:07.253937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454230b-daca-44e5-9e02-4ab59a6aebf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:07.253937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454230b-daca-44e5-9e02-4ab59a6aebf9 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:07.262802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 255/518] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:07.263767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454230b-daca-44e5-9e02-4ab59a6aebf9 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:07.361945 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:07.374163 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db3b89-dac1-434a-823a-09780dfd115c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:07.389961 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454230b-daca-44e5-9e02-4ab59a6aebf9 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:07.424390 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c51de1-0aa2-44cd-b3f0-f8ac6f97a031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:07.426245 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51de1-0aa2-44cd-b3f0-f8ac6f97a031 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:07.426754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51de1-0aa2-44cd-b3f0-f8ac6f97a031 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:07.427244 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c51de1-0aa2-44cd-b3f0-f8ac6f97a031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:07.430157 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136566d-5d19-4ed7-96d7-a240102e63d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:07.430830 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136566d-5d19-4ed7-96d7-a240102e63d3 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:07.431138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136566d-5d19-4ed7-96d7-a240102e63d3 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:07.431477 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136566d-5d19-4ed7-96d7-a240102e63d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:07.442861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c51de1-0aa2-44cd-b3f0-f8ac6f97a031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:07.444815 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c51de1-0aa2-44cd-b3f0-f8ac6f97a031 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-c3c51de1-0aa2-44cd-b3f0-f8ac6f97a031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:07.465261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:07.466792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:07.467826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:07.467826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:07.467826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:07.467826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:07.467826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 264/519] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:07.467826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c51de1-0aa2-44cd-b3f0-f8ac6f97a031 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:07.529438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136566d-5d19-4ed7-96d7-a240102e63d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:07.530907 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136566d-5d19-4ed7-96d7-a240102e63d3 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:07.537601 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:07.588571 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c51de1-0aa2-44cd-b3f0-f8ac6f97a031 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:07.625389 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:07.627981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d9177-f738-4c94-93d7-53a3272a0722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:07.628690 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d9177-f738-4c94-93d7-53a3272a0722 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:07.628979 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d9177-f738-4c94-93d7-53a3272a0722 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:07.629250 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791d9177-f738-4c94-93d7-53a3272a0722 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:07.629993 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d9177-f738-4c94-93d7-53a3272a0722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:07.630303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d9177-f738-4c94-93d7-53a3272a0722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:07.631718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791d9177-f738-4c94-93d7-53a3272a0722 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:07.645896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 265/520] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:07.646768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d9177-f738-4c94-93d7-53a3272a0722 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:07.659552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 256/521] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:07.660872 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136566d-5d19-4ed7-96d7-a240102e63d3 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:07.773106 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791d9177-f738-4c94-93d7-53a3272a0722 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:07.781821 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136566d-5d19-4ed7-96d7-a240102e63d3 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:07.789103 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:07.819405 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d493e9-5695-4b6f-886b-7d46c47eb81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:07.820281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d493e9-5695-4b6f-886b-7d46c47eb81c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:07.820744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d493e9-5695-4b6f-886b-7d46c47eb81c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:07.821129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d493e9-5695-4b6f-886b-7d46c47eb81c None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:07.822465 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d493e9-5695-4b6f-886b-7d46c47eb81c None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:07.823053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d493e9-5695-4b6f-886b-7d46c47eb81c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:07.825732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d42a9-0f28-4016-87de-f3e2b28caf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:07.825981 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d493e9-5695-4b6f-886b-7d46c47eb81c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:07.827860 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d42a9-0f28-4016-87de-f3e2b28caf28 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:07.828710 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d42a9-0f28-4016-87de-f3e2b28caf28 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:07.830343 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d42a9-0f28-4016-87de-f3e2b28caf28 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:07.850549 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 257/522] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:07.850549 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d493e9-5695-4b6f-886b-7d46c47eb81c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:07.850869 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d42a9-0f28-4016-87de-f3e2b28caf28 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:07.850869 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d42a9-0f28-4016-87de-f3e2b28caf28 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-197d42a9-0f28-4016-87de-f3e2b28caf28 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:07.868890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:07.870020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 266/523] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:07.871131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d42a9-0f28-4016-87de-f3e2b28caf28 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:07.905870 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:07.983249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d493e9-5695-4b6f-886b-7d46c47eb81c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:08.000207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d42a9-0f28-4016-87de-f3e2b28caf28 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:08.020048 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fcf03-f56f-49cb-b27b-0db7f238508f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:08.020800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fcf03-f56f-49cb-b27b-0db7f238508f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:08.021083 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fcf03-f56f-49cb-b27b-0db7f238508f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:08.021360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fcf03-f56f-49cb-b27b-0db7f238508f None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:08.028841 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fcf03-f56f-49cb-b27b-0db7f238508f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:08.030224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fcf03-f56f-49cb-b27b-0db7f238508f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:08.043224 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599342c3-5421-4046-9360-5b9d0dc0710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:08.044435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599342c3-5421-4046-9360-5b9d0dc0710d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:08.044899 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599342c3-5421-4046-9360-5b9d0dc0710d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:08.045354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599342c3-5421-4046-9360-5b9d0dc0710d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-d73fcf03-f56f-49cb-b27b-0db7f238508f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:08.046118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:08.047324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:08.048379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:08.048379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:08.048379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:08.048379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:08.048379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:08.048379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:08.048379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:08.048379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:08.048379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 258/524] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:08.048379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fcf03-f56f-49cb-b27b-0db7f238508f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:08.066975 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599342c3-5421-4046-9360-5b9d0dc0710d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:08.068512 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599342c3-5421-4046-9360-5b9d0dc0710d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-599342c3-5421-4046-9360-5b9d0dc0710d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:08.089543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:08.091560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:08.093647 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:08.093647 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:08.093647 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:08.093647 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:08.093647 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 267/525] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:08.093647 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599342c3-5421-4046-9360-5b9d0dc0710d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:08.119826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:08.187155 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:08.193079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fcf03-f56f-49cb-b27b-0db7f238508f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:08.215436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599342c3-5421-4046-9360-5b9d0dc0710d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:08.239380 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf95320-acb4-458e-b14d-dd3840ade8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:08.240301 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf95320-acb4-458e-b14d-dd3840ade8b4 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:08.240624 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf95320-acb4-458e-b14d-dd3840ade8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:08.240902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf95320-acb4-458e-b14d-dd3840ade8b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:08.248875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf95320-acb4-458e-b14d-dd3840ade8b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:08.250399 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf95320-acb4-458e-b14d-dd3840ade8b4 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:08.255938 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13cd5b-1482-47fc-9646-47df9147a0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:08.256264 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13cd5b-1482-47fc-9646-47df9147a0f0 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:08.256580 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13cd5b-1482-47fc-9646-47df9147a0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:08.256854 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13cd5b-1482-47fc-9646-47df9147a0f0 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:08.257693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13cd5b-1482-47fc-9646-47df9147a0f0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:08.258052 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13cd5b-1482-47fc-9646-47df9147a0f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:08.259719 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13cd5b-1482-47fc-9646-47df9147a0f0 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-dbf95320-acb4-458e-b14d-dd3840ade8b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:08.266862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:08.268451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:08.269497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:08.269497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:08.269497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:08.269497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:08.269497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:08.269497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:08.269497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:08.269497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:08.269497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 259/526] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:08.269497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf95320-acb4-458e-b14d-dd3840ade8b4 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:08.286523 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 268/527] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 20:50:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:08.287734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13cd5b-1482-47fc-9646-47df9147a0f0 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:08.392716 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf95320-acb4-458e-b14d-dd3840ade8b4 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:08.401917 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:08.431814 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13cd5b-1482-47fc-9646-47df9147a0f0 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:08.433071 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4315c-f44a-44f3-897c-e12e17f263f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:08.433071 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4315c-f44a-44f3-897c-e12e17f263f2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:08.433562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4315c-f44a-44f3-897c-e12e17f263f2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:08.433840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4315c-f44a-44f3-897c-e12e17f263f2 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:08.434653 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4315c-f44a-44f3-897c-e12e17f263f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:08.434974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4315c-f44a-44f3-897c-e12e17f263f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:08.436450 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4315c-f44a-44f3-897c-e12e17f263f2 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:08.460185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 260/528] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:08.460997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4315c-f44a-44f3-897c-e12e17f263f2 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:08.479614 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203284e-547d-4d63-9fe6-4a6257db286b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:08.480292 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203284e-547d-4d63-9fe6-4a6257db286b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:08.480632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203284e-547d-4d63-9fe6-4a6257db286b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:08.480893 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203284e-547d-4d63-9fe6-4a6257db286b None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:08.481632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203284e-547d-4d63-9fe6-4a6257db286b None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:08.481945 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203284e-547d-4d63-9fe6-4a6257db286b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:08.483369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203284e-547d-4d63-9fe6-4a6257db286b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:08.486358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:08.497601 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 269/529] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:08.498472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203284e-547d-4d63-9fe6-4a6257db286b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:08.604498 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4315c-f44a-44f3-897c-e12e17f263f2 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:08.626292 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203284e-547d-4d63-9fe6-4a6257db286b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:08.650556 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d95ed-5d2b-4812-8e8d-5ecbea6bac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:08.651874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d95ed-5d2b-4812-8e8d-5ecbea6bac75 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:08.652450 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d95ed-5d2b-4812-8e8d-5ecbea6bac75 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:08.652922 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d95ed-5d2b-4812-8e8d-5ecbea6bac75 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:08.654053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d95ed-5d2b-4812-8e8d-5ecbea6bac75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:08.654375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d95ed-5d2b-4812-8e8d-5ecbea6bac75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:08.655972 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d95ed-5d2b-4812-8e8d-5ecbea6bac75 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:08.674701 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bdec-d703-454b-8694-73e53f9e112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:08.675607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bdec-d703-454b-8694-73e53f9e112f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:08.676006 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bdec-d703-454b-8694-73e53f9e112f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:08.676822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692bdec-d703-454b-8694-73e53f9e112f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:08.676957 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 261/530] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:08.678067 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d95ed-5d2b-4812-8e8d-5ecbea6bac75 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:08.693505 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:08.779559 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:08.784010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692bdec-d703-454b-8694-73e53f9e112f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:08.787026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692bdec-d703-454b-8694-73e53f9e112f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:08.802771 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d95ed-5d2b-4812-8e8d-5ecbea6bac75 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:08.841295 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e9819-8193-41de-9ce1-cc20b5bf6f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:08.841958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e9819-8193-41de-9ce1-cc20b5bf6f5b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:08.842247 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e9819-8193-41de-9ce1-cc20b5bf6f5b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:08.842523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e9819-8193-41de-9ce1-cc20b5bf6f5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:08.843500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e9819-8193-41de-9ce1-cc20b5bf6f5b None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=77427e1028d349b4a740efe0a985e73b) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:08.843828 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e9819-8193-41de-9ce1-cc20b5bf6f5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:08.845461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e9819-8193-41de-9ce1-cc20b5bf6f5b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:08.852222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 262/531] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:08 2022] GET /identity/v3/role_assignments?user.id=77427e1028d349b4a740efe0a985e73b&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:08.853356 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e9819-8193-41de-9ce1-cc20b5bf6f5b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:08.972378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 270/532] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:08.972378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bdec-d703-454b-8694-73e53f9e112f None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:09.024720 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e9819-8193-41de-9ce1-cc20b5bf6f5b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:09.054259 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:09.078215 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f473a-8aeb-411b-be33-f63679737f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:09.079744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f473a-8aeb-411b-be33-f63679737f62 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:09.080055 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f473a-8aeb-411b-be33-f63679737f62 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:09.080447 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f473a-8aeb-411b-be33-f63679737f62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:09.090855 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f473a-8aeb-411b-be33-f63679737f62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:09.092457 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f473a-8aeb-411b-be33-f63679737f62 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-a62f473a-8aeb-411b-be33-f63679737f62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:09.119412 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:09.120532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:09.121558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:09.121558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:09.121558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:09.121558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:09.121558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:09.121558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:09.121558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:09.121558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:09.128500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 263/533] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:09.129518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f473a-8aeb-411b-be33-f63679737f62 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:09.133046 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:09.148653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692bdec-d703-454b-8694-73e53f9e112f None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:09.196576 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127d333-9f23-411d-86df-0c2fdb1a34c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:09.197159 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127d333-9f23-411d-86df-0c2fdb1a34c7 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:09.197442 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127d333-9f23-411d-86df-0c2fdb1a34c7 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:09.197723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127d333-9f23-411d-86df-0c2fdb1a34c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:09.198843 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127d333-9f23-411d-86df-0c2fdb1a34c7 None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=6ab3de874a0f475c8745831e76a6e4d0) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:09.198993 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127d333-9f23-411d-86df-0c2fdb1a34c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:09.200860 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127d333-9f23-411d-86df-0c2fdb1a34c7 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:09.212668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 271/534] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:08 2022] GET /identity/v3/role_assignments?user.id=6ab3de874a0f475c8745831e76a6e4d0&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:09.213698 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127d333-9f23-411d-86df-0c2fdb1a34c7 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:09.280476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f473a-8aeb-411b-be33-f63679737f62 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:09.329097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bc95b-1d83-49b9-b55c-d08867141735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:09.330919 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bc95b-1d83-49b9-b55c-d08867141735 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:09.331436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bc95b-1d83-49b9-b55c-d08867141735 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:09.331938 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494bc95b-1d83-49b9-b55c-d08867141735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:09.347146 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494bc95b-1d83-49b9-b55c-d08867141735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:09.349347 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494bc95b-1d83-49b9-b55c-d08867141735 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-494bc95b-1d83-49b9-b55c-d08867141735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:09.372921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:09.374168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:09.375286 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:09.375286 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:09.375286 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:09.375286 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:09.378103 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 264/535] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:09.385708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bc95b-1d83-49b9-b55c-d08867141735 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:09.443224 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127d333-9f23-411d-86df-0c2fdb1a34c7 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:09.453645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:09.508987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:09.516182 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622bcf6-70b2-4322-8e69-e5e86fdfbe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:09.517497 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622bcf6-70b2-4322-8e69-e5e86fdfbe0f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:09.517794 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622bcf6-70b2-4322-8e69-e5e86fdfbe0f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:09.518080 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622bcf6-70b2-4322-8e69-e5e86fdfbe0f None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:09.518924 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622bcf6-70b2-4322-8e69-e5e86fdfbe0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:09.519239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622bcf6-70b2-4322-8e69-e5e86fdfbe0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:09.520832 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e622bcf6-70b2-4322-8e69-e5e86fdfbe0f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:09.554929 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 272/536] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:09.562586 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622bcf6-70b2-4322-8e69-e5e86fdfbe0f None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:09.586253 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494bc95b-1d83-49b9-b55c-d08867141735 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:09.664867 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef136079-e1b5-46cc-8e4a-77b3af515185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:09.665637 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef136079-e1b5-46cc-8e4a-77b3af515185 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:09.666037 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef136079-e1b5-46cc-8e4a-77b3af515185 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:09.666324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef136079-e1b5-46cc-8e4a-77b3af515185 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:09.667074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef136079-e1b5-46cc-8e4a-77b3af515185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:09.667396 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef136079-e1b5-46cc-8e4a-77b3af515185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:09.668829 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef136079-e1b5-46cc-8e4a-77b3af515185 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:09.691271 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 265/537] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:09.692905 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d7442-c2a2-46e7-b0d3-b8974550e266 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:09.693200 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d7442-c2a2-46e7-b0d3-b8974550e266 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:09.693480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d7442-c2a2-46e7-b0d3-b8974550e266 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:09.698982 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 266/538] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:09.700010 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d7442-c2a2-46e7-b0d3-b8974550e266 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:09.706456 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622bcf6-70b2-4322-8e69-e5e86fdfbe0f None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:09.756034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8718a6-b23e-4700-9aa0-71c4fa8e005f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:09.757175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8718a6-b23e-4700-9aa0-71c4fa8e005f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:09.758106 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8718a6-b23e-4700-9aa0-71c4fa8e005f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:09.758497 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8718a6-b23e-4700-9aa0-71c4fa8e005f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:09.759750 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8718a6-b23e-4700-9aa0-71c4fa8e005f None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=e2f9d95b4a2d4816adceffdd1a49186f) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:09.760169 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8718a6-b23e-4700-9aa0-71c4fa8e005f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:09.762564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8718a6-b23e-4700-9aa0-71c4fa8e005f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:09.771434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 273/539] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:09 2022] GET /identity/v3/role_assignments?user.id=e2f9d95b4a2d4816adceffdd1a49186f&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:09.773844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e483ec0-b5d0-4fda-9fde-ef42d08bb1fd None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:09.774328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e483ec0-b5d0-4fda-9fde-ef42d08bb1fd None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:09.774680 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e483ec0-b5d0-4fda-9fde-ef42d08bb1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:09.795225 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-5e483ec0-b5d0-4fda-9fde-ef42d08bb1fd None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:09.796722 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-5e483ec0-b5d0-4fda-9fde-ef42d08bb1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:09.840476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:09.848951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d7442-c2a2-46e7-b0d3-b8974550e266 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:09.882429 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:09.940981 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0f2c2-1ccb-432a-80f2-5b996320abdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:09.941805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0f2c2-1ccb-432a-80f2-5b996320abdd None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:09.942091 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0f2c2-1ccb-432a-80f2-5b996320abdd None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:09.942371 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0f2c2-1ccb-432a-80f2-5b996320abdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:09.947358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 274/540] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:09.953387 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0f2c2-1ccb-432a-80f2-5b996320abdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:09.960513 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0f2c2-1ccb-432a-80f2-5b996320abdd None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:09.972853 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e483ec0-b5d0-4fda-9fde-ef42d08bb1fd None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-2df0f2c2-1ccb-432a-80f2-5b996320abdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:09.986166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:09.987235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:09.988245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:09.988245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:09.988245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:09.988245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:09.988245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:09.988245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:09.988245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:09.988245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:09.991970 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 267/541] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:09.997391 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0f2c2-1ccb-432a-80f2-5b996320abdd None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:10.086737 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e483ec0-b5d0-4fda-9fde-ef42d08bb1fd None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:10.126009 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394e083-51ea-48ca-8535-71c19c42de34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:10.126831 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394e083-51ea-48ca-8535-71c19c42de34 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:10.127118 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394e083-51ea-48ca-8535-71c19c42de34 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:10.127393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394e083-51ea-48ca-8535-71c19c42de34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:10.139792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394e083-51ea-48ca-8535-71c19c42de34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-2394e083-51ea-48ca-8535-71c19c42de34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:10.148820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:10.149976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:10.150957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 275/542] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:10.153965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0f2c2-1ccb-432a-80f2-5b996320abdd None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:10.160159 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394e083-51ea-48ca-8535-71c19c42de34 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:10.181857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:10.213457 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec1d8a-540d-4b16-868f-08ea6cf740ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:10.214727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec1d8a-540d-4b16-868f-08ea6cf740ef None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:10.215282 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec1d8a-540d-4b16-868f-08ea6cf740ef None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:10.215804 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec1d8a-540d-4b16-868f-08ea6cf740ef None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:10.217107 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec1d8a-540d-4b16-868f-08ea6cf740ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:10.217699 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec1d8a-540d-4b16-868f-08ea6cf740ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:10.219516 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec1d8a-540d-4b16-868f-08ea6cf740ef None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:10.246427 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 268/543] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:10.253250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec1d8a-540d-4b16-868f-08ea6cf740ef None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:10.272400 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394e083-51ea-48ca-8535-71c19c42de34 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:10.308062 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab3ff7d-dbb7-4daa-89f4-3abbde7b1fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:10.309043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3ff7d-dbb7-4daa-89f4-3abbde7b1fc2 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:10.309339 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3ff7d-dbb7-4daa-89f4-3abbde7b1fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:10.309621 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab3ff7d-dbb7-4daa-89f4-3abbde7b1fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:10.311428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab3ff7d-dbb7-4daa-89f4-3abbde7b1fc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:10.311428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab3ff7d-dbb7-4daa-89f4-3abbde7b1fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:10.312270 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab3ff7d-dbb7-4daa-89f4-3abbde7b1fc2 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:10.324189 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 276/544] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:10.326187 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:10.330309 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab3ff7d-dbb7-4daa-89f4-3abbde7b1fc2 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:10.420723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:10.441152 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab3ff7d-dbb7-4daa-89f4-3abbde7b1fc2 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:10.473628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec1d8a-540d-4b16-868f-08ea6cf740ef None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:10.477639 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8c35-cb19-4266-82f1-8e7177e8ff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:10.478521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8c35-cb19-4266-82f1-8e7177e8ff26 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:10.478882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8c35-cb19-4266-82f1-8e7177e8ff26 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:10.479227 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8c35-cb19-4266-82f1-8e7177e8ff26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:10.490687 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a8c35-cb19-4266-82f1-8e7177e8ff26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:10.492254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a8c35-cb19-4266-82f1-8e7177e8ff26 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-798a8c35-cb19-4266-82f1-8e7177e8ff26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:10.511605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:10.512925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:10.514251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:10.514251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:10.514251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:10.514251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:10.514531 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 277/545] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:10.521447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8c35-cb19-4266-82f1-8e7177e8ff26 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:10.556325 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd470c-d8a0-4e46-9085-41e632a8c4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:10.557026 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd470c-d8a0-4e46-9085-41e632a8c4ff None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:10.557310 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd470c-d8a0-4e46-9085-41e632a8c4ff None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:10.557588 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bd470c-d8a0-4e46-9085-41e632a8c4ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:10.558700 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd470c-d8a0-4e46-9085-41e632a8c4ff None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=4b3f694758fa483ca497fbea18061cb6) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:10.559013 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd470c-d8a0-4e46-9085-41e632a8c4ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:10.560686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bd470c-d8a0-4e46-9085-41e632a8c4ff None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:10.574817 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 269/546] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:10 2022] GET /identity/v3/role_assignments?user.id=4b3f694758fa483ca497fbea18061cb6&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:10.637584 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8c35-cb19-4266-82f1-8e7177e8ff26 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:10.675771 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f8056-b824-41cd-84f2-490b6e41b3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:10.676497 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f8056-b824-41cd-84f2-490b6e41b3f5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:10.676926 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f8056-b824-41cd-84f2-490b6e41b3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:10.677215 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f8056-b824-41cd-84f2-490b6e41b3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:10.677972 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f8056-b824-41cd-84f2-490b6e41b3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:10.678289 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f8056-b824-41cd-84f2-490b6e41b3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:10.679724 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f8056-b824-41cd-84f2-490b6e41b3f5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:10.681511 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:10.694828 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 278/547] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:10.700756 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd470c-d8a0-4e46-9085-41e632a8c4ff None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:10.872746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bd470c-d8a0-4e46-9085-41e632a8c4ff None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:10.876981 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:10.919272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414f392-1ae2-4031-879c-f90b8d3d7953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:10.920090 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414f392-1ae2-4031-879c-f90b8d3d7953 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:10.920427 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414f392-1ae2-4031-879c-f90b8d3d7953 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:10.920709 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414f392-1ae2-4031-879c-f90b8d3d7953 None admin] PATH_INFO: `/v3/users/swift` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:10.937561 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414f392-1ae2-4031-879c-f90b8d3d7953 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:10.939169 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414f392-1ae2-4031-879c-f90b8d3d7953 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-e414f392-1ae2-4031-879c-f90b8d3d7953 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:10.962171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:10.963293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:10.964443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:10.964443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:10.964443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:10.964443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:10.964443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:10.964443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:10.964443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:10.964443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:10.964443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:10.964443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:10.964443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 270/548] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:10.969524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f8056-b824-41cd-84f2-490b6e41b3f5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:11.086420 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f8056-b824-41cd-84f2-490b6e41b3f5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:11.130605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4f78-b215-422b-859b-2e8da59db03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:11.131245 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4f78-b215-422b-859b-2e8da59db03b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:11.131538 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4f78-b215-422b-859b-2e8da59db03b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:11.131800 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d4f78-b215-422b-859b-2e8da59db03b None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:11.132727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d4f78-b215-422b-859b-2e8da59db03b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:11.133046 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d4f78-b215-422b-859b-2e8da59db03b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:11.134627 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d4f78-b215-422b-859b-2e8da59db03b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:11.154157 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 279/549] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:50:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:11.160307 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414f392-1ae2-4031-879c-f90b8d3d7953 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:11.288905 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414f392-1ae2-4031-879c-f90b8d3d7953 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:11.326564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5e13e-8ccc-4667-b586-db0f3e1ec505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:11.327280 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5e13e-8ccc-4667-b586-db0f3e1ec505 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:11.327531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5e13e-8ccc-4667-b586-db0f3e1ec505 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:11.327855 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5e13e-8ccc-4667-b586-db0f3e1ec505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:11.370045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:11.429907 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5e13e-8ccc-4667-b586-db0f3e1ec505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:11.431434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5e13e-8ccc-4667-b586-db0f3e1ec505 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:11.576452 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 271/550] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:11.582675 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4f78-b215-422b-859b-2e8da59db03b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:11.672759 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:11.702057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d4f78-b215-422b-859b-2e8da59db03b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:11.743194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3402a3-406b-4430-8c14-f745f2114a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:11.744029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3402a3-406b-4430-8c14-f745f2114a84 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:11.744315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3402a3-406b-4430-8c14-f745f2114a84 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:11.744625 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3402a3-406b-4430-8c14-f745f2114a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:11.757410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3402a3-406b-4430-8c14-f745f2114a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:11.759058 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3402a3-406b-4430-8c14-f745f2114a84 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-6e3402a3-406b-4430-8c14-f745f2114a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:11.781040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:11.782153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:11.783206 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:11.783206 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:11.783206 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:11.783206 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:11.783206 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 280/551] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:11.788553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5e13e-8ccc-4667-b586-db0f3e1ec505 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:11.900993 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:11.959553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5e13e-8ccc-4667-b586-db0f3e1ec505 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:12.010448 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298dfb2a-b24a-469a-891b-23a67ed6bc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:12.011212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dfb2a-b24a-469a-891b-23a67ed6bc23 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.011508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dfb2a-b24a-469a-891b-23a67ed6bc23 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.011796 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298dfb2a-b24a-469a-891b-23a67ed6bc23 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.012709 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298dfb2a-b24a-469a-891b-23a67ed6bc23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:12.013026 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298dfb2a-b24a-469a-891b-23a67ed6bc23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:12.014989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298dfb2a-b24a-469a-891b-23a67ed6bc23 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:12.030614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 272/552] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:12.036633 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3402a3-406b-4430-8c14-f745f2114a84 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:12.058695 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab3420-b7bd-49ec-9f2b-976db90984f1 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.059037 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab3420-b7bd-49ec-9f2b-976db90984f1 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.059260 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab3420-b7bd-49ec-9f2b-976db90984f1 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.061084 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 273/553] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:12.073345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24f8c0-83cd-4d82-8702-ae50728842cd None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.074010 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24f8c0-83cd-4d82-8702-ae50728842cd None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.074467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24f8c0-83cd-4d82-8702-ae50728842cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.113909 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-0b24f8c0-83cd-4d82-8702-ae50728842cd None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:12.115186 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-0b24f8c0-83cd-4d82-8702-ae50728842cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:12.198855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3402a3-406b-4430-8c14-f745f2114a84 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:12.202118 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:12.207481 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 274/554] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:12.209667 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ce6cd-800e-4b71-bf9a-613b7770d87f None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.210188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ce6cd-800e-4b71-bf9a-613b7770d87f None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.210695 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ce6cd-800e-4b71-bf9a-613b7770d87f None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.212606 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 275/555] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:12.218136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396977-57e9-4181-b488-ee6686e7e0e5 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.218434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396977-57e9-4181-b488-ee6686e7e0e5 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.223750 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65396977-57e9-4181-b488-ee6686e7e0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.239532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-65396977-57e9-4181-b488-ee6686e7e0e5 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:12.241421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-65396977-57e9-4181-b488-ee6686e7e0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:12.244716 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f529cc-f31b-4449-a40d-4ed6f2551bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:12.245872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f529cc-f31b-4449-a40d-4ed6f2551bde None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.246417 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f529cc-f31b-4449-a40d-4ed6f2551bde None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.247025 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f529cc-f31b-4449-a40d-4ed6f2551bde None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.256421 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f529cc-f31b-4449-a40d-4ed6f2551bde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:12.258281 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f529cc-f31b-4449-a40d-4ed6f2551bde None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-e2f529cc-f31b-4449-a40d-4ed6f2551bde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:12.276304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:12.277380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:12.278464 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:12.278464 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:12.278464 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:12.278464 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:12.278464 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:12.278464 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:12.278464 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:12.278464 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:12.281290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 281/556] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:12.282471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f529cc-f31b-4449-a40d-4ed6f2551bde None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:12.294568 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:12.332487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 276/557] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:50:12.334257 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65396977-57e9-4181-b488-ee6686e7e0e5 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:12.411701 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f529cc-f31b-4449-a40d-4ed6f2551bde None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:12.503115 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65396977-57e9-4181-b488-ee6686e7e0e5 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:12.507300 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31ea8d-2b03-4ef7-9b86-d64c500d0d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:12.508481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ea8d-2b03-4ef7-9b86-d64c500d0d17 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.509018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ea8d-2b03-4ef7-9b86-d64c500d0d17 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.509528 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e31ea8d-2b03-4ef7-9b86-d64c500d0d17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.524969 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e31ea8d-2b03-4ef7-9b86-d64c500d0d17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-8e31ea8d-2b03-4ef7-9b86-d64c500d0d17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:12.537188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:12.538410 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:12.539350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:12.541953 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 282/558] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:12.543707 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31ea8d-2b03-4ef7-9b86-d64c500d0d17 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:12.555823 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42ed5d-80a9-4925-8734-00d539cbe345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:12.556539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42ed5d-80a9-4925-8734-00d539cbe345 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.556826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42ed5d-80a9-4925-8734-00d539cbe345 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.557106 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c42ed5d-80a9-4925-8734-00d539cbe345 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.557918 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42ed5d-80a9-4925-8734-00d539cbe345 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:12.558486 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42ed5d-80a9-4925-8734-00d539cbe345 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:12.560530 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c42ed5d-80a9-4925-8734-00d539cbe345 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:12.577623 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 277/559] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:12.578731 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42ed5d-80a9-4925-8734-00d539cbe345 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:12.580982 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:12.635621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:12.689503 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e31ea8d-2b03-4ef7-9b86-d64c500d0d17 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:12.724876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c42ed5d-80a9-4925-8734-00d539cbe345 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:12.742721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1584bdd-e51a-4911-b4ff-55bcdac1e09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:12.744311 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1584bdd-e51a-4911-b4ff-55bcdac1e09c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.744638 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1584bdd-e51a-4911-b4ff-55bcdac1e09c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.745534 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1584bdd-e51a-4911-b4ff-55bcdac1e09c None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.759295 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1584bdd-e51a-4911-b4ff-55bcdac1e09c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:12.767427 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2d597-e4ce-4f60-9888-fd93a948f9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:12.768522 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2d597-e4ce-4f60-9888-fd93a948f9f5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.769063 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2d597-e4ce-4f60-9888-fd93a948f9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-e1584bdd-e51a-4911-b4ff-55bcdac1e09c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:12.769199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:12.770306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:12.771251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 283/560] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:12.772066 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1584bdd-e51a-4911-b4ff-55bcdac1e09c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:12.778950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2d597-e4ce-4f60-9888-fd93a948f9f5 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.778950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2d597-e4ce-4f60-9888-fd93a948f9f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:12.778950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2d597-e4ce-4f60-9888-fd93a948f9f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:12.778950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2d597-e4ce-4f60-9888-fd93a948f9f5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:12.789468 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 278/561] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:12.790880 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2d597-e4ce-4f60-9888-fd93a948f9f5 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:12.883521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:12.904182 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:12.905361 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1584bdd-e51a-4911-b4ff-55bcdac1e09c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:12.907893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2d597-e4ce-4f60-9888-fd93a948f9f5 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:12.945038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed800c-9838-4328-a489-1836f9913e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:12.945738 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed800c-9838-4328-a489-1836f9913e08 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.946034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed800c-9838-4328-a489-1836f9913e08 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.946315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed800c-9838-4328-a489-1836f9913e08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.947164 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed800c-9838-4328-a489-1836f9913e08 None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=ba6e1d31d8004f9fa956b78d8716d6e6) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:12.947483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed800c-9838-4328-a489-1836f9913e08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:12.949020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed800c-9838-4328-a489-1836f9913e08 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:12.953379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f52a5-fe88-4e9b-ba28-944a54971541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:12.954888 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f52a5-fe88-4e9b-ba28-944a54971541 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.955564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f52a5-fe88-4e9b-ba28-944a54971541 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.956050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f52a5-fe88-4e9b-ba28-944a54971541 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.957209 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f52a5-fe88-4e9b-ba28-944a54971541 None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:12.957753 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f52a5-fe88-4e9b-ba28-944a54971541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:12.959369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f52a5-fe88-4e9b-ba28-944a54971541 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:12.960748 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 284/562] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:12 2022] GET /identity/v3/role_assignments?user.id=ba6e1d31d8004f9fa956b78d8716d6e6&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:12.962336 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed800c-9838-4328-a489-1836f9913e08 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:12.972573 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 279/563] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:12.974360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f47ec-97ed-49b2-9334-e7bdd9416f73 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:12.975467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f47ec-97ed-49b2-9334-e7bdd9416f73 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:12.975959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f47ec-97ed-49b2-9334-e7bdd9416f73 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:12.978378 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 280/564] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:12.979517 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f47ec-97ed-49b2-9334-e7bdd9416f73 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:13.086270 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed800c-9838-4328-a489-1836f9913e08 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:13.112324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f47ec-97ed-49b2-9334-e7bdd9416f73 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:13.129399 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da991646-cb92-4e9c-a08c-c7d7d7072063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:13.130706 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da991646-cb92-4e9c-a08c-c7d7d7072063 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:13.131213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da991646-cb92-4e9c-a08c-c7d7d7072063 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:13.131752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da991646-cb92-4e9c-a08c-c7d7d7072063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:13.146061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da991646-cb92-4e9c-a08c-c7d7d7072063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:13.147989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da991646-cb92-4e9c-a08c-c7d7d7072063 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:13.158747 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:13.160862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb617c1-f6f1-4730-a33c-f1640b46c8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:13.161890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb617c1-f6f1-4730-a33c-f1640b46c8cb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:13.162394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb617c1-f6f1-4730-a33c-f1640b46c8cb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:13.162647 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb617c1-f6f1-4730-a33c-f1640b46c8cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-da991646-cb92-4e9c-a08c-c7d7d7072063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:13.170381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:13.179369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:13.181138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:13.181138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:13.181138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:13.181138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:13.181138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 285/565] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:13.181138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fa58a-ec6f-49d9-8e7a-ae15fb6c3fdc None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:13.181138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fa58a-ec6f-49d9-8e7a-ae15fb6c3fdc None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:13.181138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fa58a-ec6f-49d9-8e7a-ae15fb6c3fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:13.181697 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb617c1-f6f1-4730-a33c-f1640b46c8cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:13.181944 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb617c1-f6f1-4730-a33c-f1640b46c8cb None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:13.193972 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-b50fa58a-ec6f-49d9-8e7a-ae15fb6c3fdc None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:13.195307 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-b50fa58a-ec6f-49d9-8e7a-ae15fb6c3fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-1cb617c1-f6f1-4730-a33c-f1640b46c8cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:13.201997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:13.203029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:13.204028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:13.204028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:13.204028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:13.204028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:13.204028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 281/566] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:13.205197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb617c1-f6f1-4730-a33c-f1640b46c8cb None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:13.232951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:13.300531 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 286/567] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:13.300531 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fa58a-ec6f-49d9-8e7a-ae15fb6c3fdc None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:13.339475 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb617c1-f6f1-4730-a33c-f1640b46c8cb None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:13.379292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839926bd-c174-4203-863f-343554f72256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:13.380139 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839926bd-c174-4203-863f-343554f72256 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:13.380525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839926bd-c174-4203-863f-343554f72256 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:13.380808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839926bd-c174-4203-863f-343554f72256 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:13.381573 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839926bd-c174-4203-863f-343554f72256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:13.381859 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839926bd-c174-4203-863f-343554f72256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:13.383236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839926bd-c174-4203-863f-343554f72256 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:13.400193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 282/568] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:13.400193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a014bee-d853-4522-a10e-fe012304edd3 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:13.400193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a014bee-d853-4522-a10e-fe012304edd3 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:13.400193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a014bee-d853-4522-a10e-fe012304edd3 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:13.400193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 283/569] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:13.400193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a014bee-d853-4522-a10e-fe012304edd3 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:13.422734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fa58a-ec6f-49d9-8e7a-ae15fb6c3fdc None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:13.455791 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:13.462586 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898448a-71ab-4a01-8f92-8a8e3ecd9a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:13.463315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898448a-71ab-4a01-8f92-8a8e3ecd9a5a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:13.463604 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898448a-71ab-4a01-8f92-8a8e3ecd9a5a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:13.463824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898448a-71ab-4a01-8f92-8a8e3ecd9a5a None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:13.464777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898448a-71ab-4a01-8f92-8a8e3ecd9a5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:13.470468 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898448a-71ab-4a01-8f92-8a8e3ecd9a5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:13.470468 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898448a-71ab-4a01-8f92-8a8e3ecd9a5a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:13.479297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 287/570] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:13.480134 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898448a-71ab-4a01-8f92-8a8e3ecd9a5a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:13.488868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:13.516011 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a014bee-d853-4522-a10e-fe012304edd3 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:13.554126 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936b0bf-cc4f-498a-823e-ebd72761909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:13.555022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936b0bf-cc4f-498a-823e-ebd72761909b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:13.555382 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936b0bf-cc4f-498a-823e-ebd72761909b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:13.555725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936b0bf-cc4f-498a-823e-ebd72761909b None admin] PATH_INFO: `/v3/roles/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:13.566703 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936b0bf-cc4f-498a-823e-ebd72761909b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-9936b0bf-cc4f-498a-823e-ebd72761909b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:13.574621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:13.575868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:13.577051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 284/571] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:13.577838 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3846b0-8807-4f01-bd2c-fa02940ffa0e None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:13.577921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3846b0-8807-4f01-bd2c-fa02940ffa0e None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:13.578176 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3846b0-8807-4f01-bd2c-fa02940ffa0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:13.597907 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898448a-71ab-4a01-8f92-8a8e3ecd9a5a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:13.598963 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-2f3846b0-8807-4f01-bd2c-fa02940ffa0e None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:13.600184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-2f3846b0-8807-4f01-bd2c-fa02940ffa0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:13.633286 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39d65e-5599-4e3a-90ec-b683a537777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:13.634163 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39d65e-5599-4e3a-90ec-b683a537777b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:13.634447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39d65e-5599-4e3a-90ec-b683a537777b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:13.634723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39d65e-5599-4e3a-90ec-b683a537777b None admin] PATH_INFO: `/v3/users/nova` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:13.643567 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39d65e-5599-4e3a-90ec-b683a537777b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:13.645545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39d65e-5599-4e3a-90ec-b683a537777b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-bc39d65e-5599-4e3a-90ec-b683a537777b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:13.661646 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:13.662939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:13.664021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:13.664021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:13.664021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:13.664021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:13.664021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:13.664021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:13.664021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:13.664021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:13.664021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:13.664021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:13.664021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 288/572] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:50:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:13.664807 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39d65e-5599-4e3a-90ec-b683a537777b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:13.678146 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 285/573] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:13.679252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3846b0-8807-4f01-bd2c-fa02940ffa0e None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:13.735768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:13.745997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:13.790651 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39d65e-5599-4e3a-90ec-b683a537777b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:13.793791 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3846b0-8807-4f01-bd2c-fa02940ffa0e None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:13.829681 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065f71a-3f53-41a0-8a4a-5322d5c1bd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:13.830758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065f71a-3f53-41a0-8a4a-5322d5c1bd07 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:13.831289 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065f71a-3f53-41a0-8a4a-5322d5c1bd07 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:13.831602 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d80f10-4cef-4d9d-871f-f12e7ef55bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:13.831929 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065f71a-3f53-41a0-8a4a-5322d5c1bd07 None admin] PATH_INFO: `/v3/users/glance` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:13.832282 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d80f10-4cef-4d9d-871f-f12e7ef55bba None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:13.832596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d80f10-4cef-4d9d-871f-f12e7ef55bba None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:13.832877 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d80f10-4cef-4d9d-871f-f12e7ef55bba None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:13.833629 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d80f10-4cef-4d9d-871f-f12e7ef55bba None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:13.833938 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d80f10-4cef-4d9d-871f-f12e7ef55bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:13.835396 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d80f10-4cef-4d9d-871f-f12e7ef55bba None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:13.843282 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065f71a-3f53-41a0-8a4a-5322d5c1bd07 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:13.844084 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 286/574] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:13.845554 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d80f10-4cef-4d9d-871f-f12e7ef55bba None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:13.846024 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065f71a-3f53-41a0-8a4a-5322d5c1bd07 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-c065f71a-3f53-41a0-8a4a-5322d5c1bd07 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:13.865297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:13.871348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 289/575] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:13.872741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065f71a-3f53-41a0-8a4a-5322d5c1bd07 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:13.981131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065f71a-3f53-41a0-8a4a-5322d5c1bd07 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:13.982040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d80f10-4cef-4d9d-871f-f12e7ef55bba None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:13.992136 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:14.011749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:14.019334 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f3134-8387-408b-a245-a0a67dd58c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:14.020146 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f3134-8387-408b-a245-a0a67dd58c3a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:14.020546 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f3134-8387-408b-a245-a0a67dd58c3a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:14.020832 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f3134-8387-408b-a245-a0a67dd58c3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:14.021857 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51895476-87e9-4980-8280-541e6c878631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:14.022526 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51895476-87e9-4980-8280-541e6c878631 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:14.022815 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51895476-87e9-4980-8280-541e6c878631 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:14.023228 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51895476-87e9-4980-8280-541e6c878631 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:14.024097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51895476-87e9-4980-8280-541e6c878631 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:14.024438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51895476-87e9-4980-8280-541e6c878631 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:14.025871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51895476-87e9-4980-8280-541e6c878631 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:14.033084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f3134-8387-408b-a245-a0a67dd58c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-742f3134-8387-408b-a245-a0a67dd58c3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:14.041563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:14.042857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:14.044088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:14.047980 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 290/576] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:14.048873 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 287/577] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:50:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:14.048992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f3134-8387-408b-a245-a0a67dd58c3a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:14.050010 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51895476-87e9-4980-8280-541e6c878631 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:14.167555 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f3134-8387-408b-a245-a0a67dd58c3a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:14.172967 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51895476-87e9-4980-8280-541e6c878631 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:14.211196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a7b2a-f7a8-42bb-a4e4-c324117f92ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:14.211588 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fc269-b917-4b19-b4d9-f4ae342d32f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:14.212019 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a7b2a-f7a8-42bb-a4e4-c324117f92ff None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:14.212258 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fc269-b917-4b19-b4d9-f4ae342d32f6 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:14.212557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a7b2a-f7a8-42bb-a4e4-c324117f92ff None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:14.212654 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fc269-b917-4b19-b4d9-f4ae342d32f6 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:14.212732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a7b2a-f7a8-42bb-a4e4-c324117f92ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:14.212913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fc269-b917-4b19-b4d9-f4ae342d32f6 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:14.213737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fc269-b917-4b19-b4d9-f4ae342d32f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:14.214078 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fc269-b917-4b19-b4d9-f4ae342d32f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:14.215547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fc269-b917-4b19-b4d9-f4ae342d32f6 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:14.224479 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a7b2a-f7a8-42bb-a4e4-c324117f92ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:14.226009 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a7b2a-f7a8-42bb-a4e4-c324117f92ff None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:14.235024 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 288/578] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:50:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:14.237069 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fc269-b917-4b19-b4d9-f4ae342d32f6 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-1b5a7b2a-f7a8-42bb-a4e4-c324117f92ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:14.254356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:14.255599 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:14.257074 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:14.257074 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:14.257074 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:14.257074 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:14.257074 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 291/579] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:14.257074 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a7b2a-f7a8-42bb-a4e4-c324117f92ff None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:14.284305 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:14.291391 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:14.353247 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fc269-b917-4b19-b4d9-f4ae342d32f6 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:14.379160 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a7b2a-f7a8-42bb-a4e4-c324117f92ff None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:14.404476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530b125-e4af-4e84-b37b-ac0abebe7572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:14.405167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530b125-e4af-4e84-b37b-ac0abebe7572 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:14.405461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530b125-e4af-4e84-b37b-ac0abebe7572 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:14.405744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530b125-e4af-4e84-b37b-ac0abebe7572 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:14.406499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530b125-e4af-4e84-b37b-ac0abebe7572 None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:14.406902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530b125-e4af-4e84-b37b-ac0abebe7572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:14.409074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530b125-e4af-4e84-b37b-ac0abebe7572 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:14.429420 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 289/580] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:14.429813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631cd47-3d6a-4c0f-a7c6-ef190915ef1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:14.429976 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530b125-e4af-4e84-b37b-ac0abebe7572 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:14.430084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631cd47-3d6a-4c0f-a7c6-ef190915ef1b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:14.430084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631cd47-3d6a-4c0f-a7c6-ef190915ef1b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:14.430084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631cd47-3d6a-4c0f-a7c6-ef190915ef1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:14.435248 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631cd47-3d6a-4c0f-a7c6-ef190915ef1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:14.437111 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631cd47-3d6a-4c0f-a7c6-ef190915ef1b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-e631cd47-3d6a-4c0f-a7c6-ef190915ef1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:14.458753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:14.460040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:14.461397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:14.461397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:14.461397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:14.461397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:14.467622 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 292/581] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:14.467622 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631cd47-3d6a-4c0f-a7c6-ef190915ef1b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:14.555684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:14.560491 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530b125-e4af-4e84-b37b-ac0abebe7572 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:14.563546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:14.584220 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631cd47-3d6a-4c0f-a7c6-ef190915ef1b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:14.599910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a36fcd-ce03-44d1-91ec-dede841c4bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:14.600746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a36fcd-ce03-44d1-91ec-dede841c4bbb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:14.601027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a36fcd-ce03-44d1-91ec-dede841c4bbb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:14.601303 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a36fcd-ce03-44d1-91ec-dede841c4bbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:14.613258 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a36fcd-ce03-44d1-91ec-dede841c4bbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:14.614740 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a36fcd-ce03-44d1-91ec-dede841c4bbb None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:14.623995 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b17ed-7f90-46b6-a693-85d8006cfcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:14.624777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b17ed-7f90-46b6-a693-85d8006cfcdd None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:14.625147 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b17ed-7f90-46b6-a693-85d8006cfcdd None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:14.625535 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b17ed-7f90-46b6-a693-85d8006cfcdd None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:14.626407 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b17ed-7f90-46b6-a693-85d8006cfcdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:14.626796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b17ed-7f90-46b6-a693-85d8006cfcdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:14.628260 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b17ed-7f90-46b6-a693-85d8006cfcdd None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-a9a36fcd-ce03-44d1-91ec-dede841c4bbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:14.635029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:14.636236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:14.642800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:14.642800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:14.642800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:14.642800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:14.642800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 290/582] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:14.642800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a36fcd-ce03-44d1-91ec-dede841c4bbb None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:14.652498 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 293/583] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:14.653391 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b17ed-7f90-46b6-a693-85d8006cfcdd None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:14.763439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a36fcd-ce03-44d1-91ec-dede841c4bbb None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:14.782482 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b17ed-7f90-46b6-a693-85d8006cfcdd None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:14.807316 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d8056-9747-4af5-83d7-f871eb97a2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:14.808137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d8056-9747-4af5-83d7-f871eb97a2a1 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:14.808893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d8056-9747-4af5-83d7-f871eb97a2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:14.809065 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d8056-9747-4af5-83d7-f871eb97a2a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:14.821726 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d8056-9747-4af5-83d7-f871eb97a2a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:14.822781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0ca71-642d-40cb-bfa6-2f2ec3feb653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:14.823361 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d8056-9747-4af5-83d7-f871eb97a2a1 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:14.823516 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0ca71-642d-40cb-bfa6-2f2ec3feb653 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:14.823737 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0ca71-642d-40cb-bfa6-2f2ec3feb653 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:14.824016 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0ca71-642d-40cb-bfa6-2f2ec3feb653 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:14.824811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0ca71-642d-40cb-bfa6-2f2ec3feb653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:14.825126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0ca71-642d-40cb-bfa6-2f2ec3feb653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:14.826585 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0ca71-642d-40cb-bfa6-2f2ec3feb653 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:14.828565 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:14.845823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 294/584] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:14.845823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0ca71-642d-40cb-bfa6-2f2ec3feb653 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-ff8d8056-9747-4af5-83d7-f871eb97a2a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:14.847001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:14.848125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:14.849264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:14.849264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:14.849264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:14.849264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:14.849264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 291/585] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:14.849264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d8056-9747-4af5-83d7-f871eb97a2a1 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:14.863091 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:14.972271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0ca71-642d-40cb-bfa6-2f2ec3feb653 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:14.986883 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d8056-9747-4af5-83d7-f871eb97a2a1 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:15.010553 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a002e-5ba4-4eb3-825b-cc88e354903b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:15.011348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a002e-5ba4-4eb3-825b-cc88e354903b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.011710 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a002e-5ba4-4eb3-825b-cc88e354903b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.012057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a002e-5ba4-4eb3-825b-cc88e354903b None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.012911 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a002e-5ba4-4eb3-825b-cc88e354903b None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:15.013222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a002e-5ba4-4eb3-825b-cc88e354903b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:15.014728 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a002e-5ba4-4eb3-825b-cc88e354903b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:15.031644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 295/586] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:15.031881 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d24d03-8695-4744-be28-02438f8186f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:15.032006 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a002e-5ba4-4eb3-825b-cc88e354903b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:15.032134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d24d03-8695-4744-be28-02438f8186f5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.032134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d24d03-8695-4744-be28-02438f8186f5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.032134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d24d03-8695-4744-be28-02438f8186f5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.044168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d24d03-8695-4744-be28-02438f8186f5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:15.045787 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d24d03-8695-4744-be28-02438f8186f5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-94d24d03-8695-4744-be28-02438f8186f5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:15.062738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:15.063910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 292/587] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:15.065641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d24d03-8695-4744-be28-02438f8186f5 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:15.101096 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:15.148590 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:15.155709 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a002e-5ba4-4eb3-825b-cc88e354903b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:15.188500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d24d03-8695-4744-be28-02438f8186f5 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:15.192951 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a2bc1-be41-4789-b9ee-5bede714c48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:15.193832 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a2bc1-be41-4789-b9ee-5bede714c48a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.194189 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a2bc1-be41-4789-b9ee-5bede714c48a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.194536 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a2bc1-be41-4789-b9ee-5bede714c48a None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.201993 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a2bc1-be41-4789-b9ee-5bede714c48a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:15.203571 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a2bc1-be41-4789-b9ee-5bede714c48a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-937a2bc1-be41-4789-b9ee-5bede714c48a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:15.219108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:15.220962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:15.221962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:15.221962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:15.221962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:15.221962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:15.221962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:15.221962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:15.221962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:15.221962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:15.221962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 296/588] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:15.221962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a2bc1-be41-4789-b9ee-5bede714c48a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:15.231596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa575bf-26e8-4a22-bb99-dbb0330c3bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:15.232383 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa575bf-26e8-4a22-bb99-dbb0330c3bc5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.232742 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa575bf-26e8-4a22-bb99-dbb0330c3bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.233091 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa575bf-26e8-4a22-bb99-dbb0330c3bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.233894 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa575bf-26e8-4a22-bb99-dbb0330c3bc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:15.234278 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa575bf-26e8-4a22-bb99-dbb0330c3bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:15.235943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa575bf-26e8-4a22-bb99-dbb0330c3bc5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:15.249523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 293/589] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:15.250729 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa575bf-26e8-4a22-bb99-dbb0330c3bc5 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:15.345427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a2bc1-be41-4789-b9ee-5bede714c48a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:15.357974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa575bf-26e8-4a22-bb99-dbb0330c3bc5 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:15.359477 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:15.379794 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18ca8d-4a23-497d-963c-a86bf1613032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:15.380723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18ca8d-4a23-497d-963c-a86bf1613032 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.381008 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18ca8d-4a23-497d-963c-a86bf1613032 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.381291 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18ca8d-4a23-497d-963c-a86bf1613032 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.385281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:15.388212 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18ca8d-4a23-497d-963c-a86bf1613032 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:15.389719 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18ca8d-4a23-497d-963c-a86bf1613032 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:15.393820 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fea927-ccc7-443e-8e7f-4ed4130eb398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:15.394500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fea927-ccc7-443e-8e7f-4ed4130eb398 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.394807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fea927-ccc7-443e-8e7f-4ed4130eb398 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.395068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fea927-ccc7-443e-8e7f-4ed4130eb398 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.395915 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fea927-ccc7-443e-8e7f-4ed4130eb398 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:15.396224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fea927-ccc7-443e-8e7f-4ed4130eb398 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:15.397806 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fea927-ccc7-443e-8e7f-4ed4130eb398 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-cb18ca8d-4a23-497d-963c-a86bf1613032 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:15.406130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:15.408989 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:15.410029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:15.410029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:15.410029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:15.410029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:15.410029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:15.410029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:15.410029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:15.410029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:15.410029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 297/590] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:15.410029 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18ca8d-4a23-497d-963c-a86bf1613032 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:15.416429 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 294/591] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:50:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:15.417288 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fea927-ccc7-443e-8e7f-4ed4130eb398 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:15.521034 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fea927-ccc7-443e-8e7f-4ed4130eb398 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:15.523889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18ca8d-4a23-497d-963c-a86bf1613032 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:15.557381 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e2a56-4a23-4b39-bc95-fda52ada5360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:15.558185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e2a56-4a23-4b39-bc95-fda52ada5360 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.558733 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e2a56-4a23-4b39-bc95-fda52ada5360 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.558976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320716e-bc6e-425d-8185-6c7f11356417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:15.559095 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e2a56-4a23-4b39-bc95-fda52ada5360 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.559643 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320716e-bc6e-425d-8185-6c7f11356417 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.559929 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320716e-bc6e-425d-8185-6c7f11356417 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.560205 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320716e-bc6e-425d-8185-6c7f11356417 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.561090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320716e-bc6e-425d-8185-6c7f11356417 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:15.561400 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320716e-bc6e-425d-8185-6c7f11356417 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:15.562902 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320716e-bc6e-425d-8185-6c7f11356417 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:15.567131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e2a56-4a23-4b39-bc95-fda52ada5360 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:15.568674 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e2a56-4a23-4b39-bc95-fda52ada5360 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:15.577013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 298/592] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:15.578406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1bebd-edb9-4e4e-87a1-f15d30ab953c None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.578702 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1bebd-edb9-4e4e-87a1-f15d30ab953c None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.578985 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a1bebd-edb9-4e4e-87a1-f15d30ab953c None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.581203 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 299/593] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:15.582947 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1bebd-edb9-4e4e-87a1-f15d30ab953c None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-3d6e2a56-4a23-4b39-bc95-fda52ada5360 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:15.585410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:15.586474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:15.587659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:15.587659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:15.587659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:15.587659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:15.587659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:15.587659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:15.587659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:15.587659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:15.587659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:50:15.587659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:15.591160 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 295/594] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:15.592011 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e2a56-4a23-4b39-bc95-fda52ada5360 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:15.619146 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:15.640795 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:15.688172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a1bebd-edb9-4e4e-87a1-f15d30ab953c None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:15.694555 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e2a56-4a23-4b39-bc95-fda52ada5360 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:15.722233 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6907a-7553-4e5c-8c6e-a2c6fc5818a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:15.722900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6907a-7553-4e5c-8c6e-a2c6fc5818a5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.723197 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6907a-7553-4e5c-8c6e-a2c6fc5818a5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.723463 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6907a-7553-4e5c-8c6e-a2c6fc5818a5 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.724244 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6907a-7553-4e5c-8c6e-a2c6fc5818a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:15.724587 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6907a-7553-4e5c-8c6e-a2c6fc5818a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:15.726039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6907a-7553-4e5c-8c6e-a2c6fc5818a5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:15.730699 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f01d76-f949-4e66-bca8-fed187641ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:15.730699 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f01d76-f949-4e66-bca8-fed187641ff8 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.730699 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f01d76-f949-4e66-bca8-fed187641ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.730699 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f01d76-f949-4e66-bca8-fed187641ff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.739695 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 300/595] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:15.740542 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6907a-7553-4e5c-8c6e-a2c6fc5818a5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:15.741552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f01d76-f949-4e66-bca8-fed187641ff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:15.743031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f01d76-f949-4e66-bca8-fed187641ff8 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-b3f01d76-f949-4e66-bca8-fed187641ff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:15.761983 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:15.763241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:15.764272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:15.764272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:15.764272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:15.764272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:15.768527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 296/596] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:15.769946 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4497-439e-4d2c-a2b7-b41bbb921ee7 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.770241 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4497-439e-4d2c-a2b7-b41bbb921ee7 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.770523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc4497-439e-4d2c-a2b7-b41bbb921ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.787967 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-f7dc4497-439e-4d2c-a2b7-b41bbb921ee7 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:15.789240 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-f7dc4497-439e-4d2c-a2b7-b41bbb921ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:15.846916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:15.853884 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6907a-7553-4e5c-8c6e-a2c6fc5818a5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:15.873961 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 297/597] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:15.873961 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc4497-439e-4d2c-a2b7-b41bbb921ee7 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:15.904267 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df8bb3-1c06-4a45-8857-f5fbe0e6e6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:15.905494 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df8bb3-1c06-4a45-8857-f5fbe0e6e6db None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:15.905947 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df8bb3-1c06-4a45-8857-f5fbe0e6e6db None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:15.908610 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df8bb3-1c06-4a45-8857-f5fbe0e6e6db None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/77427e1028d349b4a740efe0a985e73b/roles/41728d78a94d4b7abb951f8a3cee2ce2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:15.909462 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:15.921175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df8bb3-1c06-4a45-8857-f5fbe0e6e6db None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=77427e1028d349b4a740efe0a985e73b, role_id=41728d78a94d4b7abb951f8a3cee2ce2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:15.925813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df8bb3-1c06-4a45-8857-f5fbe0e6e6db None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:15.988616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc4497-439e-4d2c-a2b7-b41bbb921ee7 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:16.013706 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 301/598] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:50:15 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/77427e1028d349b4a740efe0a985e73b/roles/41728d78a94d4b7abb951f8a3cee2ce2 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:50:16.014547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df8bb3-1c06-4a45-8857-f5fbe0e6e6db None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:16.024790 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c11c92-130c-4a8e-b1d0-57f4ee05d3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:16.025490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11c92-130c-4a8e-b1d0-57f4ee05d3d7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:16.025776 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11c92-130c-4a8e-b1d0-57f4ee05d3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:16.026050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c11c92-130c-4a8e-b1d0-57f4ee05d3d7 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:16.032683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c11c92-130c-4a8e-b1d0-57f4ee05d3d7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:16.032683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c11c92-130c-4a8e-b1d0-57f4ee05d3d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:16.032683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c11c92-130c-4a8e-b1d0-57f4ee05d3d7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:16.050888 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 298/599] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 20:50:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:16.051581 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c11c92-130c-4a8e-b1d0-57f4ee05d3d7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:16.156525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:16.158387 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df8bb3-1c06-4a45-8857-f5fbe0e6e6db None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:16.175660 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c11c92-130c-4a8e-b1d0-57f4ee05d3d7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:16.199593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42d580-9d61-4093-b04b-d3b6fee62a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:16.201134 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42d580-9d61-4093-b04b-d3b6fee62a0e None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:16.201478 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42d580-9d61-4093-b04b-d3b6fee62a0e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:16.201705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b42d580-9d61-4093-b04b-d3b6fee62a0e None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/6ab3de874a0f475c8745831e76a6e4d0/roles/41728d78a94d4b7abb951f8a3cee2ce2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:16.218521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42d580-9d61-4093-b04b-d3b6fee62a0e None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=6ab3de874a0f475c8745831e76a6e4d0, role_id=41728d78a94d4b7abb951f8a3cee2ce2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:16.220606 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b42d580-9d61-4093-b04b-d3b6fee62a0e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:16.223052 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:16.226787 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc62510-b16d-45bd-aca4-78ee350a15cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:16.227500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc62510-b16d-45bd-aca4-78ee350a15cb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:16.227794 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc62510-b16d-45bd-aca4-78ee350a15cb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:16.228072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc62510-b16d-45bd-aca4-78ee350a15cb None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:16.229380 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc62510-b16d-45bd-aca4-78ee350a15cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:16.229380 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc62510-b16d-45bd-aca4-78ee350a15cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:16.230542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc62510-b16d-45bd-aca4-78ee350a15cb None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:16.243141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 299/600] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:16.243947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc62510-b16d-45bd-aca4-78ee350a15cb None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:16.332628 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 302/601] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:50:16 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/6ab3de874a0f475c8745831e76a6e4d0/roles/41728d78a94d4b7abb951f8a3cee2ce2 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:50:16.337395 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42d580-9d61-4093-b04b-d3b6fee62a0e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:16.364111 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc62510-b16d-45bd-aca4-78ee350a15cb None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:16.418992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c21fa-9719-430b-8876-d007ef2f161f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:16.420977 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c21fa-9719-430b-8876-d007ef2f161f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:16.421466 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c21fa-9719-430b-8876-d007ef2f161f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:16.421960 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c21fa-9719-430b-8876-d007ef2f161f None admin] PATH_INFO: `/v3/roles/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:16.440207 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c21fa-9719-430b-8876-d007ef2f161f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-835c21fa-9719-430b-8876-d007ef2f161f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:16.451577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:16.452785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:16.453744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:16.457478 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 300/602] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:16.458661 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c21fa-9719-430b-8876-d007ef2f161f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:16.472566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:16.480126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b42d580-9d61-4093-b04b-d3b6fee62a0e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:16.525077 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193471fd-dffd-4aca-842e-8407f271c53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:16.526169 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193471fd-dffd-4aca-842e-8407f271c53d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:16.526443 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193471fd-dffd-4aca-842e-8407f271c53d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:16.526727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193471fd-dffd-4aca-842e-8407f271c53d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:16.539823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193471fd-dffd-4aca-842e-8407f271c53d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:16.541364 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193471fd-dffd-4aca-842e-8407f271c53d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-193471fd-dffd-4aca-842e-8407f271c53d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:16.565475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:16.566566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:16.568222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:16.568222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:16.568222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:16.568222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:16.568222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 303/603] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:16.568222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193471fd-dffd-4aca-842e-8407f271c53d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:16.599773 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c21fa-9719-430b-8876-d007ef2f161f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:16.635406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:16.637272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd32b6-73ff-4170-aaee-bd402b6a3bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:16.638182 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd32b6-73ff-4170-aaee-bd402b6a3bf7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:16.638535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd32b6-73ff-4170-aaee-bd402b6a3bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:16.638885 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd32b6-73ff-4170-aaee-bd402b6a3bf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:16.647174 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd32b6-73ff-4170-aaee-bd402b6a3bf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:16.648881 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd32b6-73ff-4170-aaee-bd402b6a3bf7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-07dd32b6-73ff-4170-aaee-bd402b6a3bf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:16.664267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:16.665369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:16.666439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:16.666439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:16.666439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:16.666439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:16.666439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:16.666439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:16.666439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:16.666439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:16.666439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 301/604] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:16.667025 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd32b6-73ff-4170-aaee-bd402b6a3bf7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:16.705538 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193471fd-dffd-4aca-842e-8407f271c53d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:16.740822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:16.743321 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bdde1-cf26-40db-821d-b8b2af6b55ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:16.744053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bdde1-cf26-40db-821d-b8b2af6b55ef None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:16.744362 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bdde1-cf26-40db-821d-b8b2af6b55ef None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:16.744686 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bdde1-cf26-40db-821d-b8b2af6b55ef None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:16.745457 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bdde1-cf26-40db-821d-b8b2af6b55ef None admin] RBAC: Adding query filter params (name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:16.745827 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bdde1-cf26-40db-821d-b8b2af6b55ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:16.747265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bdde1-cf26-40db-821d-b8b2af6b55ef None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:16.762328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 304/605] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:16.763228 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bdde1-cf26-40db-821d-b8b2af6b55ef None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:16.776933 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd32b6-73ff-4170-aaee-bd402b6a3bf7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:16.816532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f028aeb-765b-4b66-8be4-d27902858804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:16.817352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f028aeb-765b-4b66-8be4-d27902858804 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:16.817713 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f028aeb-765b-4b66-8be4-d27902858804 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:16.818092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f028aeb-765b-4b66-8be4-d27902858804 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:16.818936 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f028aeb-765b-4b66-8be4-d27902858804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:16.819328 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f028aeb-765b-4b66-8be4-d27902858804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:16.820946 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f028aeb-765b-4b66-8be4-d27902858804 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:16.838424 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 302/606] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:16.839486 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f028aeb-765b-4b66-8be4-d27902858804 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:16.885262 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bdde1-cf26-40db-821d-b8b2af6b55ef None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:16.896160 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:16.923937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023aabb-b979-43d6-a318-cf64a98a552c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:16.924678 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023aabb-b979-43d6-a318-cf64a98a552c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:16.927653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023aabb-b979-43d6-a318-cf64a98a552c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:16.927653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023aabb-b979-43d6-a318-cf64a98a552c None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:16.927653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023aabb-b979-43d6-a318-cf64a98a552c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:16.927653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023aabb-b979-43d6-a318-cf64a98a552c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:16.929067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023aabb-b979-43d6-a318-cf64a98a552c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:16.948531 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 305/607] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:16.952580 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023aabb-b979-43d6-a318-cf64a98a552c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:16.968154 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f028aeb-765b-4b66-8be4-d27902858804 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:17.003879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:17.016269 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2eecb-d266-4aab-ae60-f0cfb69a3dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:17.018004 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2eecb-d266-4aab-ae60-f0cfb69a3dad None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:17.018290 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2eecb-d266-4aab-ae60-f0cfb69a3dad None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:17.018565 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a2eecb-d266-4aab-ae60-f0cfb69a3dad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:17.034807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2eecb-d266-4aab-ae60-f0cfb69a3dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:17.036736 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a2eecb-d266-4aab-ae60-f0cfb69a3dad None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-58a2eecb-d266-4aab-ae60-f0cfb69a3dad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:17.062840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:17.064589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:17.065591 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:17.065591 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:17.065591 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:17.065591 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:17.072092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 303/608] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:17.073302 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2eecb-d266-4aab-ae60-f0cfb69a3dad None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:17.085279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023aabb-b979-43d6-a318-cf64a98a552c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:17.125225 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bf184-c862-478f-89f1-12224e8cef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:17.126674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bf184-c862-478f-89f1-12224e8cef34 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:17.126959 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bf184-c862-478f-89f1-12224e8cef34 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:17.127239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bf184-c862-478f-89f1-12224e8cef34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:17.135819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bf184-c862-478f-89f1-12224e8cef34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:17.137438 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bf184-c862-478f-89f1-12224e8cef34 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-6f0bf184-c862-478f-89f1-12224e8cef34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:17.154530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:17.155805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:17.157038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:17.157038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:17.157038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:17.157038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:17.157038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:17.157038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:17.157038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:17.157038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:17.160535 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 306/609] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:17.161569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bf184-c862-478f-89f1-12224e8cef34 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:17.190442 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a2eecb-d266-4aab-ae60-f0cfb69a3dad None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:17.196019 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:17.229689 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef96d7-263e-4291-9c90-44e195b6b602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:17.230905 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef96d7-263e-4291-9c90-44e195b6b602 None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:17.231384 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef96d7-263e-4291-9c90-44e195b6b602 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:17.231826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef96d7-263e-4291-9c90-44e195b6b602 None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/e2f9d95b4a2d4816adceffdd1a49186f/roles/41728d78a94d4b7abb951f8a3cee2ce2` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:17.248472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef96d7-263e-4291-9c90-44e195b6b602 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=e2f9d95b4a2d4816adceffdd1a49186f, role_id=41728d78a94d4b7abb951f8a3cee2ce2)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:17.250623 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef96d7-263e-4291-9c90-44e195b6b602 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:17.283071 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bf184-c862-478f-89f1-12224e8cef34 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:17.321654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce763a-231c-4430-bbd7-710eb1a2c00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:17.322331 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce763a-231c-4430-bbd7-710eb1a2c00d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:17.322612 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce763a-231c-4430-bbd7-710eb1a2c00d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:17.322944 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efce763a-231c-4430-bbd7-710eb1a2c00d None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:17.323865 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce763a-231c-4430-bbd7-710eb1a2c00d None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:17.324225 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce763a-231c-4430-bbd7-710eb1a2c00d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:17.325655 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efce763a-231c-4430-bbd7-710eb1a2c00d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:17.339530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 307/610] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:17.340821 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce763a-231c-4430-bbd7-710eb1a2c00d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:17.404519 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 304/611] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:50:17 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/e2f9d95b4a2d4816adceffdd1a49186f/roles/41728d78a94d4b7abb951f8a3cee2ce2 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:50:17.405941 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef96d7-263e-4291-9c90-44e195b6b602 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:17.460054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:17.478549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efce763a-231c-4430-bbd7-710eb1a2c00d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:17.486378 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:17.533769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917cfc8-fdc6-4542-9af8-444243f6ff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:17.534524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917cfc8-fdc6-4542-9af8-444243f6ff7e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:17.534809 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917cfc8-fdc6-4542-9af8-444243f6ff7e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:17.535083 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917cfc8-fdc6-4542-9af8-444243f6ff7e None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:17.537374 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917cfc8-fdc6-4542-9af8-444243f6ff7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:17.539463 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917cfc8-fdc6-4542-9af8-444243f6ff7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:17.542138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3917cfc8-fdc6-4542-9af8-444243f6ff7e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:17.579934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 308/612] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:17.587017 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917cfc8-fdc6-4542-9af8-444243f6ff7e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:17.656161 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef96d7-263e-4291-9c90-44e195b6b602 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:17.712713 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e65f8-6980-461d-abc4-1adcb450e536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:17.713407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e65f8-6980-461d-abc4-1adcb450e536 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:17.713691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e65f8-6980-461d-abc4-1adcb450e536 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:17.713970 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e65f8-6980-461d-abc4-1adcb450e536 None admin] PATH_INFO: `/v3/users/swift` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:17.722318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e65f8-6980-461d-abc4-1adcb450e536 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:17.723990 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e65f8-6980-461d-abc4-1adcb450e536 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-a75e65f8-6980-461d-abc4-1adcb450e536 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:17.740329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:17.741553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:17.742613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:17.742613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:17.742613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:17.742613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:17.742613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:17.742613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:17.742613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:17.742613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:17.742613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:17.742613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:17.742613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 305/613] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:17.747480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e65f8-6980-461d-abc4-1adcb450e536 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:17.764926 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917cfc8-fdc6-4542-9af8-444243f6ff7e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:17.805203 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376f28c-f48a-48de-bd20-5f7df3919156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:17.806512 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376f28c-f48a-48de-bd20-5f7df3919156 None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:17.807019 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376f28c-f48a-48de-bd20-5f7df3919156 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:17.807517 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b376f28c-f48a-48de-bd20-5f7df3919156 None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/4b3f694758fa483ca497fbea18061cb6/roles/41728d78a94d4b7abb951f8a3cee2ce2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:17.825807 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b376f28c-f48a-48de-bd20-5f7df3919156 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=4b3f694758fa483ca497fbea18061cb6, role_id=41728d78a94d4b7abb951f8a3cee2ce2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:17.827801 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b376f28c-f48a-48de-bd20-5f7df3919156 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:17.888364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:17.893696 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e65f8-6980-461d-abc4-1adcb450e536 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:17.904755 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:17.916095 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 309/614] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:50:17 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/4b3f694758fa483ca497fbea18061cb6/roles/41728d78a94d4b7abb951f8a3cee2ce2 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:50:17.938167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed367453-b34f-483b-ae2e-30ee6f16a272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:17.939672 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed367453-b34f-483b-ae2e-30ee6f16a272 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:17.940004 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed367453-b34f-483b-ae2e-30ee6f16a272 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:17.940421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed367453-b34f-483b-ae2e-30ee6f16a272 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:17.941617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed367453-b34f-483b-ae2e-30ee6f16a272 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:17.951800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed367453-b34f-483b-ae2e-30ee6f16a272 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:17.951800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed367453-b34f-483b-ae2e-30ee6f16a272 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:17.976316 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 306/615] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:50:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:17.982294 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376f28c-f48a-48de-bd20-5f7df3919156 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:18.143776 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b376f28c-f48a-48de-bd20-5f7df3919156 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:18.196668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf019d4d-5ffb-48e8-a05b-d71bcd0aca23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:18.197497 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf019d4d-5ffb-48e8-a05b-d71bcd0aca23 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:18.197783 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf019d4d-5ffb-48e8-a05b-d71bcd0aca23 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:18.198062 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf019d4d-5ffb-48e8-a05b-d71bcd0aca23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:18.214285 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf019d4d-5ffb-48e8-a05b-d71bcd0aca23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:18.215791 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf019d4d-5ffb-48e8-a05b-d71bcd0aca23 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-bf019d4d-5ffb-48e8-a05b-d71bcd0aca23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:18.234819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:18.236483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:18.237532 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:18.237532 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:18.237532 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:18.237532 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:18.237532 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 310/616] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:18.248944 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed367453-b34f-483b-ae2e-30ee6f16a272 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:18.416374 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed367453-b34f-483b-ae2e-30ee6f16a272 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:18.466063 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd4c4b1-e5b7-4f45-9d16-4af433977219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:18.466763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4c4b1-e5b7-4f45-9d16-4af433977219 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:18.467055 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4c4b1-e5b7-4f45-9d16-4af433977219 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:18.467339 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd4c4b1-e5b7-4f45-9d16-4af433977219 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:18.468166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4c4b1-e5b7-4f45-9d16-4af433977219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:18.468589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4c4b1-e5b7-4f45-9d16-4af433977219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:18.470578 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd4c4b1-e5b7-4f45-9d16-4af433977219 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:18.486579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 307/617] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:18.492419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf019d4d-5ffb-48e8-a05b-d71bcd0aca23 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:18.593521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:18.596051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b2a7b-69cc-430f-97c6-388e94d013dd None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:18.596599 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b2a7b-69cc-430f-97c6-388e94d013dd None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:18.597955 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b2a7b-69cc-430f-97c6-388e94d013dd None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:18.600150 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 308/618] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:50:18.608568 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262934eb-0320-4393-b313-00790cd7b91c None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:18.609376 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262934eb-0320-4393-b313-00790cd7b91c None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:18.611684 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262934eb-0320-4393-b313-00790cd7b91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:18.629482 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:18.637692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-262934eb-0320-4393-b313-00790cd7b91c None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:18.640261 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-262934eb-0320-4393-b313-00790cd7b91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:18.708535 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf019d4d-5ffb-48e8-a05b-d71bcd0aca23 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:18.720381 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 309/619] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:50:18.745196 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262934eb-0320-4393-b313-00790cd7b91c None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:18.809935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1690f35-ad6e-45da-96dd-67cd7893c794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:18.810761 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1690f35-ad6e-45da-96dd-67cd7893c794 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:18.811054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1690f35-ad6e-45da-96dd-67cd7893c794 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:18.811343 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1690f35-ad6e-45da-96dd-67cd7893c794 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:18.829495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1690f35-ad6e-45da-96dd-67cd7893c794 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:18.831246 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1690f35-ad6e-45da-96dd-67cd7893c794 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-f1690f35-ad6e-45da-96dd-67cd7893c794 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:18.856721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:18.857870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:18.858882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:18.858882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:18.858882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:18.858882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:18.858882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:18.858882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:18.858882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:18.858882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:18.858882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 311/620] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:18.867075 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1690f35-ad6e-45da-96dd-67cd7893c794 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:18.884528 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262934eb-0320-4393-b313-00790cd7b91c None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:18.930656 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3b0fb-3e75-4a3a-828c-951949127df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:18.932624 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3b0fb-3e75-4a3a-828c-951949127df7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:18.932921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3b0fb-3e75-4a3a-828c-951949127df7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:18.933203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3b0fb-3e75-4a3a-828c-951949127df7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:18.940677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:18.961548 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3b0fb-3e75-4a3a-828c-951949127df7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-abe3b0fb-3e75-4a3a-828c-951949127df7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:18.979052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:18.980861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:18.983092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:18.984272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 310/621] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:18.997433 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3b0fb-3e75-4a3a-828c-951949127df7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:19.067753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1690f35-ad6e-45da-96dd-67cd7893c794 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:19.081722 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:19.117195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a5cfc-7e14-4898-b61a-b0608a784a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:19.118725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a5cfc-7e14-4898-b61a-b0608a784a75 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:19.119259 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a5cfc-7e14-4898-b61a-b0608a784a75 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:19.119760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a5cfc-7e14-4898-b61a-b0608a784a75 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:19.121405 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a5cfc-7e14-4898-b61a-b0608a784a75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:19.122027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a5cfc-7e14-4898-b61a-b0608a784a75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:19.125010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a5cfc-7e14-4898-b61a-b0608a784a75 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:19.141999 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 312/622] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:19.143755 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8541ad2-eea7-4ec1-95a8-bde1fdcd4a6d None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:19.143950 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8541ad2-eea7-4ec1-95a8-bde1fdcd4a6d None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:19.144371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8541ad2-eea7-4ec1-95a8-bde1fdcd4a6d None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:19.150173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 313/623] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:19.151166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8541ad2-eea7-4ec1-95a8-bde1fdcd4a6d None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:19.194182 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3b0fb-3e75-4a3a-828c-951949127df7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:19.244641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a3ffb7-7d15-461a-8b3a-8e088343d6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:19.246177 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3ffb7-7d15-461a-8b3a-8e088343d6ab None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:19.247219 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3ffb7-7d15-461a-8b3a-8e088343d6ab None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:19.247882 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a3ffb7-7d15-461a-8b3a-8e088343d6ab None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:19.248948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3ffb7-7d15-461a-8b3a-8e088343d6ab None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:19.249680 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3ffb7-7d15-461a-8b3a-8e088343d6ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:19.252693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a3ffb7-7d15-461a-8b3a-8e088343d6ab None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:19.271238 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 311/624] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:19.271238 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6683b4-937e-4897-a6b2-63bb04213248 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:19.271238 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6683b4-937e-4897-a6b2-63bb04213248 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:19.271238 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6683b4-937e-4897-a6b2-63bb04213248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:19.273480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8541ad2-eea7-4ec1-95a8-bde1fdcd4a6d None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:19.293547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-5e6683b4-937e-4897-a6b2-63bb04213248 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:19.295468 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-5e6683b4-937e-4897-a6b2-63bb04213248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:19.314865 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf4142-1e73-436f-bb7d-59448e32165a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:19.315941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf4142-1e73-436f-bb7d-59448e32165a None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:19.316291 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf4142-1e73-436f-bb7d-59448e32165a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:19.316732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cf4142-1e73-436f-bb7d-59448e32165a None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/ba6e1d31d8004f9fa956b78d8716d6e6/roles/41728d78a94d4b7abb951f8a3cee2ce2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:19.335359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf4142-1e73-436f-bb7d-59448e32165a None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=ba6e1d31d8004f9fa956b78d8716d6e6, role_id=41728d78a94d4b7abb951f8a3cee2ce2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:19.337261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cf4142-1e73-436f-bb7d-59448e32165a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:19.367721 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:19.384009 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 312/625] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:19.384713 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6683b4-937e-4897-a6b2-63bb04213248 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:19.415796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 314/626] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:50:19 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/ba6e1d31d8004f9fa956b78d8716d6e6/roles/41728d78a94d4b7abb951f8a3cee2ce2 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:50:19.424979 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf4142-1e73-436f-bb7d-59448e32165a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:19.489400 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:19.535479 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6683b4-937e-4897-a6b2-63bb04213248 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:19.587068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30faecd-d33d-4e55-be03-eef2445d72e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:19.588330 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30faecd-d33d-4e55-be03-eef2445d72e6 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:19.588864 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30faecd-d33d-4e55-be03-eef2445d72e6 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:19.589380 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30faecd-d33d-4e55-be03-eef2445d72e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:19.603951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30faecd-d33d-4e55-be03-eef2445d72e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:19.605892 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30faecd-d33d-4e55-be03-eef2445d72e6 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:19.715425 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cf4142-1e73-436f-bb7d-59448e32165a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:19.764112 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc746b3-c64e-4b04-8fe5-a2a73d1c3a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:19.764112 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc746b3-c64e-4b04-8fe5-a2a73d1c3a84 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:19.764112 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc746b3-c64e-4b04-8fe5-a2a73d1c3a84 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:19.764112 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc746b3-c64e-4b04-8fe5-a2a73d1c3a84 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-a30faecd-d33d-4e55-be03-eef2445d72e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:19.764614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:19.765546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:19.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:19.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:19.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:19.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:19.766746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 313/627] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:19.775671 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30faecd-d33d-4e55-be03-eef2445d72e6 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:19.793441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc746b3-c64e-4b04-8fe5-a2a73d1c3a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-2bc746b3-c64e-4b04-8fe5-a2a73d1c3a84 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:19.805065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:19.806200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:19.807154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:19.816386 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 315/628] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:19.822364 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc746b3-c64e-4b04-8fe5-a2a73d1c3a84 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:19.881737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:19.960660 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30faecd-d33d-4e55-be03-eef2445d72e6 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:20.013116 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4be213-65b8-4be0-8ac5-91ba00110318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:20.014417 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4be213-65b8-4be0-8ac5-91ba00110318 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:20.015760 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4be213-65b8-4be0-8ac5-91ba00110318 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:20.016883 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4be213-65b8-4be0-8ac5-91ba00110318 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:20.018021 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4be213-65b8-4be0-8ac5-91ba00110318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:20.020906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4be213-65b8-4be0-8ac5-91ba00110318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:20.023563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4be213-65b8-4be0-8ac5-91ba00110318 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:20.053021 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 314/629] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:20.060536 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4be213-65b8-4be0-8ac5-91ba00110318 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:20.062373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:20.115690 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc746b3-c64e-4b04-8fe5-a2a73d1c3a84 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:20.185264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:20.188931 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4be213-65b8-4be0-8ac5-91ba00110318 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:20.189929 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db1059-9319-4bab-943a-a971a31e0a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:20.190801 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db1059-9319-4bab-943a-a971a31e0a2b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:20.191183 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db1059-9319-4bab-943a-a971a31e0a2b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:20.192443 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db1059-9319-4bab-943a-a971a31e0a2b None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:20.193222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db1059-9319-4bab-943a-a971a31e0a2b None admin] RBAC: Adding query filter params (name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:20.193536 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db1059-9319-4bab-943a-a971a31e0a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:20.195165 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db1059-9319-4bab-943a-a971a31e0a2b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:20.212482 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 316/630] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:20.214449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d8aca-84ba-406f-b119-d06831e906f7 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:20.214737 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d8aca-84ba-406f-b119-d06831e906f7 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:20.218629 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d8aca-84ba-406f-b119-d06831e906f7 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:20.218629 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 317/631] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:20.218629 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d8aca-84ba-406f-b119-d06831e906f7 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:20.242264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a1de2-1069-484d-87a1-2376c7ec3afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:20.243149 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a1de2-1069-484d-87a1-2376c7ec3afb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:20.243433 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a1de2-1069-484d-87a1-2376c7ec3afb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:20.243714 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a1de2-1069-484d-87a1-2376c7ec3afb None admin] PATH_INFO: `/v3/users/nova` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:20.255917 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a1de2-1069-484d-87a1-2376c7ec3afb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:20.257805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a1de2-1069-484d-87a1-2376c7ec3afb None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-9f3a1de2-1069-484d-87a1-2376c7ec3afb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:20.278541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:20.279686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:20.280812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:20.280812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:20.280812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:20.280812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:20.280812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:20.280812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:20.280812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:20.280812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:20.280812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:20.280812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:20.280812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 315/632] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:50:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:20.284738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed87b8-dbc0-46a1-a0b4-d5ebb9c294ee None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:20.284738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed87b8-dbc0-46a1-a0b4-d5ebb9c294ee None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:20.284738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed87b8-dbc0-46a1-a0b4-d5ebb9c294ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:20.307187 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-f9ed87b8-dbc0-46a1-a0b4-d5ebb9c294ee None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:20.308669 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-f9ed87b8-dbc0-46a1-a0b4-d5ebb9c294ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:20.346723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d8aca-84ba-406f-b119-d06831e906f7 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:20.387154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe32d17-967d-4614-bb82-3dd6bc0a447d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:20.388193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe32d17-967d-4614-bb82-3dd6bc0a447d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:20.388612 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe32d17-967d-4614-bb82-3dd6bc0a447d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:20.389043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe32d17-967d-4614-bb82-3dd6bc0a447d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:20.405790 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe32d17-967d-4614-bb82-3dd6bc0a447d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:20.408258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe32d17-967d-4614-bb82-3dd6bc0a447d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:20.412496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 316/633] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:20.413494 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed87b8-dbc0-46a1-a0b4-d5ebb9c294ee None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:20.416922 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-abe32d17-967d-4614-bb82-3dd6bc0a447d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:20.460992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:20.462090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:20.463084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:20.463084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:20.463084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:20.463084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:20.467893 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 318/634] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:20.469239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe32d17-967d-4614-bb82-3dd6bc0a447d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:20.569751 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:20.581472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ed87b8-dbc0-46a1-a0b4-d5ebb9c294ee None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:20.589939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe32d17-967d-4614-bb82-3dd6bc0a447d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:20.621007 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000dbc7-9ec6-4fb6-8f55-c50c36ea216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:20.621798 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000dbc7-9ec6-4fb6-8f55-c50c36ea216c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:20.622096 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000dbc7-9ec6-4fb6-8f55-c50c36ea216c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:20.622383 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000dbc7-9ec6-4fb6-8f55-c50c36ea216c None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:20.623536 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000dbc7-9ec6-4fb6-8f55-c50c36ea216c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:20.623749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000dbc7-9ec6-4fb6-8f55-c50c36ea216c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:20.625211 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000dbc7-9ec6-4fb6-8f55-c50c36ea216c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:20.629498 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f29eac-12ea-4da5-8a79-c6d1d5f9d79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:20.631159 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f29eac-12ea-4da5-8a79-c6d1d5f9d79d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:20.631732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f29eac-12ea-4da5-8a79-c6d1d5f9d79d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:20.632265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f29eac-12ea-4da5-8a79-c6d1d5f9d79d None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:20.644587 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f29eac-12ea-4da5-8a79-c6d1d5f9d79d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:20.647138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 317/635] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:50:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:20.648877 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000dbc7-9ec6-4fb6-8f55-c50c36ea216c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-70f29eac-12ea-4da5-8a79-c6d1d5f9d79d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:20.653894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:20.655054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:20.655991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:20.661892 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 319/636] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:20.666904 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f29eac-12ea-4da5-8a79-c6d1d5f9d79d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:20.765019 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:20.774988 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000dbc7-9ec6-4fb6-8f55-c50c36ea216c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:20.814307 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d6626-7111-47a5-9190-2ca3c0108607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:20.814985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6626-7111-47a5-9190-2ca3c0108607 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:20.815337 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6626-7111-47a5-9190-2ca3c0108607 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:20.815530 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6626-7111-47a5-9190-2ca3c0108607 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:20.816431 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d6626-7111-47a5-9190-2ca3c0108607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:20.816635 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d6626-7111-47a5-9190-2ca3c0108607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:20.817955 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d6626-7111-47a5-9190-2ca3c0108607 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:20.835835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 318/637] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:20.835835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d6626-7111-47a5-9190-2ca3c0108607 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:20.841012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f29eac-12ea-4da5-8a79-c6d1d5f9d79d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:20.850051 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:20.883080 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a5633-dbde-4352-a46c-929420728408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:20.884208 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a5633-dbde-4352-a46c-929420728408 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:20.884739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a5633-dbde-4352-a46c-929420728408 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:20.885230 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a5633-dbde-4352-a46c-929420728408 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:20.967481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a5633-dbde-4352-a46c-929420728408 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:20.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d6626-7111-47a5-9190-2ca3c0108607 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:20.969026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a5633-dbde-4352-a46c-929420728408 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:21.007936 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c811011-a294-48b5-a68b-18d969f4a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:21.008837 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c811011-a294-48b5-a68b-18d969f4a75f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:21.009123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c811011-a294-48b5-a68b-18d969f4a75f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:21.009403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c811011-a294-48b5-a68b-18d969f4a75f None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:21.010160 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c811011-a294-48b5-a68b-18d969f4a75f None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:21.010467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c811011-a294-48b5-a68b-18d969f4a75f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:21.011837 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c811011-a294-48b5-a68b-18d969f4a75f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:21.021411 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 319/638] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:21.027269 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf75ec3-55c7-46e0-86d9-12323211c5ad None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:21.027756 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf75ec3-55c7-46e0-86d9-12323211c5ad None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:21.028186 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf75ec3-55c7-46e0-86d9-12323211c5ad None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:21.033177 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 320/639] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:21 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:21.034550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf75ec3-55c7-46e0-86d9-12323211c5ad None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:21.037805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:21.095997 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 320/640] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:21.097273 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a5633-dbde-4352-a46c-929420728408 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:21.135645 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:21.176366 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf75ec3-55c7-46e0-86d9-12323211c5ad None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:21.216891 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb5c31-aa6e-4771-b026-24736ade13c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:21.217713 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb5c31-aa6e-4771-b026-24736ade13c0 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:21.217995 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb5c31-aa6e-4771-b026-24736ade13c0 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:21.218273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb5c31-aa6e-4771-b026-24736ade13c0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:21.228023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb5c31-aa6e-4771-b026-24736ade13c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:21.229548 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb5c31-aa6e-4771-b026-24736ade13c0 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:21.236249 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a5633-dbde-4352-a46c-929420728408 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-38fb5c31-aa6e-4771-b026-24736ade13c0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:21.245725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:21.246868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 321/641] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0263e94-cd2a-4393-be07-f10b90905a04 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:21.253281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0263e94-cd2a-4393-be07-f10b90905a04 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:21.254466 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0263e94-cd2a-4393-be07-f10b90905a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:21.268147 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-c0263e94-cd2a-4393-be07-f10b90905a04 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:21.269549 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-c0263e94-cd2a-4393-be07-f10b90905a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:21.289167 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f03e-bbba-476e-aa94-addc082e4cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:21.290047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f03e-bbba-476e-aa94-addc082e4cc3 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:21.290493 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f03e-bbba-476e-aa94-addc082e4cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:21.290628 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733f03e-bbba-476e-aa94-addc082e4cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:21.303351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f03e-bbba-476e-aa94-addc082e4cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:21.304906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733f03e-bbba-476e-aa94-addc082e4cc3 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-f733f03e-bbba-476e-aa94-addc082e4cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:21.324960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:21.326140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:21.327178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:21.327178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:21.327178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:21.327178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:21.327178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 321/642] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:21.327178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f03e-bbba-476e-aa94-addc082e4cc3 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:21.338236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:21.360426 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 322/643] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:21.361191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0263e94-cd2a-4393-be07-f10b90905a04 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:21.461311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:21.484101 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733f03e-bbba-476e-aa94-addc082e4cc3 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:21.527425 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0263e94-cd2a-4393-be07-f10b90905a04 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:21.539594 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f9e94-edb9-4615-ae32-eb31f445bcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:21.540920 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f9e94-edb9-4615-ae32-eb31f445bcb8 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:21.541405 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f9e94-edb9-4615-ae32-eb31f445bcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:21.541895 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f9e94-edb9-4615-ae32-eb31f445bcb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:21.555525 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f9e94-edb9-4615-ae32-eb31f445bcb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:21.557233 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f9e94-edb9-4615-ae32-eb31f445bcb8 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:21.573423 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b2e61-b739-4c0c-b94a-a847b21936d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:21.574113 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b2e61-b739-4c0c-b94a-a847b21936d4 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:21.574454 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b2e61-b739-4c0c-b94a-a847b21936d4 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:21.574733 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b2e61-b739-4c0c-b94a-a847b21936d4 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:21.575580 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b2e61-b739-4c0c-b94a-a847b21936d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:21.575890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b2e61-b739-4c0c-b94a-a847b21936d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:21.577751 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b2e61-b739-4c0c-b94a-a847b21936d4 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-b57f9e94-edb9-4615-ae32-eb31f445bcb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:21.580432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:21.581434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:21.582430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:21.582430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:21.582430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:21.582430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:21.583190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 322/644] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:21.583711 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f9e94-edb9-4615-ae32-eb31f445bcb8 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:21.603831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 323/645] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:50:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:21.603831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b2e61-b739-4c0c-b94a-a847b21936d4 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:21.661978 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:21.711366 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b2e61-b739-4c0c-b94a-a847b21936d4 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:21.711600 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f9e94-edb9-4615-ae32-eb31f445bcb8 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:21.732214 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:21.750118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf59ec-016f-4150-a56d-ab99e330a505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:21.750477 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d194d9-1b18-42fc-a476-695280278ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:21.751599 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf59ec-016f-4150-a56d-ab99e330a505 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:21.751781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d194d9-1b18-42fc-a476-695280278ecc None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:21.751884 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf59ec-016f-4150-a56d-ab99e330a505 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:21.752055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d194d9-1b18-42fc-a476-695280278ecc None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:21.752166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf59ec-016f-4150-a56d-ab99e330a505 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:21.752327 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d194d9-1b18-42fc-a476-695280278ecc None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:21.753066 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d194d9-1b18-42fc-a476-695280278ecc None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:21.753378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d194d9-1b18-42fc-a476-695280278ecc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:21.754789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d194d9-1b18-42fc-a476-695280278ecc None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:21.762988 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaf59ec-016f-4150-a56d-ab99e330a505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:21.768738 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 323/646] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:21.770228 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d194d9-1b18-42fc-a476-695280278ecc None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-6aaf59ec-016f-4150-a56d-ab99e330a505 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:21.773327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:21.774758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:21.775724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:21.776503 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 324/647] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:21.782267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf59ec-016f-4150-a56d-ab99e330a505 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:21.887613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d194d9-1b18-42fc-a476-695280278ecc None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:21.892040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaf59ec-016f-4150-a56d-ab99e330a505 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:21.906397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:21.923491 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14263da-7e2e-4da4-b04a-f8e75b62b117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:21.924169 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14263da-7e2e-4da4-b04a-f8e75b62b117 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:21.924575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14263da-7e2e-4da4-b04a-f8e75b62b117 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:21.924920 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14263da-7e2e-4da4-b04a-f8e75b62b117 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:21.925738 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14263da-7e2e-4da4-b04a-f8e75b62b117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:21.926115 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14263da-7e2e-4da4-b04a-f8e75b62b117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:21.927230 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647033e-83e4-4dab-b2fe-accd17a3eee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:21.927757 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14263da-7e2e-4da4-b04a-f8e75b62b117 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:21.927885 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647033e-83e4-4dab-b2fe-accd17a3eee6 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:21.928196 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647033e-83e4-4dab-b2fe-accd17a3eee6 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:21.928473 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647033e-83e4-4dab-b2fe-accd17a3eee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:21.941536 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 324/648] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:21.943273 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14263da-7e2e-4da4-b04a-f8e75b62b117 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:21.976079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:21.998732 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647033e-83e4-4dab-b2fe-accd17a3eee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:22.000195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647033e-83e4-4dab-b2fe-accd17a3eee6 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:22.047266 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14263da-7e2e-4da4-b04a-f8e75b62b117 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:22.082216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e5eb4-0a48-48b4-98a3-dc3f2be84de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:22.083065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5eb4-0a48-48b4-98a3-dc3f2be84de2 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:22.083344 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5eb4-0a48-48b4-98a3-dc3f2be84de2 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:22.083618 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5eb4-0a48-48b4-98a3-dc3f2be84de2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:22.090735 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e5eb4-0a48-48b4-98a3-dc3f2be84de2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:22.092192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e5eb4-0a48-48b4-98a3-dc3f2be84de2 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:22.101606 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 325/649] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:22.102455 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647033e-83e4-4dab-b2fe-accd17a3eee6 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-b18e5eb4-0a48-48b4-98a3-dc3f2be84de2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:22.107578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:22.108792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 325/650] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:22.110777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacde15-0500-49c9-b356-3047089dbe3c None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:22.111585 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacde15-0500-49c9-b356-3047089dbe3c None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:22.111585 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cacde15-0500-49c9-b356-3047089dbe3c None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:22.112691 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 326/651] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:22.113254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cacde15-0500-49c9-b356-3047089dbe3c None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:22.154738 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:22.187607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:22.208578 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647033e-83e4-4dab-b2fe-accd17a3eee6 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:22.215014 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cacde15-0500-49c9-b356-3047089dbe3c None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:22.245554 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4c74c-b457-4f40-bfc6-86f62e43e0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:22.246233 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4c74c-b457-4f40-bfc6-86f62e43e0aa None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:22.246513 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4c74c-b457-4f40-bfc6-86f62e43e0aa None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:22.246795 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4c74c-b457-4f40-bfc6-86f62e43e0aa None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:22.247544 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4c74c-b457-4f40-bfc6-86f62e43e0aa None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:22.247860 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4c74c-b457-4f40-bfc6-86f62e43e0aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:22.249258 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4c74c-b457-4f40-bfc6-86f62e43e0aa None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:22.250222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff4f22-f09a-4962-bef4-c147a95006a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:22.251035 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff4f22-f09a-4962-bef4-c147a95006a6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:22.251315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff4f22-f09a-4962-bef4-c147a95006a6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:22.251588 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff4f22-f09a-4962-bef4-c147a95006a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:22.257334 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 326/652] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:22.258141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4c74c-b457-4f40-bfc6-86f62e43e0aa None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:22.258583 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff4f22-f09a-4962-bef4-c147a95006a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:22.260086 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff4f22-f09a-4962-bef4-c147a95006a6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-dbff4f22-f09a-4962-bef4-c147a95006a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:22.274485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:22.275575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:22.276662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:22.276662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:22.276662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:22.276662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:22.276662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:22.276662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:22.276662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:22.276662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:22.276662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 327/653] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:22.276662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff4f22-f09a-4962-bef4-c147a95006a6 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:22.367005 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4c74c-b457-4f40-bfc6-86f62e43e0aa None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:22.374874 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:22.377879 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff4f22-f09a-4962-bef4-c147a95006a6 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:22.400165 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cf9ee-5dbb-46da-9081-104708be1ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:22.400990 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cf9ee-5dbb-46da-9081-104708be1ee8 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:22.401271 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cf9ee-5dbb-46da-9081-104708be1ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:22.401555 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cf9ee-5dbb-46da-9081-104708be1ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:22.410822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:22.412179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae38b7d-5c2a-485a-8cee-312fc66d4889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:22.412893 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae38b7d-5c2a-485a-8cee-312fc66d4889 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:22.413183 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae38b7d-5c2a-485a-8cee-312fc66d4889 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:22.413461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae38b7d-5c2a-485a-8cee-312fc66d4889 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:22.413779 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cf9ee-5dbb-46da-9081-104708be1ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:22.414333 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae38b7d-5c2a-485a-8cee-312fc66d4889 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:22.414647 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae38b7d-5c2a-485a-8cee-312fc66d4889 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:22.415259 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cf9ee-5dbb-46da-9081-104708be1ee8 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:22.416092 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae38b7d-5c2a-485a-8cee-312fc66d4889 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-027cf9ee-5dbb-46da-9081-104708be1ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:22.432954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:22.439484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:22.440752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:22.440752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:22.440752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:22.440752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:22.441027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 328/654] 10.222.0.88 () {56 vars in 1137 bytes} [Thu May 19 20:50:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:22.441111 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 327/655] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:22.441111 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cf9ee-5dbb-46da-9081-104708be1ee8 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:22.441256 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345a5c2-74d5-4962-aa5f-6d36b0b7d3ee None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:22.441256 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345a5c2-74d5-4962-aa5f-6d36b0b7d3ee None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:22.441256 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345a5c2-74d5-4962-aa5f-6d36b0b7d3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:22.454915 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-5345a5c2-74d5-4962-aa5f-6d36b0b7d3ee None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:22.456111 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-5345a5c2-74d5-4962-aa5f-6d36b0b7d3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:22.528811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 329/656] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:22.529686 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345a5c2-74d5-4962-aa5f-6d36b0b7d3ee None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:22.545149 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cf9ee-5dbb-46da-9081-104708be1ee8 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:22.580447 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839bddc-1b65-4ea3-9919-983ec4519627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:22.581239 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839bddc-1b65-4ea3-9919-983ec4519627 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:22.581523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839bddc-1b65-4ea3-9919-983ec4519627 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:22.581805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839bddc-1b65-4ea3-9919-983ec4519627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:22.592949 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839bddc-1b65-4ea3-9919-983ec4519627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:22.594406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839bddc-1b65-4ea3-9919-983ec4519627 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-7839bddc-1b65-4ea3-9919-983ec4519627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:22.612125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:22.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:22.614554 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:22.614554 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:22.614554 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:22.614554 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:22.616502 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 328/657] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:22.617494 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839bddc-1b65-4ea3-9919-983ec4519627 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:22.619472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:22.636309 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345a5c2-74d5-4962-aa5f-6d36b0b7d3ee None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:22.664776 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:22.671845 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a5ea4-b156-4b78-98a2-a350e709f02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:22.672545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5ea4-b156-4b78-98a2-a350e709f02e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:22.672831 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5ea4-b156-4b78-98a2-a350e709f02e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:22.677692 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a5ea4-b156-4b78-98a2-a350e709f02e None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:22.677692 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a5ea4-b156-4b78-98a2-a350e709f02e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:22.677692 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a5ea4-b156-4b78-98a2-a350e709f02e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:22.677692 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a5ea4-b156-4b78-98a2-a350e709f02e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:22.687489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 330/658] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:22.688296 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a5ea4-b156-4b78-98a2-a350e709f02e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:22.724923 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839bddc-1b65-4ea3-9919-983ec4519627 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:22.758236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fbc0c-ea08-4bed-a59e-9daa341b8078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:22.758895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fbc0c-ea08-4bed-a59e-9daa341b8078 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:22.759181 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fbc0c-ea08-4bed-a59e-9daa341b8078 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:22.759463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fbc0c-ea08-4bed-a59e-9daa341b8078 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:22.816094 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a5ea4-b156-4b78-98a2-a350e709f02e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:22.832813 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fbc0c-ea08-4bed-a59e-9daa341b8078 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:22.834288 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fbc0c-ea08-4bed-a59e-9daa341b8078 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:22.855144 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e213b-f086-491a-8bab-d85fe50e108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:22.855817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e213b-f086-491a-8bab-d85fe50e108d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:22.856101 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e213b-f086-491a-8bab-d85fe50e108d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:22.856418 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e213b-f086-491a-8bab-d85fe50e108d None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:22.857140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e213b-f086-491a-8bab-d85fe50e108d None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:22.857450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e213b-f086-491a-8bab-d85fe50e108d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:22.858884 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e213b-f086-491a-8bab-d85fe50e108d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:22.870946 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 331/659] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:22.871753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e213b-f086-491a-8bab-d85fe50e108d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:22.880561 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:22.880794 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:22.937550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 329/660] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:22.944275 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fbc0c-ea08-4bed-a59e-9daa341b8078 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:22.977419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e213b-f086-491a-8bab-d85fe50e108d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.016018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364cfc8-e120-409b-b139-fbba7deb6bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.017412 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364cfc8-e120-409b-b139-fbba7deb6bb8 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.017699 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364cfc8-e120-409b-b139-fbba7deb6bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.017973 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364cfc8-e120-409b-b139-fbba7deb6bb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.028214 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4364cfc8-e120-409b-b139-fbba7deb6bb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-4364cfc8-e120-409b-b139-fbba7deb6bb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:23.036054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:23.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:23.038416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 332/661] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:23.039223 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fbc0c-ea08-4bed-a59e-9daa341b8078 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.039307 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364cfc8-e120-409b-b139-fbba7deb6bb8 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:23.076196 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c761900-327c-488a-b201-15772920d6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.076900 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c761900-327c-488a-b201-15772920d6b3 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.077285 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c761900-327c-488a-b201-15772920d6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.077572 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c761900-327c-488a-b201-15772920d6b3 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.078303 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c761900-327c-488a-b201-15772920d6b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:23.078615 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c761900-327c-488a-b201-15772920d6b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:23.080658 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c761900-327c-488a-b201-15772920d6b3 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:23.092628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 330/662] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:23.093517 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c761900-327c-488a-b201-15772920d6b3 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:23.097595 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:23.105795 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:23.141257 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364cfc8-e120-409b-b139-fbba7deb6bb8 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.176118 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2e844-5d57-47cc-943e-dfd995a54d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.176821 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2e844-5d57-47cc-943e-dfd995a54d10 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.177115 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2e844-5d57-47cc-943e-dfd995a54d10 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.177398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2e844-5d57-47cc-943e-dfd995a54d10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.178247 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2e844-5d57-47cc-943e-dfd995a54d10 None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=77427e1028d349b4a740efe0a985e73b) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:23.178565 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2e844-5d57-47cc-943e-dfd995a54d10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:23.180026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2e844-5d57-47cc-943e-dfd995a54d10 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:23.186269 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 333/663] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/role_assignments?user.id=77427e1028d349b4a740efe0a985e73b&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:23.187036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2e844-5d57-47cc-943e-dfd995a54d10 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:23.198288 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c761900-327c-488a-b201-15772920d6b3 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.238019 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858304ee-34b2-407f-9f34-e57e7477afc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.238754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858304ee-34b2-407f-9f34-e57e7477afc2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.239036 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858304ee-34b2-407f-9f34-e57e7477afc2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.239312 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858304ee-34b2-407f-9f34-e57e7477afc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.247162 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858304ee-34b2-407f-9f34-e57e7477afc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:23.248709 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858304ee-34b2-407f-9f34-e57e7477afc2 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-858304ee-34b2-407f-9f34-e57e7477afc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:23.265320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:23.267248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:23.268509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:23.268509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:23.268509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:23.268509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:23.268509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:23.268509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:23.268509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:23.268509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:23.272685 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 331/664] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:23.273695 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858304ee-34b2-407f-9f34-e57e7477afc2 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:23.325871 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2e844-5d57-47cc-943e-dfd995a54d10 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.350343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:23.361197 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:23.367635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16feb389-ebd9-4eae-b84a-136a773273e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.368489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feb389-ebd9-4eae-b84a-136a773273e3 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.368775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feb389-ebd9-4eae-b84a-136a773273e3 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.369051 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16feb389-ebd9-4eae-b84a-136a773273e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.381053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16feb389-ebd9-4eae-b84a-136a773273e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:23.382538 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16feb389-ebd9-4eae-b84a-136a773273e3 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:23.398048 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858304ee-34b2-407f-9f34-e57e7477afc2 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-16feb389-ebd9-4eae-b84a-136a773273e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:23.402873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:23.403896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:23.405666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:23.405666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:23.405666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:23.405666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:23.405666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 334/665] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:23.405666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16feb389-ebd9-4eae-b84a-136a773273e3 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:23.441501 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c5de2-4c65-44f0-b8f0-7c23a8769334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.442538 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c5de2-4c65-44f0-b8f0-7c23a8769334 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.444639 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c5de2-4c65-44f0-b8f0-7c23a8769334 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.444639 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c5de2-4c65-44f0-b8f0-7c23a8769334 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.444639 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c5de2-4c65-44f0-b8f0-7c23a8769334 None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:23.444639 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c5de2-4c65-44f0-b8f0-7c23a8769334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:23.445583 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c5de2-4c65-44f0-b8f0-7c23a8769334 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:23.456817 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 332/666] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:23.458263 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c5de2-4c65-44f0-b8f0-7c23a8769334 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:23.518108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16feb389-ebd9-4eae-b84a-136a773273e3 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.555666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b49dc-1735-4b19-a32c-ac8d47c55135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.556593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b49dc-1735-4b19-a32c-ac8d47c55135 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.556880 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b49dc-1735-4b19-a32c-ac8d47c55135 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.557155 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b49dc-1735-4b19-a32c-ac8d47c55135 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.566368 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b49dc-1735-4b19-a32c-ac8d47c55135 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:23.567845 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c5de2-4c65-44f0-b8f0-7c23a8769334 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.567965 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b49dc-1735-4b19-a32c-ac8d47c55135 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-b51b49dc-1735-4b19-a32c-ac8d47c55135 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:23.583758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:23.585240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 335/667] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:23.586607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b49dc-1735-4b19-a32c-ac8d47c55135 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:23.603118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:23.612846 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acb447-841e-4838-af0d-8bf6dc6ae15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.613594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acb447-841e-4838-af0d-8bf6dc6ae15c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.613934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acb447-841e-4838-af0d-8bf6dc6ae15c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.614160 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98acb447-841e-4838-af0d-8bf6dc6ae15c None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.614965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acb447-841e-4838-af0d-8bf6dc6ae15c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:23.615278 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acb447-841e-4838-af0d-8bf6dc6ae15c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:23.616807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98acb447-841e-4838-af0d-8bf6dc6ae15c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:23.633851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 333/668] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:23.633851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acb447-841e-4838-af0d-8bf6dc6ae15c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:23.639011 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:23.700875 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b49dc-1735-4b19-a32c-ac8d47c55135 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.738935 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98acb447-841e-4838-af0d-8bf6dc6ae15c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.739194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33fc4f-c041-4cc7-aa60-19c334f30ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.739811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33fc4f-c041-4cc7-aa60-19c334f30ed2 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.740170 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33fc4f-c041-4cc7-aa60-19c334f30ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.740541 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e33fc4f-c041-4cc7-aa60-19c334f30ed2 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.741354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33fc4f-c041-4cc7-aa60-19c334f30ed2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:23.742369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33fc4f-c041-4cc7-aa60-19c334f30ed2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:23.743147 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e33fc4f-c041-4cc7-aa60-19c334f30ed2 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:23.756303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 336/669] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:23.756980 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33fc4f-c041-4cc7-aa60-19c334f30ed2 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:23.781537 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89372706-aa90-4159-ae45-936348ae6fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.782409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89372706-aa90-4159-ae45-936348ae6fa9 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.782693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89372706-aa90-4159-ae45-936348ae6fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.782974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89372706-aa90-4159-ae45-936348ae6fa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.794200 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89372706-aa90-4159-ae45-936348ae6fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:23.795647 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89372706-aa90-4159-ae45-936348ae6fa9 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-89372706-aa90-4159-ae45-936348ae6fa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:23.814360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:23.815611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:23.816991 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:23.816991 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:23.816991 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:23.816991 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:23.820204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 334/670] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:23.821074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89372706-aa90-4159-ae45-936348ae6fa9 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:23.865949 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:23.879749 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e33fc4f-c041-4cc7-aa60-19c334f30ed2 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.886987 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:23.916620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502ce10-381f-4d46-98ef-33ce084c7972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.917390 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502ce10-381f-4d46-98ef-33ce084c7972 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.919437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502ce10-381f-4d46-98ef-33ce084c7972 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.919437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502ce10-381f-4d46-98ef-33ce084c7972 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.919437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502ce10-381f-4d46-98ef-33ce084c7972 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:23.919437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502ce10-381f-4d46-98ef-33ce084c7972 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:23.920650 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502ce10-381f-4d46-98ef-33ce084c7972 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:23.938939 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89372706-aa90-4159-ae45-936348ae6fa9 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:23.949580 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 337/671] 10.222.0.88 () {56 vars in 1139 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:23.949580 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502ce10-381f-4d46-98ef-33ce084c7972 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:23.975903 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858b328-81a9-4d3f-a8bf-69802e35cba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:23.976721 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858b328-81a9-4d3f-a8bf-69802e35cba7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:23.977117 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858b328-81a9-4d3f-a8bf-69802e35cba7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:23.977398 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858b328-81a9-4d3f-a8bf-69802e35cba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:23.978227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858b328-81a9-4d3f-a8bf-69802e35cba7 None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=6ab3de874a0f475c8745831e76a6e4d0) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:23.978546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858b328-81a9-4d3f-a8bf-69802e35cba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:23.980188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858b328-81a9-4d3f-a8bf-69802e35cba7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:23.996559 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 335/672] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/role_assignments?user.id=6ab3de874a0f475c8745831e76a6e4d0&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:23.996559 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858b328-81a9-4d3f-a8bf-69802e35cba7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:24.077814 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502ce10-381f-4d46-98ef-33ce084c7972 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:24.123651 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97067a5-b22c-4063-95b2-d35088a12d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:24.124653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97067a5-b22c-4063-95b2-d35088a12d87 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:24.125038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97067a5-b22c-4063-95b2-d35088a12d87 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:24.125415 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97067a5-b22c-4063-95b2-d35088a12d87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:24.139271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97067a5-b22c-4063-95b2-d35088a12d87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:24.139271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97067a5-b22c-4063-95b2-d35088a12d87 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:24.154932 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858b328-81a9-4d3f-a8bf-69802e35cba7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:24.156667 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-f97067a5-b22c-4063-95b2-d35088a12d87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:24.161193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:24.162436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:24.163697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:24.163697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:24.163697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:24.163697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:24.163697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:24.163697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:24.163697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:24.163697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:24.168515 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 338/673] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:24.169699 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97067a5-b22c-4063-95b2-d35088a12d87 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:24.203999 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:24.209808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ea89f-469e-4432-b6a4-6f0a5f2f7da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:24.210672 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ea89f-469e-4432-b6a4-6f0a5f2f7da9 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:24.210672 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ea89f-469e-4432-b6a4-6f0a5f2f7da9 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:24.210896 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ea89f-469e-4432-b6a4-6f0a5f2f7da9 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:24.211575 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ea89f-469e-4432-b6a4-6f0a5f2f7da9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:24.211892 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ea89f-469e-4432-b6a4-6f0a5f2f7da9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:24.213881 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ea89f-469e-4432-b6a4-6f0a5f2f7da9 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:24.229917 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 336/674] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:24.231093 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ea89f-469e-4432-b6a4-6f0a5f2f7da9 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:24.319811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97067a5-b22c-4063-95b2-d35088a12d87 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:24.356963 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ea89f-469e-4432-b6a4-6f0a5f2f7da9 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:24.359582 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025fef7-882a-4add-b277-2e52ed99caeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:24.361071 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025fef7-882a-4add-b277-2e52ed99caeb None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:24.361565 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025fef7-882a-4add-b277-2e52ed99caeb None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:24.362034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025fef7-882a-4add-b277-2e52ed99caeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:24.393539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afd1c2-dee0-40ac-81ae-e4ec72c5cd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:24.394235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afd1c2-dee0-40ac-81ae-e4ec72c5cd47 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:24.394521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afd1c2-dee0-40ac-81ae-e4ec72c5cd47 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:24.394802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afd1c2-dee0-40ac-81ae-e4ec72c5cd47 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:24.395621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afd1c2-dee0-40ac-81ae-e4ec72c5cd47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:24.395989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afd1c2-dee0-40ac-81ae-e4ec72c5cd47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:24.397576 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afd1c2-dee0-40ac-81ae-e4ec72c5cd47 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:24.417663 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 337/675] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:24.418566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afd1c2-dee0-40ac-81ae-e4ec72c5cd47 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:24.443800 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025fef7-882a-4add-b277-2e52ed99caeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:24.445761 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025fef7-882a-4add-b277-2e52ed99caeb None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:24.449498 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:24.470246 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:24.532619 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afd1c2-dee0-40ac-81ae-e4ec72c5cd47 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:24.559854 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 339/676] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:24.560874 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025fef7-882a-4add-b277-2e52ed99caeb None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:24.574415 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aecc3a-f5a6-4874-a4a6-ba35e47b4dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:24.575506 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aecc3a-f5a6-4874-a4a6-ba35e47b4dfc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:24.576191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aecc3a-f5a6-4874-a4a6-ba35e47b4dfc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:24.576692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aecc3a-f5a6-4874-a4a6-ba35e47b4dfc None admin] PATH_INFO: `/v3/users/swift` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:24.587433 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aecc3a-f5a6-4874-a4a6-ba35e47b4dfc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:24.589725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aecc3a-f5a6-4874-a4a6-ba35e47b4dfc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-57aecc3a-f5a6-4874-a4a6-ba35e47b4dfc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:24.609965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:24.611893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:24.613227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:24.613227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:24.613227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:24.613227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:24.613227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:24.613227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:24.613227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:24.613227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:24.613227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:24.613227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:24.616493 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 338/677] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:24.618197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aecc3a-f5a6-4874-a4a6-ba35e47b4dfc None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:24.680100 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025fef7-882a-4add-b277-2e52ed99caeb None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:24.709636 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:24.716839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:24.729680 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e1502-dacf-41f9-ae2f-1491b768ff7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:24.730501 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e1502-dacf-41f9-ae2f-1491b768ff7a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:24.730804 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e1502-dacf-41f9-ae2f-1491b768ff7a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:24.731087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e1502-dacf-41f9-ae2f-1491b768ff7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:24.732374 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e1502-dacf-41f9-ae2f-1491b768ff7a None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=e2f9d95b4a2d4816adceffdd1a49186f) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:24.732732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e1502-dacf-41f9-ae2f-1491b768ff7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:24.734690 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e1502-dacf-41f9-ae2f-1491b768ff7a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:24.738895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aecc3a-f5a6-4874-a4a6-ba35e47b4dfc None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:24.748366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 340/678] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:24 2022] GET /identity/v3/role_assignments?user.id=e2f9d95b4a2d4816adceffdd1a49186f&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:24.748366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e1502-dacf-41f9-ae2f-1491b768ff7a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:24.848604 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d663e9-c71f-4f0a-8db2-cc25625f764d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:24.849433 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d663e9-c71f-4f0a-8db2-cc25625f764d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:24.849721 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d663e9-c71f-4f0a-8db2-cc25625f764d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:24.850000 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d663e9-c71f-4f0a-8db2-cc25625f764d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:24.869086 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e1502-dacf-41f9-ae2f-1491b768ff7a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:24.871206 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d663e9-c71f-4f0a-8db2-cc25625f764d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:24.873072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d663e9-c71f-4f0a-8db2-cc25625f764d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-31d663e9-c71f-4f0a-8db2-cc25625f764d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:24.909443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:24.910523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:24.911516 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:24.911516 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:24.911516 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:24.911516 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:24.911516 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 339/679] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:24.916863 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d663e9-c71f-4f0a-8db2-cc25625f764d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:24.928501 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1cbe5-3e5f-40ea-a76a-24b7cbe62cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:24.929207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1cbe5-3e5f-40ea-a76a-24b7cbe62cec None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:24.929481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1cbe5-3e5f-40ea-a76a-24b7cbe62cec None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:24.929758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1cbe5-3e5f-40ea-a76a-24b7cbe62cec None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:24.930629 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1cbe5-3e5f-40ea-a76a-24b7cbe62cec None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:24.930945 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1cbe5-3e5f-40ea-a76a-24b7cbe62cec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:24.932542 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1cbe5-3e5f-40ea-a76a-24b7cbe62cec None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:24.967815 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 341/680] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:50:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:24.975216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1cbe5-3e5f-40ea-a76a-24b7cbe62cec None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:25.042087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:25.073930 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d663e9-c71f-4f0a-8db2-cc25625f764d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:25.113896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1cbe5-3e5f-40ea-a76a-24b7cbe62cec None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:25.120374 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f64893-970b-418c-bae8-c51f9a3d1912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:25.121102 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f64893-970b-418c-bae8-c51f9a3d1912 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:25.121388 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f64893-970b-418c-bae8-c51f9a3d1912 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:25.121670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f64893-970b-418c-bae8-c51f9a3d1912 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:25.122415 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f64893-970b-418c-bae8-c51f9a3d1912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:25.122728 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f64893-970b-418c-bae8-c51f9a3d1912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:25.124191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f64893-970b-418c-bae8-c51f9a3d1912 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:25.126446 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:25.140263 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 340/681] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:25.147944 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f64893-970b-418c-bae8-c51f9a3d1912 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:25.167700 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfde646-cf4b-43f2-a76c-c9f34c062dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:25.168580 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde646-cf4b-43f2-a76c-c9f34c062dfb None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:25.168987 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde646-cf4b-43f2-a76c-c9f34c062dfb None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:25.169278 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfde646-cf4b-43f2-a76c-c9f34c062dfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:25.263415 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde646-cf4b-43f2-a76c-c9f34c062dfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:25.265203 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfde646-cf4b-43f2-a76c-c9f34c062dfb None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:25.273749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f64893-970b-418c-bae8-c51f9a3d1912 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:25.322806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:25.323316 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6fa3c-1e8b-419d-8a7d-2a6b679b6163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:25.324145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6fa3c-1e8b-419d-8a7d-2a6b679b6163 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:25.324674 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6fa3c-1e8b-419d-8a7d-2a6b679b6163 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:25.324789 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6fa3c-1e8b-419d-8a7d-2a6b679b6163 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:25.341327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6fa3c-1e8b-419d-8a7d-2a6b679b6163 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:25.342824 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6fa3c-1e8b-419d-8a7d-2a6b679b6163 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-b5f6fa3c-1e8b-419d-8a7d-2a6b679b6163 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:25.374042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:25.375080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:25.376691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:25.376691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:25.376691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:25.376691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:25.376691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:25.376691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:25.376691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:25.376691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:25.376691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 341/682] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:25.382089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6fa3c-1e8b-419d-8a7d-2a6b679b6163 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:25.417894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 342/683] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:25.426131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfde646-cf4b-43f2-a76c-c9f34c062dfb None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:25.472495 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:25.524495 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6fa3c-1e8b-419d-8a7d-2a6b679b6163 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:25.567213 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25638cf-5aec-44d0-8c17-244737fc33b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:25.567885 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25638cf-5aec-44d0-8c17-244737fc33b5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:25.568229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25638cf-5aec-44d0-8c17-244737fc33b5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:25.568543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25638cf-5aec-44d0-8c17-244737fc33b5 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:25.569526 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25638cf-5aec-44d0-8c17-244737fc33b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:25.569843 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25638cf-5aec-44d0-8c17-244737fc33b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:25.571290 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25638cf-5aec-44d0-8c17-244737fc33b5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:25.585021 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 342/684] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:25.590335 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25638cf-5aec-44d0-8c17-244737fc33b5 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:25.616297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfde646-cf4b-43f2-a76c-c9f34c062dfb None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:25.676063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc483aa6-17f5-489f-9d27-64d79e23420d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:25.676860 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc483aa6-17f5-489f-9d27-64d79e23420d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:25.677180 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc483aa6-17f5-489f-9d27-64d79e23420d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:25.677476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc483aa6-17f5-489f-9d27-64d79e23420d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:25.690113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc483aa6-17f5-489f-9d27-64d79e23420d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:25.691692 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc483aa6-17f5-489f-9d27-64d79e23420d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:25.693656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-fc483aa6-17f5-489f-9d27-64d79e23420d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:25.711539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:25.712630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:25.713666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:25.713666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:25.713666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:25.713666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:25.717487 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 343/685] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:25.718881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09817bc3-1606-41a5-aba9-391636180f08 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:25.719173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09817bc3-1606-41a5-aba9-391636180f08 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:25.719486 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09817bc3-1606-41a5-aba9-391636180f08 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:25.721107 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 344/686] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:25.721821 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09817bc3-1606-41a5-aba9-391636180f08 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:25.730652 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25638cf-5aec-44d0-8c17-244737fc33b5 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:25.746303 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:25.769194 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f460130-01ef-4198-a3df-db6cfa74b17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:25.769874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460130-01ef-4198-a3df-db6cfa74b17a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:25.770167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460130-01ef-4198-a3df-db6cfa74b17a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:25.770452 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f460130-01ef-4198-a3df-db6cfa74b17a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:25.771302 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f460130-01ef-4198-a3df-db6cfa74b17a None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=4b3f694758fa483ca497fbea18061cb6) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:25.771622 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f460130-01ef-4198-a3df-db6cfa74b17a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:25.773105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f460130-01ef-4198-a3df-db6cfa74b17a None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:25.779907 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 343/687] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:25 2022] GET /identity/v3/role_assignments?user.id=4b3f694758fa483ca497fbea18061cb6&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:25.781749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e22946-472d-46e3-aa09-73dd38ef82d0 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:25.782400 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e22946-472d-46e3-aa09-73dd38ef82d0 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:25.782722 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e22946-472d-46e3-aa09-73dd38ef82d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:25.802331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-17e22946-472d-46e3-aa09-73dd38ef82d0 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:25.803674 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-17e22946-472d-46e3-aa09-73dd38ef82d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:25.871188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09817bc3-1606-41a5-aba9-391636180f08 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:25.917403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 344/688] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:25.944097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e22946-472d-46e3-aa09-73dd38ef82d0 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:25.986416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fe985-44ec-4642-b4f4-a25ddbacc192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:25.987139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fe985-44ec-4642-b4f4-a25ddbacc192 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:25.987381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fe985-44ec-4642-b4f4-a25ddbacc192 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:25.987654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6fe985-44ec-4642-b4f4-a25ddbacc192 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:25.988418 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fe985-44ec-4642-b4f4-a25ddbacc192 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:25.988731 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fe985-44ec-4642-b4f4-a25ddbacc192 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:25.990150 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6fe985-44ec-4642-b4f4-a25ddbacc192 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:26.025629 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 345/689] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:26.030954 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fe985-44ec-4642-b4f4-a25ddbacc192 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:26.077507 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e22946-472d-46e3-aa09-73dd38ef82d0 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:26.091507 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:26.126683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad4b6e-4518-495f-bce1-65adf84680f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:26.127842 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad4b6e-4518-495f-bce1-65adf84680f7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:26.128130 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad4b6e-4518-495f-bce1-65adf84680f7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:26.128702 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad4b6e-4518-495f-bce1-65adf84680f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:26.130832 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:26.144477 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad4b6e-4518-495f-bce1-65adf84680f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-6cad4b6e-4518-495f-bce1-65adf84680f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:26.154053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:26.155243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:26.156225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:26.159552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 345/690] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:26.165278 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad4b6e-4518-495f-bce1-65adf84680f7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:26.193253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6fe985-44ec-4642-b4f4-a25ddbacc192 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:26.239103 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5f4e6-9488-48d7-a4fb-bc4b4d21b518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:26.239938 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5f4e6-9488-48d7-a4fb-bc4b4d21b518 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:26.240228 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5f4e6-9488-48d7-a4fb-bc4b4d21b518 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:26.240533 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5f4e6-9488-48d7-a4fb-bc4b4d21b518 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:26.251312 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5f4e6-9488-48d7-a4fb-bc4b4d21b518 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:26.252841 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5f4e6-9488-48d7-a4fb-bc4b4d21b518 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-2de5f4e6-9488-48d7-a4fb-bc4b4d21b518 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:26.272543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:26.273619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:26.274779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:26.274779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:26.274779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:26.274779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:26.274779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:26.274779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:26.274779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:26.274779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:26.274779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 346/691] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:26.280306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5f4e6-9488-48d7-a4fb-bc4b4d21b518 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:26.301589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad4b6e-4518-495f-bce1-65adf84680f7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:26.354026 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce392d-9c50-4271-b0be-7eec51063144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:26.355186 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce392d-9c50-4271-b0be-7eec51063144 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:26.355703 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce392d-9c50-4271-b0be-7eec51063144 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:26.356218 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce392d-9c50-4271-b0be-7eec51063144 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:26.357234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce392d-9c50-4271-b0be-7eec51063144 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:26.361202 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce392d-9c50-4271-b0be-7eec51063144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:26.362859 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce392d-9c50-4271-b0be-7eec51063144 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:26.384767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 346/692] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:26.390205 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce392d-9c50-4271-b0be-7eec51063144 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:26.443049 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5f4e6-9488-48d7-a4fb-bc4b4d21b518 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:26.447174 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:26.463342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:26.510895 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15007b6-d5d3-4589-b7d0-14a25be2e8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:26.512002 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15007b6-d5d3-4589-b7d0-14a25be2e8d6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:26.512539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15007b6-d5d3-4589-b7d0-14a25be2e8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:26.513032 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15007b6-d5d3-4589-b7d0-14a25be2e8d6 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:26.514069 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15007b6-d5d3-4589-b7d0-14a25be2e8d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:26.514609 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15007b6-d5d3-4589-b7d0-14a25be2e8d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:26.516931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15007b6-d5d3-4589-b7d0-14a25be2e8d6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:26.531248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce392d-9c50-4271-b0be-7eec51063144 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:26.541090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 347/693] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:26.546734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15007b6-d5d3-4589-b7d0-14a25be2e8d6 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:26.567800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f909be-cd10-47a4-a654-3b89f8c4e598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:26.568735 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f909be-cd10-47a4-a654-3b89f8c4e598 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:26.569020 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f909be-cd10-47a4-a654-3b89f8c4e598 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:26.569297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f909be-cd10-47a4-a654-3b89f8c4e598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:26.596752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f909be-cd10-47a4-a654-3b89f8c4e598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:26.598225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f909be-cd10-47a4-a654-3b89f8c4e598 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-70f909be-cd10-47a4-a654-3b89f8c4e598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:26.640655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:26.642311 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:26.643340 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:26.643340 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:26.643340 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:26.643340 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:26.643340 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 347/694] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:26.648678 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f909be-cd10-47a4-a654-3b89f8c4e598 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:26.721105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15007b6-d5d3-4589-b7d0-14a25be2e8d6 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:26.752440 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:26.758547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bb713-771b-48fc-938d-d0940cf65f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:26.759534 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bb713-771b-48fc-938d-d0940cf65f51 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:26.760036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bb713-771b-48fc-938d-d0940cf65f51 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:26.760604 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bb713-771b-48fc-938d-d0940cf65f51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:26.761679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bb713-771b-48fc-938d-d0940cf65f51 None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=ba6e1d31d8004f9fa956b78d8716d6e6) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:26.762210 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bb713-771b-48fc-938d-d0940cf65f51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:26.763928 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bb713-771b-48fc-938d-d0940cf65f51 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:26.771120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 348/695] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:26 2022] GET /identity/v3/role_assignments?user.id=ba6e1d31d8004f9fa956b78d8716d6e6&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:26.772851 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d57237-6c0c-4b21-9691-4c8d5de4bf9b None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:26.773384 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d57237-6c0c-4b21-9691-4c8d5de4bf9b None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:26.773680 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f909be-cd10-47a4-a654-3b89f8c4e598 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:26.774151 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d57237-6c0c-4b21-9691-4c8d5de4bf9b None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:26.776191 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 349/696] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:26.782091 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9362b-1ca6-41e4-8f93-ae360adede4c None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:26.782714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9362b-1ca6-41e4-8f93-ae360adede4c None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:26.783207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9362b-1ca6-41e4-8f93-ae360adede4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:26.803028 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-b5c9362b-1ca6-41e4-8f93-ae360adede4c None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:26.804899 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-b5c9362b-1ca6-41e4-8f93-ae360adede4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:26.825296 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5c600-9346-4125-af5d-49bb8f81ecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:26.826377 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5c600-9346-4125-af5d-49bb8f81ecc6 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:26.826876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5c600-9346-4125-af5d-49bb8f81ecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:26.827727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5c600-9346-4125-af5d-49bb8f81ecc6 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:26.829627 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5c600-9346-4125-af5d-49bb8f81ecc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:26.831873 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5c600-9346-4125-af5d-49bb8f81ecc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:26.833856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5c600-9346-4125-af5d-49bb8f81ecc6 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:26.864723 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 348/697] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:26.868398 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5c600-9346-4125-af5d-49bb8f81ecc6 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:26.876734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:26.959908 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 350/698] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:27.001512 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9362b-1ca6-41e4-8f93-ae360adede4c None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:27.032365 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5c600-9346-4125-af5d-49bb8f81ecc6 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:27.080769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356536b0-32ca-4b8d-983c-15ce0e219c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:27.081934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356536b0-32ca-4b8d-983c-15ce0e219c26 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:27.082246 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356536b0-32ca-4b8d-983c-15ce0e219c26 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:27.082568 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356536b0-32ca-4b8d-983c-15ce0e219c26 None admin] PATH_INFO: `/v3/users/nova` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:27.092767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356536b0-32ca-4b8d-983c-15ce0e219c26 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:27.095558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356536b0-32ca-4b8d-983c-15ce0e219c26 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-356536b0-32ca-4b8d-983c-15ce0e219c26 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:27.114500 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:27.115621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:27.117343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:27.117343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:27.117343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:27.117343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:27.117343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:27.117343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:27.117343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:27.117343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:27.117343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:27.117343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:27.117343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 349/699] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:50:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:27.122706 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356536b0-32ca-4b8d-983c-15ce0e219c26 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:27.125512 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:27.134648 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9362b-1ca6-41e4-8f93-ae360adede4c None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:27.173608 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231f20a-f43c-43bd-82f4-f74ebdd37239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:27.174785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231f20a-f43c-43bd-82f4-f74ebdd37239 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:27.175313 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231f20a-f43c-43bd-82f4-f74ebdd37239 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:27.175953 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231f20a-f43c-43bd-82f4-f74ebdd37239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:27.189972 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231f20a-f43c-43bd-82f4-f74ebdd37239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:27.191750 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231f20a-f43c-43bd-82f4-f74ebdd37239 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-2231f20a-f43c-43bd-82f4-f74ebdd37239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:27.211981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:27.213276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:27.214299 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:27.214299 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:27.214299 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:27.214299 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:27.216545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 351/700] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:27.222889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231f20a-f43c-43bd-82f4-f74ebdd37239 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:27.248561 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356536b0-32ca-4b8d-983c-15ce0e219c26 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:27.258655 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:27.296850 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e635a5-721f-4ee0-afae-c9576370d4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:27.298643 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e635a5-721f-4ee0-afae-c9576370d4b6 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:27.299188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e635a5-721f-4ee0-afae-c9576370d4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:27.300789 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e635a5-721f-4ee0-afae-c9576370d4b6 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:27.302515 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e635a5-721f-4ee0-afae-c9576370d4b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:27.303309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e635a5-721f-4ee0-afae-c9576370d4b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:27.305737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e635a5-721f-4ee0-afae-c9576370d4b6 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:27.331234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 350/701] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:50:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:27.336407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e635a5-721f-4ee0-afae-c9576370d4b6 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:27.362185 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231f20a-f43c-43bd-82f4-f74ebdd37239 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:27.408318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e287f-38a9-4d37-b483-f1394f2b5871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:27.409310 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e287f-38a9-4d37-b483-f1394f2b5871 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:27.409605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e287f-38a9-4d37-b483-f1394f2b5871 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:27.410414 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e287f-38a9-4d37-b483-f1394f2b5871 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:27.411167 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e287f-38a9-4d37-b483-f1394f2b5871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:27.411556 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e287f-38a9-4d37-b483-f1394f2b5871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:27.413829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e287f-38a9-4d37-b483-f1394f2b5871 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:27.415545 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:27.428209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 352/702] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:27.437424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e287f-38a9-4d37-b483-f1394f2b5871 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:27.467477 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e635a5-721f-4ee0-afae-c9576370d4b6 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:27.509299 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc34cb5-4d36-4333-96ff-de9c2aac7ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:27.509894 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc34cb5-4d36-4333-96ff-de9c2aac7ab5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:27.510181 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc34cb5-4d36-4333-96ff-de9c2aac7ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:27.510462 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc34cb5-4d36-4333-96ff-de9c2aac7ab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:27.585192 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc34cb5-4d36-4333-96ff-de9c2aac7ab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:27.586661 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc34cb5-4d36-4333-96ff-de9c2aac7ab5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:27.624768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:27.643630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e287f-38a9-4d37-b483-f1394f2b5871 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:27.651187 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:27.692737 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30bab6-b67d-4c71-b450-c9cca98e5b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:27.693567 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30bab6-b67d-4c71-b450-c9cca98e5b7e None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:27.693921 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30bab6-b67d-4c71-b450-c9cca98e5b7e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:27.694271 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30bab6-b67d-4c71-b450-c9cca98e5b7e None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:27.695233 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30bab6-b67d-4c71-b450-c9cca98e5b7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:27.697451 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30bab6-b67d-4c71-b450-c9cca98e5b7e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:27.702889 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 351/703] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:27.709162 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deab51e-f85e-42ca-99cc-3d2b1671ba3f None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:27.709479 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deab51e-f85e-42ca-99cc-3d2b1671ba3f None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:27.709810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deab51e-f85e-42ca-99cc-3d2b1671ba3f None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:27.711328 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 352/704] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:27.711952 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deab51e-f85e-42ca-99cc-3d2b1671ba3f None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:27.828984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 353/705] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:50:27 2022] POST /identity/v3/users => generated 254 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:27.831132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866c5c3-f3e6-48e2-8bc5-bf56ecda5b52 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:27.831711 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866c5c3-f3e6-48e2-8bc5-bf56ecda5b52 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:27.832227 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866c5c3-f3e6-48e2-8bc5-bf56ecda5b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:27.846734 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deab51e-f85e-42ca-99cc-3d2b1671ba3f None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:27.865056 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-6866c5c3-f3e6-48e2-8bc5-bf56ecda5b52 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:27.867300 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-6866c5c3-f3e6-48e2-8bc5-bf56ecda5b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:27.961975 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 354/706] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:27.977718 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1a021-5cd2-452f-a7c7-12f60d20f4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:27.978549 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1a021-5cd2-452f-a7c7-12f60d20f4ec None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:27.978832 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1a021-5cd2-452f-a7c7-12f60d20f4ec None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:27.979107 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1a021-5cd2-452f-a7c7-12f60d20f4ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:27.993484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866c5c3-f3e6-48e2-8bc5-bf56ecda5b52 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:28.010208 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1a021-5cd2-452f-a7c7-12f60d20f4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:28.011686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1a021-5cd2-452f-a7c7-12f60d20f4ec None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-50c1a021-5cd2-452f-a7c7-12f60d20f4ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:28.044675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:28.045761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:28.046840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:28.046840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:28.046840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:28.046840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:28.046840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 353/707] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:28.052805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1a021-5cd2-452f-a7c7-12f60d20f4ec None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:28.070275 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:28.129609 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:28.177355 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866c5c3-f3e6-48e2-8bc5-bf56ecda5b52 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:28.185707 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1a021-5cd2-452f-a7c7-12f60d20f4ec None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:28.225526 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6ef08-a867-4fe1-b70d-acdcf31953cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:28.226199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6ef08-a867-4fe1-b70d-acdcf31953cc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:28.226483 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6ef08-a867-4fe1-b70d-acdcf31953cc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:28.226768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6ef08-a867-4fe1-b70d-acdcf31953cc None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:28.227121 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f64d68-1120-4ecf-b410-360dba6826c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:28.227497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6ef08-a867-4fe1-b70d-acdcf31953cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:28.227806 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6ef08-a867-4fe1-b70d-acdcf31953cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:28.228005 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f64d68-1120-4ecf-b410-360dba6826c0 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:28.228296 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f64d68-1120-4ecf-b410-360dba6826c0 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:28.228601 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f64d68-1120-4ecf-b410-360dba6826c0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:28.229209 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6ef08-a867-4fe1-b70d-acdcf31953cc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:28.229339 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f64d68-1120-4ecf-b410-360dba6826c0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:28.231007 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f64d68-1120-4ecf-b410-360dba6826c0 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:28.243969 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 354/708] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:28.249415 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6ef08-a867-4fe1-b70d-acdcf31953cc None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-85f64d68-1120-4ecf-b410-360dba6826c0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:28.250703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:28.251861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:28.252878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 355/709] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 20:50:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:28.259040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f64d68-1120-4ecf-b410-360dba6826c0 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:28.430748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:28.504443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6ef08-a867-4fe1-b70d-acdcf31953cc None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:28.570465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f64d68-1120-4ecf-b410-360dba6826c0 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:28.585176 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378ada1-f96f-4d75-a279-a2e816f1b49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:28.586119 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ada1-f96f-4d75-a279-a2e816f1b49c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:28.586665 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ada1-f96f-4d75-a279-a2e816f1b49c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:28.586765 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ada1-f96f-4d75-a279-a2e816f1b49c None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:28.607615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:28.609304 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378ada1-f96f-4d75-a279-a2e816f1b49c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:28.610852 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378ada1-f96f-4d75-a279-a2e816f1b49c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:28.614345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d440a-69ce-44a6-b9b7-69082970c881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:28.615035 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d440a-69ce-44a6-b9b7-69082970c881 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:28.615321 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d440a-69ce-44a6-b9b7-69082970c881 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:28.625178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d440a-69ce-44a6-b9b7-69082970c881 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:28.625178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d440a-69ce-44a6-b9b7-69082970c881 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:28.625178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d440a-69ce-44a6-b9b7-69082970c881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:28.625178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d440a-69ce-44a6-b9b7-69082970c881 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:28.625178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 356/710] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:50:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:28.629366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d440a-69ce-44a6-b9b7-69082970c881 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-4378ada1-f96f-4d75-a279-a2e816f1b49c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:28.633509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:28.634508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:28.635525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:28.635525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:28.635525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:28.635525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:28.635525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:28.635525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:28.635525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:28.635525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:28.635525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 355/711] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:28.642411 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378ada1-f96f-4d75-a279-a2e816f1b49c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:28.760025 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378ada1-f96f-4d75-a279-a2e816f1b49c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:28.803612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e1066-a382-493b-ba32-6030b1aaefac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:28.804499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1066-a382-493b-ba32-6030b1aaefac None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:28.804899 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1066-a382-493b-ba32-6030b1aaefac None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:28.805722 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1066-a382-493b-ba32-6030b1aaefac None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:28.806929 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e1066-a382-493b-ba32-6030b1aaefac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:28.807368 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e1066-a382-493b-ba32-6030b1aaefac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:28.809748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e1066-a382-493b-ba32-6030b1aaefac None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:28.817903 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d440a-69ce-44a6-b9b7-69082970c881 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:28.825753 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 356/712] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:28.827217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efabf2-d648-435e-b0fa-22b51ced0ba7 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:28.827517 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efabf2-d648-435e-b0fa-22b51ced0ba7 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:28.827803 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efabf2-d648-435e-b0fa-22b51ced0ba7 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:28.830655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 357/713] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:28.832377 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efabf2-d648-435e-b0fa-22b51ced0ba7 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:28.848584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:28.863087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e7aa5-5146-418d-ab9c-2b71c54d7ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:28.864051 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e7aa5-5146-418d-ab9c-2b71c54d7ec4 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:28.865490 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e7aa5-5146-418d-ab9c-2b71c54d7ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:28.865490 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e7aa5-5146-418d-ab9c-2b71c54d7ec4 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:28.866437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e7aa5-5146-418d-ab9c-2b71c54d7ec4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:28.866964 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e7aa5-5146-418d-ab9c-2b71c54d7ec4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:28.868627 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e7aa5-5146-418d-ab9c-2b71c54d7ec4 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:28.875404 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 357/714] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:50:28 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:28.877179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b636e0-3aef-4831-8aab-776e9b61fe48 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:28.877722 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b636e0-3aef-4831-8aab-776e9b61fe48 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:28.878238 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b636e0-3aef-4831-8aab-776e9b61fe48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:28.908838 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-49b636e0-3aef-4831-8aab-776e9b61fe48 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:28.910472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-49b636e0-3aef-4831-8aab-776e9b61fe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:28.978605 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efabf2-d648-435e-b0fa-22b51ced0ba7 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:29.008690 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:29.021378 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200264b-7917-431f-a133-b3b2aac77585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:29.021378 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200264b-7917-431f-a133-b3b2aac77585 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:29.021378 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200264b-7917-431f-a133-b3b2aac77585 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:29.021378 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200264b-7917-431f-a133-b3b2aac77585 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:29.022079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200264b-7917-431f-a133-b3b2aac77585 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=77427e1028d349b4a740efe0a985e73b) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:29.022561 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200264b-7917-431f-a133-b3b2aac77585 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:29.024187 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200264b-7917-431f-a133-b3b2aac77585 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:29.031925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 358/715] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:28 2022] GET /identity/v3/role_assignments?user.id=77427e1028d349b4a740efe0a985e73b&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:29.038269 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 358/716] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:29.075938 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200264b-7917-431f-a133-b3b2aac77585 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:29.145634 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:29.203060 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200264b-7917-431f-a133-b3b2aac77585 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:29.242752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cd5a7-4a8e-4f36-943e-ca517ba0b156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:29.243566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cd5a7-4a8e-4f36-943e-ca517ba0b156 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:29.243856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cd5a7-4a8e-4f36-943e-ca517ba0b156 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:29.244140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0cd5a7-4a8e-4f36-943e-ca517ba0b156 None admin] PATH_INFO: `/v3/services/network` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:29.244836 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0cd5a7-4a8e-4f36-943e-ca517ba0b156 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:29.246222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0cd5a7-4a8e-4f36-943e-ca517ba0b156 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-3a0cd5a7-4a8e-4f36-943e-ca517ba0b156 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:29.261313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:29.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:29.263484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 359/717] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:29 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:29.271840 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b636e0-3aef-4831-8aab-776e9b61fe48 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:29.407875 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b636e0-3aef-4831-8aab-776e9b61fe48 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:29.456869 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eec56d-1fb2-4a05-adca-8f6eda58e2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:29.457943 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eec56d-1fb2-4a05-adca-8f6eda58e2da None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:29.458473 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eec56d-1fb2-4a05-adca-8f6eda58e2da None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:29.458966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eec56d-1fb2-4a05-adca-8f6eda58e2da None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:29.459899 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eec56d-1fb2-4a05-adca-8f6eda58e2da None admin] RBAC: Adding query filter params (name=network) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:29.460462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eec56d-1fb2-4a05-adca-8f6eda58e2da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:29.465123 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eec56d-1fb2-4a05-adca-8f6eda58e2da None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:29.471220 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 359/718] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:50:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:29.476957 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cd5a7-4a8e-4f36-943e-ca517ba0b156 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:29.610126 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0cd5a7-4a8e-4f36-943e-ca517ba0b156 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:29.631308 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:29.646911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd2c12-e264-4685-93bf-63609ccc0bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:29.647603 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd2c12-e264-4685-93bf-63609ccc0bfc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:29.647892 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd2c12-e264-4685-93bf-63609ccc0bfc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:29.648177 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd2c12-e264-4685-93bf-63609ccc0bfc None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:29.648938 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd2c12-e264-4685-93bf-63609ccc0bfc None admin] RBAC: Adding query filter params (type=network) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:29.649253 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd2c12-e264-4685-93bf-63609ccc0bfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:29.650628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd2c12-e264-4685-93bf-63609ccc0bfc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:29.656902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 360/719] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:50:29 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:30.053086 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900df458-0555-4157-aa26-ca96efdf2987 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:30.053793 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900df458-0555-4157-aa26-ca96efdf2987 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:30.054355 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900df458-0555-4157-aa26-ca96efdf2987 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:30.056462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 360/720] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:30.062611 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651f301-86e0-4028-9548-b5b032bf4d96 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:30.062907 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651f301-86e0-4028-9548-b5b032bf4d96 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:30.063334 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651f301-86e0-4028-9548-b5b032bf4d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:30.083235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-2651f301-86e0-4028-9548-b5b032bf4d96 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:30.092589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-2651f301-86e0-4028-9548-b5b032bf4d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:30.182473 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 361/721] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 19 20:50:30.226576 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900df458-0555-4157-aa26-ca96efdf2987 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:30.307499 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:30.375600 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900df458-0555-4157-aa26-ca96efdf2987 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:30.429273 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b68c1-e988-47de-830d-812233c5d2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:30.430100 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b68c1-e988-47de-830d-812233c5d2fe None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:30.430388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b68c1-e988-47de-830d-812233c5d2fe None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:30.430672 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b68c1-e988-47de-830d-812233c5d2fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:30.443569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b68c1-e988-47de-830d-812233c5d2fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-b35b68c1-e988-47de-830d-812233c5d2fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:30.451337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:30.452448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:30.453422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 361/722] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:30.459519 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651f301-86e0-4028-9548-b5b032bf4d96 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:30.613154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f79d2f-39c5-4faa-b873-458a20747f12 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:30.613508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f79d2f-39c5-4faa-b873-458a20747f12 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:30.613744 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f79d2f-39c5-4faa-b873-458a20747f12 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:30.615266 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 362/723] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:30.616266 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651f301-86e0-4028-9548-b5b032bf4d96 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:30.622767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72916-439a-447f-9625-a720c10b5b3d None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:30.623065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72916-439a-447f-9625-a720c10b5b3d None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:30.623354 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc72916-439a-447f-9625-a720c10b5b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:30.632057 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:30.654409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a5322-2555-40cd-84ed-bb6a7404a62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:30.655086 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a5322-2555-40cd-84ed-bb6a7404a62a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:30.655369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a5322-2555-40cd-84ed-bb6a7404a62a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:30.655653 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a5322-2555-40cd-84ed-bb6a7404a62a None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:30.655764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-7fc72916-439a-447f-9625-a720c10b5b3d None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:30.656426 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a5322-2555-40cd-84ed-bb6a7404a62a None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:30.656738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a5322-2555-40cd-84ed-bb6a7404a62a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:30.656966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-7fc72916-439a-447f-9625-a720c10b5b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:30.658141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a5322-2555-40cd-84ed-bb6a7404a62a None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:30.668205 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 362/724] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:30.673904 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a5322-2555-40cd-84ed-bb6a7404a62a None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:30.783416 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a5322-2555-40cd-84ed-bb6a7404a62a None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:30.834431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 363/725] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 19 20:50:30.855836 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc72916-439a-447f-9625-a720c10b5b3d None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:30.870635 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7d20a-1097-4320-b481-196a106c6b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:30.871465 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7d20a-1097-4320-b481-196a106c6b86 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:30.871755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7d20a-1097-4320-b481-196a106c6b86 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:30.872042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7d20a-1097-4320-b481-196a106c6b86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:30.904060 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7d20a-1097-4320-b481-196a106c6b86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:30.905581 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee7d20a-1097-4320-b481-196a106c6b86 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-8ee7d20a-1097-4320-b481-196a106c6b86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:30.944203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:30.945439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:30.946480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:30.946480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:30.946480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:30.946480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:30.952428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 363/726] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:30.952766 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:30.953335 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7d20a-1097-4320-b481-196a106c6b86 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:31.014101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:31.042654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc72916-439a-447f-9625-a720c10b5b3d None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:31.088869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee7d20a-1097-4320-b481-196a106c6b86 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:31.117403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867173b3-0719-4a7b-9336-c443185afd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:31.118241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867173b3-0719-4a7b-9336-c443185afd9d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:31.118614 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867173b3-0719-4a7b-9336-c443185afd9d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:31.118900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867173b3-0719-4a7b-9336-c443185afd9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:31.127910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249256d-528b-46a3-a7d4-8b5e54d14b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:31.128616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249256d-528b-46a3-a7d4-8b5e54d14b66 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:31.129043 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249256d-528b-46a3-a7d4-8b5e54d14b66 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:31.129325 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249256d-528b-46a3-a7d4-8b5e54d14b66 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:31.130125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249256d-528b-46a3-a7d4-8b5e54d14b66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:31.130526 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249256d-528b-46a3-a7d4-8b5e54d14b66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:31.131989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2249256d-528b-46a3-a7d4-8b5e54d14b66 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:31.147638 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 364/727] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:31.148674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867173b3-0719-4a7b-9336-c443185afd9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:31.152981 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249256d-528b-46a3-a7d4-8b5e54d14b66 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-867173b3-0719-4a7b-9336-c443185afd9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:31.157575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:31.160688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:31.164547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 364/728] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:31.167014 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867173b3-0719-4a7b-9336-c443185afd9d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:31.312918 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:31.327135 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249256d-528b-46a3-a7d4-8b5e54d14b66 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:31.369328 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867173b3-0719-4a7b-9336-c443185afd9d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:31.371695 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:31.394548 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495333-334c-49d3-8e9b-6a379d91ccf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:31.395375 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495333-334c-49d3-8e9b-6a379d91ccf7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:31.395661 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495333-334c-49d3-8e9b-6a379d91ccf7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:31.395938 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495333-334c-49d3-8e9b-6a379d91ccf7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:31.413393 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd495333-334c-49d3-8e9b-6a379d91ccf7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:31.414873 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd495333-334c-49d3-8e9b-6a379d91ccf7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-cd495333-334c-49d3-8e9b-6a379d91ccf7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:31.437562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:31.439372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:31.441224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:31.441224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:31.441224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:31.441224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:31.441224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:31.441224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:31.441224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:31.441224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:31.441224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:31.441224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:31.446776 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 365/729] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:31.447719 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcaf14-5890-4dca-b92a-de006fb5b9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:31.448481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcaf14-5890-4dca-b92a-de006fb5b9e5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:31.448775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcaf14-5890-4dca-b92a-de006fb5b9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:31.449060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcaf14-5890-4dca-b92a-de006fb5b9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:31.449829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcaf14-5890-4dca-b92a-de006fb5b9e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:31.450149 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcaf14-5890-4dca-b92a-de006fb5b9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:31.452679 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495333-334c-49d3-8e9b-6a379d91ccf7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:31.453164 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcaf14-5890-4dca-b92a-de006fb5b9e5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:31.473024 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 365/730] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:31.477835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcaf14-5890-4dca-b92a-de006fb5b9e5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:31.751743 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495333-334c-49d3-8e9b-6a379d91ccf7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:31.784621 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcaf14-5890-4dca-b92a-de006fb5b9e5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:31.838201 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea59806-c8f5-4e0d-9765-2e11e3580f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:31.838903 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea59806-c8f5-4e0d-9765-2e11e3580f69 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:31.840370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea59806-c8f5-4e0d-9765-2e11e3580f69 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:31.840370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea59806-c8f5-4e0d-9765-2e11e3580f69 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:31.840601 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea59806-c8f5-4e0d-9765-2e11e3580f69 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:31.840837 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea59806-c8f5-4e0d-9765-2e11e3580f69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:31.842300 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea59806-c8f5-4e0d-9765-2e11e3580f69 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:31.849967 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6f2e3-4b2a-4b67-98db-12bdc89b8fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:31.850776 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6f2e3-4b2a-4b67-98db-12bdc89b8fa3 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:31.851063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6f2e3-4b2a-4b67-98db-12bdc89b8fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:31.852459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6f2e3-4b2a-4b67-98db-12bdc89b8fa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:31.863402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6f2e3-4b2a-4b67-98db-12bdc89b8fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:31.863637 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 366/731] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:50:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:31.864833 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b6f2e3-4b2a-4b67-98db-12bdc89b8fa3 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:31.864988 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8668dd-40ab-4cfb-b34e-0b8e23a35231 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:31.865398 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8668dd-40ab-4cfb-b34e-0b8e23a35231 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:31.865690 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8668dd-40ab-4cfb-b34e-0b8e23a35231 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:31.867348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 367/732] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:31.872865 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8668dd-40ab-4cfb-b34e-0b8e23a35231 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:31.882869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-c5b6f2e3-4b2a-4b67-98db-12bdc89b8fa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:31.917283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:31.918496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:31.919518 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:31.919518 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:31.919518 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:31.919518 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:31.927871 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 366/733] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:31.929546 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40490-eb5d-45c3-aafc-21c485b4ff06 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:31.933084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40490-eb5d-45c3-aafc-21c485b4ff06 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:31.934118 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a40490-eb5d-45c3-aafc-21c485b4ff06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:31.970699 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-29a40490-eb5d-45c3-aafc-21c485b4ff06 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:31.972762 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-29a40490-eb5d-45c3-aafc-21c485b4ff06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:31.986167 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:32.028134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8668dd-40ab-4cfb-b34e-0b8e23a35231 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:32.063715 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682cfcc-264d-4551-be6d-770059726223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:32.064458 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682cfcc-264d-4551-be6d-770059726223 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.064748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682cfcc-264d-4551-be6d-770059726223 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.065024 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682cfcc-264d-4551-be6d-770059726223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.076838 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 367/734] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:32.078120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a40490-eb5d-45c3-aafc-21c485b4ff06 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:32.156124 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682cfcc-264d-4551-be6d-770059726223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:32.157668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682cfcc-264d-4551-be6d-770059726223 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:32.165915 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:32.189036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a40490-eb5d-45c3-aafc-21c485b4ff06 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:32.224785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865637e-539d-4214-84a9-3a05bdc81366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:32.225458 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865637e-539d-4214-84a9-3a05bdc81366 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.225748 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865637e-539d-4214-84a9-3a05bdc81366 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.226028 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865637e-539d-4214-84a9-3a05bdc81366 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.226755 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865637e-539d-4214-84a9-3a05bdc81366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:32.227062 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865637e-539d-4214-84a9-3a05bdc81366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:32.228549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865637e-539d-4214-84a9-3a05bdc81366 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:32.244303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 368/735] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:32.244303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb4273-0c4d-4f91-ad2a-454871796a86 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.244303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb4273-0c4d-4f91-ad2a-454871796a86 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.244303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb4273-0c4d-4f91-ad2a-454871796a86 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.244303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 369/736] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:32.244303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb4273-0c4d-4f91-ad2a-454871796a86 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:32.257920 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:32.270706 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 368/737] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:32.271680 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682cfcc-264d-4551-be6d-770059726223 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:32.367521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb4273-0c4d-4f91-ad2a-454871796a86 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:32.382533 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682cfcc-264d-4551-be6d-770059726223 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:32.390442 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:32.408014 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea224b-7fb6-42db-a704-3a3217e58dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:32.408723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea224b-7fb6-42db-a704-3a3217e58dff None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.409012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea224b-7fb6-42db-a704-3a3217e58dff None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.409288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea224b-7fb6-42db-a704-3a3217e58dff None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.410025 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea224b-7fb6-42db-a704-3a3217e58dff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:32.411420 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea224b-7fb6-42db-a704-3a3217e58dff None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:32.420253 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab991a67-6522-4b4d-a8b4-a1df0c43f35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:32.421193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab991a67-6522-4b4d-a8b4-a1df0c43f35f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.421479 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab991a67-6522-4b4d-a8b4-a1df0c43f35f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.421759 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab991a67-6522-4b4d-a8b4-a1df0c43f35f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.429279 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab991a67-6522-4b4d-a8b4-a1df0c43f35f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:32.434478 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab991a67-6522-4b4d-a8b4-a1df0c43f35f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-ab991a67-6522-4b4d-a8b4-a1df0c43f35f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:32.446428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:32.447691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 369/738] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:50:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39edd67-1c96-4721-9e2f-2de746c581f8 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.449122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39edd67-1c96-4721-9e2f-2de746c581f8 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.449980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39edd67-1c96-4721-9e2f-2de746c581f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.466759 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-b39edd67-1c96-4721-9e2f-2de746c581f8 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:32.467953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-b39edd67-1c96-4721-9e2f-2de746c581f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:32.527751 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 370/739] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:50:32 2022] POST /identity/v3/services => generated 251 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:32.528631 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea224b-7fb6-42db-a704-3a3217e58dff None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:32.560525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 370/740] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:32.561905 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39edd67-1c96-4721-9e2f-2de746c581f8 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:32.608461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:32.649203 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea224b-7fb6-42db-a704-3a3217e58dff None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:32.676717 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:32.685918 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39edd67-1c96-4721-9e2f-2de746c581f8 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:32.694357 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bf926-be9d-45df-82ad-831634496851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:32.694741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bf926-be9d-45df-82ad-831634496851 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.694979 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bf926-be9d-45df-82ad-831634496851 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.695261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bf926-be9d-45df-82ad-831634496851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.707847 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bf926-be9d-45df-82ad-831634496851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:32.709422 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bf926-be9d-45df-82ad-831634496851 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:32.726506 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9e2c8-7183-409b-9cea-769c51fffe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:32.727195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9e2c8-7183-409b-9cea-769c51fffe26 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.727479 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9e2c8-7183-409b-9cea-769c51fffe26 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.727790 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9e2c8-7183-409b-9cea-769c51fffe26 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.728646 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9e2c8-7183-409b-9cea-769c51fffe26 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-3f4bf926-be9d-45df-82ad-831634496851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:32.728817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:32.729859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:32.730888 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:32.730888 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:32.730888 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:32.730888 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:32.731142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9e2c8-7183-409b-9cea-769c51fffe26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:32.731225 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 371/741] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:32.731306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9e2c8-7183-409b-9cea-769c51fffe26 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:32.731393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c8322-063f-4e4e-bbf5-4e036388ab6a None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.731763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c8322-063f-4e4e-bbf5-4e036388ab6a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.731872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c8322-063f-4e4e-bbf5-4e036388ab6a None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.733165 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 372/742] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:32.733819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c8322-063f-4e4e-bbf5-4e036388ab6a None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:32.752485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 371/743] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:50:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:32.754066 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9e2c8-7183-409b-9cea-769c51fffe26 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:32.859960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c8322-063f-4e4e-bbf5-4e036388ab6a None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:32.864474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9e2c8-7183-409b-9cea-769c51fffe26 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:32.878535 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:32.901373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e633d33-18c4-4753-8faf-a1e934f19519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:32.902097 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e633d33-18c4-4753-8faf-a1e934f19519 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.902379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e633d33-18c4-4753-8faf-a1e934f19519 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.902656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e633d33-18c4-4753-8faf-a1e934f19519 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.907519 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e2050-0bce-4cca-9e06-8cb618c1e6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:32.908195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e2050-0bce-4cca-9e06-8cb618c1e6be None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.908532 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e2050-0bce-4cca-9e06-8cb618c1e6be None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.908797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e2050-0bce-4cca-9e06-8cb618c1e6be None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.909539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e2050-0bce-4cca-9e06-8cb618c1e6be None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:32.909881 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e2050-0bce-4cca-9e06-8cb618c1e6be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:32.911452 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e2050-0bce-4cca-9e06-8cb618c1e6be None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:32.914659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e633d33-18c4-4753-8faf-a1e934f19519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-3e633d33-18c4-4753-8faf-a1e934f19519 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:32.922564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:32.923656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:32.926013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 373/744] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:32.927189 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922644b-2565-45aa-84f0-031be23b9eec None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:32.927505 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922644b-2565-45aa-84f0-031be23b9eec None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:32.927930 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922644b-2565-45aa-84f0-031be23b9eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:32.932684 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 372/745] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:32.933925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e2050-0bce-4cca-9e06-8cb618c1e6be None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:32.950164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:32.950499 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-9922644b-2565-45aa-84f0-031be23b9eec None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:32.951702 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-9922644b-2565-45aa-84f0-031be23b9eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:33.026780 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 374/746] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:32 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 19 20:50:33.027656 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922644b-2565-45aa-84f0-031be23b9eec None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:33.059174 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e2050-0bce-4cca-9e06-8cb618c1e6be None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:33.099964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cff82-7490-4fb6-90e6-a1a8fcbe6aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:33.100660 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cff82-7490-4fb6-90e6-a1a8fcbe6aca None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:33.100947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cff82-7490-4fb6-90e6-a1a8fcbe6aca None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:33.101223 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cff82-7490-4fb6-90e6-a1a8fcbe6aca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:33.148187 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922644b-2565-45aa-84f0-031be23b9eec None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:33.159113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:33.174768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cff82-7490-4fb6-90e6-a1a8fcbe6aca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:33.176447 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cff82-7490-4fb6-90e6-a1a8fcbe6aca None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:33.186659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3a11d-4ca0-465f-bf72-c464027b1236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:33.187492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3a11d-4ca0-465f-bf72-c464027b1236 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:33.187781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3a11d-4ca0-465f-bf72-c464027b1236 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:33.188078 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3a11d-4ca0-465f-bf72-c464027b1236 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:33.189633 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:33.196116 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3a11d-4ca0-465f-bf72-c464027b1236 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:33.197545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3a11d-4ca0-465f-bf72-c464027b1236 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-fdd3a11d-4ca0-465f-bf72-c464027b1236 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:33.212363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:33.213844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:33.214912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:33.214912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:33.214912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:33.214912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:33.214912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:33.214912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:33.214912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:33.214912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:33.214912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 375/747] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:33.214912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3a11d-4ca0-465f-bf72-c464027b1236 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:33.292426 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 373/748] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:32 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 19 20:50:33.293624 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cff82-7490-4fb6-90e6-a1a8fcbe6aca None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:33.329570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3a11d-4ca0-465f-bf72-c464027b1236 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:33.365624 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fc1bf-22ed-406f-93df-168836661f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:33.366449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fc1bf-22ed-406f-93df-168836661f5c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:33.366801 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fc1bf-22ed-406f-93df-168836661f5c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:33.367167 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fc1bf-22ed-406f-93df-168836661f5c None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:33.367940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fc1bf-22ed-406f-93df-168836661f5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:33.368252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fc1bf-22ed-406f-93df-168836661f5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:33.369700 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fc1bf-22ed-406f-93df-168836661f5c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:33.383590 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 376/749] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:33.383590 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fc1bf-22ed-406f-93df-168836661f5c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:33.406866 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cff82-7490-4fb6-90e6-a1a8fcbe6aca None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:33.416795 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:33.419910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:33.444460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39eb20-767e-4d6b-926c-c1d46056b988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:33.445223 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39eb20-767e-4d6b-926c-c1d46056b988 None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:33.445577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39eb20-767e-4d6b-926c-c1d46056b988 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:33.445919 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39eb20-767e-4d6b-926c-c1d46056b988 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:33.446706 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39eb20-767e-4d6b-926c-c1d46056b988 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:33.448232 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39eb20-767e-4d6b-926c-c1d46056b988 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:33.492883 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fc1bf-22ed-406f-93df-168836661f5c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:33.527230 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4fc90-ab84-43fc-af1b-610daf51ce07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:33.527889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4fc90-ab84-43fc-af1b-610daf51ce07 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:33.528180 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4fc90-ab84-43fc-af1b-610daf51ce07 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:33.528485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4fc90-ab84-43fc-af1b-610daf51ce07 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:33.529275 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4fc90-ab84-43fc-af1b-610daf51ce07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:33.529584 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4fc90-ab84-43fc-af1b-610daf51ce07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:33.531070 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4fc90-ab84-43fc-af1b-610daf51ce07 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:33.534377 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 374/750] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:50:33 2022] POST /identity/v3/services => generated 240 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:33.535383 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39eb20-767e-4d6b-926c-c1d46056b988 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:33.552944 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 377/751] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:33.554017 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4fc90-ab84-43fc-af1b-610daf51ce07 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:33.680509 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4fc90-ab84-43fc-af1b-610daf51ce07 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:33.681989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39eb20-767e-4d6b-926c-c1d46056b988 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:33.684694 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:33.728152 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec959-80ed-44a8-ab60-122933f0aa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:33.729120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec959-80ed-44a8-ab60-122933f0aa81 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:33.729476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec959-80ed-44a8-ab60-122933f0aa81 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:33.729622 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c31866-3fde-43be-bc15-47d8a53409b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:33.729874 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec959-80ed-44a8-ab60-122933f0aa81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:33.730828 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c31866-3fde-43be-bc15-47d8a53409b7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:33.731317 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c31866-3fde-43be-bc15-47d8a53409b7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:33.731797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c31866-3fde-43be-bc15-47d8a53409b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:33.743928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c31866-3fde-43be-bc15-47d8a53409b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:33.744156 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ec959-80ed-44a8-ab60-122933f0aa81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:33.744773 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c31866-3fde-43be-bc15-47d8a53409b7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:33.745831 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ec959-80ed-44a8-ab60-122933f0aa81 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-90c31866-3fde-43be-bc15-47d8a53409b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:33.768256 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:33.770100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:33.771647 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:33.771647 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:33.771647 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:33.771647 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-b55ec959-80ed-44a8-ab60-122933f0aa81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:33.771913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:33.772942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:33.774018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:33.774018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:33.774018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:33.774018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:33.774313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 375/752] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:33.774402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 378/753] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:33.774493 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c31866-3fde-43be-bc15-47d8a53409b7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:33.779408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec959-80ed-44a8-ab60-122933f0aa81 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:33.796010 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:33.890761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c31866-3fde-43be-bc15-47d8a53409b7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:33.921994 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec959-80ed-44a8-ab60-122933f0aa81 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:33.945798 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76ead7-adb6-4144-9ed3-702f89200c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:33.946392 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76ead7-adb6-4144-9ed3-702f89200c92 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:33.946744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76ead7-adb6-4144-9ed3-702f89200c92 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:33.947033 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d76ead7-adb6-4144-9ed3-702f89200c92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:33.947933 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76ead7-adb6-4144-9ed3-702f89200c92 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=ba6e1d31d8004f9fa956b78d8716d6e6) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:33.948257 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76ead7-adb6-4144-9ed3-702f89200c92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:33.949735 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d76ead7-adb6-4144-9ed3-702f89200c92 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:33.960559 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 376/754] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:33 2022] GET /identity/v3/role_assignments?user.id=ba6e1d31d8004f9fa956b78d8716d6e6&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:33.961484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76ead7-adb6-4144-9ed3-702f89200c92 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:33.967000 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3803bfb-d536-4959-b4dc-bb4731de7777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:33.967955 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3803bfb-d536-4959-b4dc-bb4731de7777 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:33.968475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3803bfb-d536-4959-b4dc-bb4731de7777 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:33.968973 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3803bfb-d536-4959-b4dc-bb4731de7777 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:33.970519 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3803bfb-d536-4959-b4dc-bb4731de7777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:33.971037 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3803bfb-d536-4959-b4dc-bb4731de7777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:33.972822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3803bfb-d536-4959-b4dc-bb4731de7777 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:33.986843 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 379/755] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:33.994165 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3803bfb-d536-4959-b4dc-bb4731de7777 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:34.010711 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:34.158620 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:34.191210 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3803bfb-d536-4959-b4dc-bb4731de7777 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:34.201936 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d76ead7-adb6-4144-9ed3-702f89200c92 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:34.258208 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1a5db-1be5-499b-85ab-252e5a68209b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:34.259580 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1a5db-1be5-499b-85ab-252e5a68209b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:34.260093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1a5db-1be5-499b-85ab-252e5a68209b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:34.260696 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1a5db-1be5-499b-85ab-252e5a68209b None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:34.273474 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1a5db-1be5-499b-85ab-252e5a68209b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:34.276123 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1a5db-1be5-499b-85ab-252e5a68209b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-f6f1a5db-1be5-499b-85ab-252e5a68209b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:34.294679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:34.295806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:34.296927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:34.296927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:34.296927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:34.296927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:34.296927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:34.296927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:34.296927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:34.296927 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:34.300015 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 380/756] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:34.305345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1a5db-1be5-499b-85ab-252e5a68209b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:34.315092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc0ac6-a876-4da0-a4db-e4d4c1c4dba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:34.315788 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc0ac6-a876-4da0-a4db-e4d4c1c4dba5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:34.316079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc0ac6-a876-4da0-a4db-e4d4c1c4dba5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:34.316381 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc0ac6-a876-4da0-a4db-e4d4c1c4dba5 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:34.317129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc0ac6-a876-4da0-a4db-e4d4c1c4dba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:34.317443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc0ac6-a876-4da0-a4db-e4d4c1c4dba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:34.318832 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc0ac6-a876-4da0-a4db-e4d4c1c4dba5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:34.352693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 377/757] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:34.354545 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc0ac6-a876-4da0-a4db-e4d4c1c4dba5 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:34.421722 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:34.511906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1a5db-1be5-499b-85ab-252e5a68209b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:34.569804 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc0ac6-a876-4da0-a4db-e4d4c1c4dba5 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:34.593589 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbccf727-bd9d-4cdd-a706-feca419b555c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:34.594424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccf727-bd9d-4cdd-a706-feca419b555c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:34.594720 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccf727-bd9d-4cdd-a706-feca419b555c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:34.595003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccf727-bd9d-4cdd-a706-feca419b555c None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:34.595810 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbccf727-bd9d-4cdd-a706-feca419b555c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:34.596123 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbccf727-bd9d-4cdd-a706-feca419b555c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:34.599166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbccf727-bd9d-4cdd-a706-feca419b555c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:34.618462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 381/758] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:34.624883 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbccf727-bd9d-4cdd-a706-feca419b555c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:34.646925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1f44b-ca6d-42fa-aed5-e6db895d6fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:34.647791 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1f44b-ca6d-42fa-aed5-e6db895d6fc3 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:34.648081 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1f44b-ca6d-42fa-aed5-e6db895d6fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:34.648377 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1f44b-ca6d-42fa-aed5-e6db895d6fc3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:34.661660 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1f44b-ca6d-42fa-aed5-e6db895d6fc3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:34.663157 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1f44b-ca6d-42fa-aed5-e6db895d6fc3 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-51d1f44b-ca6d-42fa-aed5-e6db895d6fc3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:34.693464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:34.694876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:34.696031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:34.696031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:34.696031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:34.696031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:34.696031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:34.696031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:34.696031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:34.696031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:34.696031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:34.696031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:34.700842 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 378/759] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:50:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:34.706442 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1f44b-ca6d-42fa-aed5-e6db895d6fc3 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:34.715089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:34.759176 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbccf727-bd9d-4cdd-a706-feca419b555c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:34.783497 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:34.801027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8b096-a24f-4292-8dc6-4916fe7617c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:34.801699 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8b096-a24f-4292-8dc6-4916fe7617c9 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:34.801987 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8b096-a24f-4292-8dc6-4916fe7617c9 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:34.802270 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8b096-a24f-4292-8dc6-4916fe7617c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:34.803120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8b096-a24f-4292-8dc6-4916fe7617c9 None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=e44797dd45ca43bcbcbb8ac732423782) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:34.803436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8b096-a24f-4292-8dc6-4916fe7617c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:34.804948 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8b096-a24f-4292-8dc6-4916fe7617c9 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:34.818184 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 382/760] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:34 2022] GET /identity/v3/role_assignments?user.id=e44797dd45ca43bcbcbb8ac732423782&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:34.908847 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97abdcf-de33-4ef4-897a-f23dd1efafab None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:34.909227 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97abdcf-de33-4ef4-897a-f23dd1efafab None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:34.909429 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97abdcf-de33-4ef4-897a-f23dd1efafab None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:34.911162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 383/761] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:34.920194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521e6bc-adef-4aa2-bb0b-b1670f8bdfda None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:34.920916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521e6bc-adef-4aa2-bb0b-b1670f8bdfda None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:34.921435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521e6bc-adef-4aa2-bb0b-b1670f8bdfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:34.928812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1f44b-ca6d-42fa-aed5-e6db895d6fc3 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:34.944112 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-9521e6bc-adef-4aa2-bb0b-b1670f8bdfda None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:34.945832 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-9521e6bc-adef-4aa2-bb0b-b1670f8bdfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:34.998485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e02fee-3437-42fc-b45a-e2e8a4cf285c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:34.999624 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02fee-3437-42fc-b45a-e2e8a4cf285c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:35.000137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02fee-3437-42fc-b45a-e2e8a4cf285c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:35.000716 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e02fee-3437-42fc-b45a-e2e8a4cf285c None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:35.001830 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e02fee-3437-42fc-b45a-e2e8a4cf285c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:35.002381 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e02fee-3437-42fc-b45a-e2e8a4cf285c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:35.004431 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e02fee-3437-42fc-b45a-e2e8a4cf285c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:35.025638 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 379/762] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:50:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:35.033944 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e02fee-3437-42fc-b45a-e2e8a4cf285c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:35.102284 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 384/763] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:35.132670 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521e6bc-adef-4aa2-bb0b-b1670f8bdfda None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:35.145640 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:35.182791 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e02fee-3437-42fc-b45a-e2e8a4cf285c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:35.221578 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba1336-fc27-4906-855b-dd62b1516cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:35.222891 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba1336-fc27-4906-855b-dd62b1516cd6 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:35.223549 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba1336-fc27-4906-855b-dd62b1516cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:35.224044 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba1336-fc27-4906-855b-dd62b1516cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:35.236915 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba1336-fc27-4906-855b-dd62b1516cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:35.239024 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba1336-fc27-4906-855b-dd62b1516cd6 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-c5ba1336-fc27-4906-855b-dd62b1516cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:35.258985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:35.260142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:35.261268 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:35.261268 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:35.261268 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:35.261268 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:35.264113 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 380/764] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:35.274939 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba1336-fc27-4906-855b-dd62b1516cd6 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:35.355336 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521e6bc-adef-4aa2-bb0b-b1670f8bdfda None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:35.383911 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:35.424099 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ac18d-8924-42dc-ac70-02f2849482f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:35.424966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ac18d-8924-42dc-ac70-02f2849482f8 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:35.425253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ac18d-8924-42dc-ac70-02f2849482f8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:35.425533 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ac18d-8924-42dc-ac70-02f2849482f8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:35.426202 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ac18d-8924-42dc-ac70-02f2849482f8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:35.427759 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ac18d-8924-42dc-ac70-02f2849482f8 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:35.442351 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba1336-fc27-4906-855b-dd62b1516cd6 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-966ac18d-8924-42dc-ac70-02f2849482f8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:35.453034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:35.454277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:35.455314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 385/765] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 20:50:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:35.463639 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ac18d-8924-42dc-ac70-02f2849482f8 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:35.472675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:35.507036 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8f9c6-ec8e-41a1-afd3-e29f068e590c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:35.507804 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8f9c6-ec8e-41a1-afd3-e29f068e590c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:35.508100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8f9c6-ec8e-41a1-afd3-e29f068e590c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:35.508404 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8f9c6-ec8e-41a1-afd3-e29f068e590c None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:35.509156 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8f9c6-ec8e-41a1-afd3-e29f068e590c None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:35.509471 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8f9c6-ec8e-41a1-afd3-e29f068e590c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:35.510875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8f9c6-ec8e-41a1-afd3-e29f068e590c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:35.533482 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 381/766] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:35.539322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8f9c6-ec8e-41a1-afd3-e29f068e590c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:35.684278 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ac18d-8924-42dc-ac70-02f2849482f8 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:35.698963 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8f9c6-ec8e-41a1-afd3-e29f068e590c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:35.736051 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6f5b6-45b7-4224-b21c-acf710b16246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:35.736763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6f5b6-45b7-4224-b21c-acf710b16246 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:35.737056 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6f5b6-45b7-4224-b21c-acf710b16246 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:35.737339 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6f5b6-45b7-4224-b21c-acf710b16246 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:35.738060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6f5b6-45b7-4224-b21c-acf710b16246 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:35.738374 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6f5b6-45b7-4224-b21c-acf710b16246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:35.739788 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6f5b6-45b7-4224-b21c-acf710b16246 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:35.745359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 386/767] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:50:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:35.751785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6f5b6-45b7-4224-b21c-acf710b16246 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:35.813290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:35.825067 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c9680-739e-4c57-901f-b42b2ee4ea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:35.825896 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c9680-739e-4c57-901f-b42b2ee4ea60 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:35.826194 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c9680-739e-4c57-901f-b42b2ee4ea60 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:35.826480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c9680-739e-4c57-901f-b42b2ee4ea60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:35.857233 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c9680-739e-4c57-901f-b42b2ee4ea60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:35.858720 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c9680-739e-4c57-901f-b42b2ee4ea60 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:35.865200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6f5b6-45b7-4224-b21c-acf710b16246 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-be9c9680-739e-4c57-901f-b42b2ee4ea60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:35.896928 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:35.898249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:35.899322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:35.899322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:35.899322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:35.899322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:35.899322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:35.899322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:35.899322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:35.899322 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:35.904915 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 382/768] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:35.909163 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97610e-2abe-4bde-9efc-82286725eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:35.909855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97610e-2abe-4bde-9efc-82286725eb10 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:35.910148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97610e-2abe-4bde-9efc-82286725eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:35.910487 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97610e-2abe-4bde-9efc-82286725eb10 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:35.911225 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97610e-2abe-4bde-9efc-82286725eb10 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:35.911421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c9680-739e-4c57-901f-b42b2ee4ea60 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:35.911593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97610e-2abe-4bde-9efc-82286725eb10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:35.913058 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97610e-2abe-4bde-9efc-82286725eb10 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:35.919132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 387/769] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:50:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:35.926300 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97610e-2abe-4bde-9efc-82286725eb10 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:36.038547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97610e-2abe-4bde-9efc-82286725eb10 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:36.041722 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:36.056770 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:36.082779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea97b13-2937-45cd-8068-0fba0971769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:36.083531 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea97b13-2937-45cd-8068-0fba0971769b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:36.084018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea97b13-2937-45cd-8068-0fba0971769b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:36.084312 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea97b13-2937-45cd-8068-0fba0971769b None admin] PATH_INFO: `/v3/endpoints` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:36.085182 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea97b13-2937-45cd-8068-0fba0971769b None admin] RBAC: Adding query filter params (interface=public, service_id=c315138ac3744e0a89ec04ea3deff914, region_id=RegionOne) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:36.085505 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea97b13-2937-45cd-8068-0fba0971769b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:36.086965 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea97b13-2937-45cd-8068-0fba0971769b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:36.097528 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 388/770] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:50:35 2022] GET /identity/v3/endpoints?service_id=c315138ac3744e0a89ec04ea3deff914&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:36.104001 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea97b13-2937-45cd-8068-0fba0971769b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:36.224044 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea97b13-2937-45cd-8068-0fba0971769b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:36.230743 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c9680-739e-4c57-901f-b42b2ee4ea60 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:36.260039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd08c6b-4ab4-4e18-bbad-1acce050e256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:36.260773 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08c6b-4ab4-4e18-bbad-1acce050e256 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:36.261067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08c6b-4ab4-4e18-bbad-1acce050e256 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:36.261351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd08c6b-4ab4-4e18-bbad-1acce050e256 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:36.262062 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd08c6b-4ab4-4e18-bbad-1acce050e256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:36.263509 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd08c6b-4ab4-4e18-bbad-1acce050e256 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:36.270622 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 389/771] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:50:36 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:36.324103 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c40831-0829-41d7-8052-dabcde5f20ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:36.324743 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c40831-0829-41d7-8052-dabcde5f20ca None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:36.325040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c40831-0829-41d7-8052-dabcde5f20ca None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:36.325320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c40831-0829-41d7-8052-dabcde5f20ca None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:36.326155 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c40831-0829-41d7-8052-dabcde5f20ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:36.326473 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c40831-0829-41d7-8052-dabcde5f20ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:36.328001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c40831-0829-41d7-8052-dabcde5f20ca None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:36.353391 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 383/772] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:36.359613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd08c6b-4ab4-4e18-bbad-1acce050e256 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:36.389073 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3398e1-85cd-4a9b-af8a-523a10a8ea92 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:36.390189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3398e1-85cd-4a9b-af8a-523a10a8ea92 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:36.390189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3398e1-85cd-4a9b-af8a-523a10a8ea92 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:36.391986 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 384/773] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:36.397548 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851969-7ad9-4087-992d-e43b2dcd27fa None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:36.397845 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851969-7ad9-4087-992d-e43b2dcd27fa None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:36.398137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b851969-7ad9-4087-992d-e43b2dcd27fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:36.425084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:36.430651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-5b851969-7ad9-4087-992d-e43b2dcd27fa None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:36.433430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-5b851969-7ad9-4087-992d-e43b2dcd27fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:36.490684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd08c6b-4ab4-4e18-bbad-1acce050e256 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:36.531049 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966916a-7628-4fa5-8c95-9d2dbe2e735a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:36.532103 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966916a-7628-4fa5-8c95-9d2dbe2e735a None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:36.532443 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966916a-7628-4fa5-8c95-9d2dbe2e735a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:36.532738 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966916a-7628-4fa5-8c95-9d2dbe2e735a None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/77427e1028d349b4a740efe0a985e73b/roles/ed68c2195de147c0a30d9150b33e47cf` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:36.534831 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966916a-7628-4fa5-8c95-9d2dbe2e735a None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=77427e1028d349b4a740efe0a985e73b, role_id=ed68c2195de147c0a30d9150b33e47cf)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:36.536462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966916a-7628-4fa5-8c95-9d2dbe2e735a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:36.554973 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 385/774] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:36.556185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8bb60-b097-4284-a73b-1d855b5dca55 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:36.556517 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8bb60-b097-4284-a73b-1d855b5dca55 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:36.556795 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8bb60-b097-4284-a73b-1d855b5dca55 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:36.558218 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 386/775] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:36.564706 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579f0c2-aaef-4939-96ec-44cf1519e94b None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:36.565000 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579f0c2-aaef-4939-96ec-44cf1519e94b None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:36.565283 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579f0c2-aaef-4939-96ec-44cf1519e94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:36.584577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-5579f0c2-aaef-4939-96ec-44cf1519e94b None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:36.585692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-5579f0c2-aaef-4939-96ec-44cf1519e94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:36.604287 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:36.626499 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 390/776] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:50:36 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/77427e1028d349b4a740efe0a985e73b/roles/ed68c2195de147c0a30d9150b33e47cf => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:50:36.632620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966916a-7628-4fa5-8c95-9d2dbe2e735a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:36.710860 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 387/777] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:36.745599 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579f0c2-aaef-4939-96ec-44cf1519e94b None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:36.837675 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:36.849500 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966916a-7628-4fa5-8c95-9d2dbe2e735a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:36.913736 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d42478-60f9-42c7-856f-36e8cc7f4cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:36.915393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d42478-60f9-42c7-856f-36e8cc7f4cf8 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:36.916229 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d42478-60f9-42c7-856f-36e8cc7f4cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:36.916882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d42478-60f9-42c7-856f-36e8cc7f4cf8 None admin] PATH_INFO: `/v3/services/network` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:36.918028 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d42478-60f9-42c7-856f-36e8cc7f4cf8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:36.920360 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d42478-60f9-42c7-856f-36e8cc7f4cf8 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:36.931086 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579f0c2-aaef-4939-96ec-44cf1519e94b None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-50d42478-60f9-42c7-856f-36e8cc7f4cf8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:36.946573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:36.947850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:36.948896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 391/778] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:36 2022] GET /identity/v3/services/network => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:36.955007 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d42478-60f9-42c7-856f-36e8cc7f4cf8 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:36.987210 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c463b27-8e36-409b-9cad-ce0ee22e219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:36.989121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c463b27-8e36-409b-9cad-ce0ee22e219a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:36.989582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c463b27-8e36-409b-9cad-ce0ee22e219a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:36.990325 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c463b27-8e36-409b-9cad-ce0ee22e219a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:37.007070 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c463b27-8e36-409b-9cad-ce0ee22e219a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-9c463b27-8e36-409b-9cad-ce0ee22e219a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:37.018564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:37.020023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:37.021159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:37.022359 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 388/779] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:37.028742 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c463b27-8e36-409b-9cad-ce0ee22e219a None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:37.064024 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:37.122712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d42478-60f9-42c7-856f-36e8cc7f4cf8 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:37.160507 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86f0f4-24ae-42a2-8b31-acdb3df9f4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:37.161180 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86f0f4-24ae-42a2-8b31-acdb3df9f4b0 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:37.161466 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86f0f4-24ae-42a2-8b31-acdb3df9f4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:37.161739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86f0f4-24ae-42a2-8b31-acdb3df9f4b0 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:37.162453 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86f0f4-24ae-42a2-8b31-acdb3df9f4b0 None admin] RBAC: Adding query filter params (name=network) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:37.162760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86f0f4-24ae-42a2-8b31-acdb3df9f4b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:37.164164 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86f0f4-24ae-42a2-8b31-acdb3df9f4b0 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:37.170426 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 392/780] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:50:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:37.178692 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86f0f4-24ae-42a2-8b31-acdb3df9f4b0 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:37.235853 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:37.263172 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c463b27-8e36-409b-9cad-ce0ee22e219a None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:37.319093 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafa8a8-e502-4ddf-8f68-2779b5f4f1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:37.319777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafa8a8-e502-4ddf-8f68-2779b5f4f1c1 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:37.320066 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafa8a8-e502-4ddf-8f68-2779b5f4f1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:37.320363 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffafa8a8-e502-4ddf-8f68-2779b5f4f1c1 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:37.321125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafa8a8-e502-4ddf-8f68-2779b5f4f1c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:37.321439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafa8a8-e502-4ddf-8f68-2779b5f4f1c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:37.322814 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffafa8a8-e502-4ddf-8f68-2779b5f4f1c1 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:37.334872 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 389/781] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:37.341464 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafa8a8-e502-4ddf-8f68-2779b5f4f1c1 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:37.425752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86f0f4-24ae-42a2-8b31-acdb3df9f4b0 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:37.490520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:37.501715 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa0c8b-a9f1-4098-aa4b-90854f7e4438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:37.509805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa0c8b-a9f1-4098-aa4b-90854f7e4438 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:37.509805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa0c8b-a9f1-4098-aa4b-90854f7e4438 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:37.509805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa0c8b-a9f1-4098-aa4b-90854f7e4438 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:37.509805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa0c8b-a9f1-4098-aa4b-90854f7e4438 None admin] RBAC: Adding query filter params (type=network) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:37.509805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa0c8b-a9f1-4098-aa4b-90854f7e4438 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:37.509805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa0c8b-a9f1-4098-aa4b-90854f7e4438 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:37.515330 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 393/782] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:50:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:37.522893 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa0c8b-a9f1-4098-aa4b-90854f7e4438 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:37.523729 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffafa8a8-e502-4ddf-8f68-2779b5f4f1c1 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:37.603973 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81e732-3467-48bd-95f0-f7371bdb2261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:37.604813 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81e732-3467-48bd-95f0-f7371bdb2261 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:37.605201 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81e732-3467-48bd-95f0-f7371bdb2261 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:37.605491 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81e732-3467-48bd-95f0-f7371bdb2261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:37.634292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81e732-3467-48bd-95f0-f7371bdb2261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:37.635893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e81e732-3467-48bd-95f0-f7371bdb2261 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-3e81e732-3467-48bd-95f0-f7371bdb2261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:37.665292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:37.666659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:37.668280 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:37.668280 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:37.668280 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:37.668280 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:37.668582 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:37.668669 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 390/783] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:37.671687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056183c9-a210-4ffa-9474-1879ff4301e8 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:37.671980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056183c9-a210-4ffa-9474-1879ff4301e8 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:37.673165 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056183c9-a210-4ffa-9474-1879ff4301e8 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:37.680952 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 391/784] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:37.680952 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056183c9-a210-4ffa-9474-1879ff4301e8 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:37.689117 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa0c8b-a9f1-4098-aa4b-90854f7e4438 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:37.738383 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d473b277-7916-44db-9f6a-b7aa11e06c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:37.739724 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473b277-7916-44db-9f6a-b7aa11e06c14 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:37.740403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473b277-7916-44db-9f6a-b7aa11e06c14 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:37.740786 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d473b277-7916-44db-9f6a-b7aa11e06c14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:37.741957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473b277-7916-44db-9f6a-b7aa11e06c14 None admin] RBAC: Adding query filter params (interface=public, service_id=b94df6bb2f6241fdb231d3d88aa8c742, region_id=RegionOne) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:37.742750 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473b277-7916-44db-9f6a-b7aa11e06c14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:37.744265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d473b277-7916-44db-9f6a-b7aa11e06c14 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:37.752877 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 394/785] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:50:37 2022] GET /identity/v3/endpoints?service_id=b94df6bb2f6241fdb231d3d88aa8c742&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:37.754272 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce61e8-02b3-4f84-84a7-89eeea7b95db None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:37.754683 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce61e8-02b3-4f84-84a7-89eeea7b95db None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:37.755059 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce61e8-02b3-4f84-84a7-89eeea7b95db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:37.777086 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-2bce61e8-02b3-4f84-84a7-89eeea7b95db None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:37.778881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-2bce61e8-02b3-4f84-84a7-89eeea7b95db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:37.804627 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056183c9-a210-4ffa-9474-1879ff4301e8 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:37.833922 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:37.846332 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c48962-cc00-4c90-b3e0-c0b13667ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:37.847179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c48962-cc00-4c90-b3e0-c0b13667ec1a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:37.847700 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c48962-cc00-4c90-b3e0-c0b13667ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:37.848144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c48962-cc00-4c90-b3e0-c0b13667ec1a None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:37.849122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c48962-cc00-4c90-b3e0-c0b13667ec1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:37.849615 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c48962-cc00-4c90-b3e0-c0b13667ec1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:37.851253 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c48962-cc00-4c90-b3e0-c0b13667ec1a None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:37.864260 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 392/786] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:37.866442 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c48962-cc00-4c90-b3e0-c0b13667ec1a None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:37.880477 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 395/787] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:37.881680 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce61e8-02b3-4f84-84a7-89eeea7b95db None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:37.996465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:38.012762 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c48962-cc00-4c90-b3e0-c0b13667ec1a None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:38.017404 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce61e8-02b3-4f84-84a7-89eeea7b95db None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:38.059022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90212ab6-4323-4799-8a88-41537b3c60e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:38.059565 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7addb-f273-4d87-8624-7e8d584a128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:38.059712 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90212ab6-4323-4799-8a88-41537b3c60e5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:38.059965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90212ab6-4323-4799-8a88-41537b3c60e5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:38.060238 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90212ab6-4323-4799-8a88-41537b3c60e5 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:38.060463 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7addb-f273-4d87-8624-7e8d584a128c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:38.060811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7addb-f273-4d87-8624-7e8d584a128c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:38.064363 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90212ab6-4323-4799-8a88-41537b3c60e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:38.064837 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7addb-f273-4d87-8624-7e8d584a128c None admin] PATH_INFO: `/v3/users/swift` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:38.064932 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90212ab6-4323-4799-8a88-41537b3c60e5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:38.069547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 393/788] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:50:37 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:38.070387 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90212ab6-4323-4799-8a88-41537b3c60e5 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:38.070634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7addb-f273-4d87-8624-7e8d584a128c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:38.072196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7addb-f273-4d87-8624-7e8d584a128c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-c8f7addb-f273-4d87-8624-7e8d584a128c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:38.093988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:38.095241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:38.097162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:38.097162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:38.097162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:38.097162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:38.097162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:38.097162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:38.097162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:38.097162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:38.097162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:38.097162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:38.097162 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 396/789] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:38.104365 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7addb-f273-4d87-8624-7e8d584a128c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:38.165645 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:38.229892 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90212ab6-4323-4799-8a88-41537b3c60e5 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:38.285804 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e5231-79ce-4ce0-92db-c3c415960ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:38.287016 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5231-79ce-4ce0-92db-c3c415960ee9 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:38.287560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5231-79ce-4ce0-92db-c3c415960ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:38.288060 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e5231-79ce-4ce0-92db-c3c415960ee9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:38.309526 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7addb-f273-4d87-8624-7e8d584a128c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:38.317456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e5231-79ce-4ce0-92db-c3c415960ee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-618e5231-79ce-4ce0-92db-c3c415960ee9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:38.325217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:38.326358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:38.328438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 394/790] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:38.337419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e5231-79ce-4ce0-92db-c3c415960ee9 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:38.370592 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f102fff-029e-41de-b57f-00c115b32c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:38.371627 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f102fff-029e-41de-b57f-00c115b32c4e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:38.372178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f102fff-029e-41de-b57f-00c115b32c4e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:38.372568 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f102fff-029e-41de-b57f-00c115b32c4e None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:38.373578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f102fff-029e-41de-b57f-00c115b32c4e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:38.373933 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f102fff-029e-41de-b57f-00c115b32c4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:38.376109 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f102fff-029e-41de-b57f-00c115b32c4e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:38.398861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:38.406110 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 397/791] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:50:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:38.412393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f102fff-029e-41de-b57f-00c115b32c4e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:38.515941 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e5231-79ce-4ce0-92db-c3c415960ee9 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:38.536812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:38.580598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d814967-425c-443e-867e-04c45541bd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:38.581341 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814967-425c-443e-867e-04c45541bd68 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:38.581627 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814967-425c-443e-867e-04c45541bd68 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:38.581957 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814967-425c-443e-867e-04c45541bd68 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:38.582720 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d814967-425c-443e-867e-04c45541bd68 None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:38.583032 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d814967-425c-443e-867e-04c45541bd68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:38.584435 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d814967-425c-443e-867e-04c45541bd68 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:38.594565 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f102fff-029e-41de-b57f-00c115b32c4e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:38.599950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 395/792] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:38.609409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d814967-425c-443e-867e-04c45541bd68 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:38.644727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fad2c-085f-4fbf-b78b-7f32e54978cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:38.645594 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fad2c-085f-4fbf-b78b-7f32e54978cc None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:38.646073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fad2c-085f-4fbf-b78b-7f32e54978cc None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:38.646353 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343fad2c-085f-4fbf-b78b-7f32e54978cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:38.659616 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fad2c-085f-4fbf-b78b-7f32e54978cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:38.661663 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343fad2c-085f-4fbf-b78b-7f32e54978cc None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-343fad2c-085f-4fbf-b78b-7f32e54978cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:38.687817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:38.689039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:38.690878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:38.690878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:38.690878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:38.690878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:38.690878 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 398/793] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:38.699217 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fad2c-085f-4fbf-b78b-7f32e54978cc None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:38.785258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:38.812607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d814967-425c-443e-867e-04c45541bd68 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:38.848380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343fad2c-085f-4fbf-b78b-7f32e54978cc None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:38.880567 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a65e04-d422-485e-bbf5-b35c80fcde22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:38.881380 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a65e04-d422-485e-bbf5-b35c80fcde22 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:38.881663 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a65e04-d422-485e-bbf5-b35c80fcde22 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:38.881938 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a65e04-d422-485e-bbf5-b35c80fcde22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:38.907931 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a65e04-d422-485e-bbf5-b35c80fcde22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:38.909565 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a65e04-d422-485e-bbf5-b35c80fcde22 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:38.915997 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f8c40-c27f-47ad-873f-0e225bbde245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:38.917559 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f8c40-c27f-47ad-873f-0e225bbde245 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:38.918173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f8c40-c27f-47ad-873f-0e225bbde245 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:38.918709 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f8c40-c27f-47ad-873f-0e225bbde245 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:38.920346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f8c40-c27f-47ad-873f-0e225bbde245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:38.921009 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f8c40-c27f-47ad-873f-0e225bbde245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:38.923600 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f8c40-c27f-47ad-873f-0e225bbde245 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-24a65e04-d422-485e-bbf5-b35c80fcde22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:38.930105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:38.931145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:38.937141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:38.937141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:38.937141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:38.937141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:38.937141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 396/794] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:38.937141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f5af8-ea44-4e99-ac1c-0fb7d92c0e75 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:38.937141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f5af8-ea44-4e99-ac1c-0fb7d92c0e75 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:38.937141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f5af8-ea44-4e99-ac1c-0fb7d92c0e75 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:38.937141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 397/795] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:38.937749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f5af8-ea44-4e99-ac1c-0fb7d92c0e75 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:38.954951 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 399/796] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:38.956708 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc801bbc-5e8b-4eae-945e-32dbad5e4e7c None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:38.957217 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc801bbc-5e8b-4eae-945e-32dbad5e4e7c None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:38.957727 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc801bbc-5e8b-4eae-945e-32dbad5e4e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:38.973267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:38.986964 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-bc801bbc-5e8b-4eae-945e-32dbad5e4e7c None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:38.988187 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-bc801bbc-5e8b-4eae-945e-32dbad5e4e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:39.072461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 400/797] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:39.072461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc801bbc-5e8b-4eae-945e-32dbad5e4e7c None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:39.085835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f5af8-ea44-4e99-ac1c-0fb7d92c0e75 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:39.114508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:39.137571 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b77cc-964f-4d0b-9df7-0c72b66eb95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:39.138815 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b77cc-964f-4d0b-9df7-0c72b66eb95c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:39.139788 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b77cc-964f-4d0b-9df7-0c72b66eb95c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:39.140280 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b77cc-964f-4d0b-9df7-0c72b66eb95c None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:39.142031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b77cc-964f-4d0b-9df7-0c72b66eb95c None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:39.142718 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b77cc-964f-4d0b-9df7-0c72b66eb95c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:39.145251 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b77cc-964f-4d0b-9df7-0c72b66eb95c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:39.166245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 398/798] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:39.166245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b77cc-964f-4d0b-9df7-0c72b66eb95c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:39.185604 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc801bbc-5e8b-4eae-945e-32dbad5e4e7c None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:39.234538 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149bf16-8bfb-4a9c-b8f8-465a926b697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:39.235958 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149bf16-8bfb-4a9c-b8f8-465a926b697a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:39.236437 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149bf16-8bfb-4a9c-b8f8-465a926b697a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:39.237125 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149bf16-8bfb-4a9c-b8f8-465a926b697a None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:39.247484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149bf16-8bfb-4a9c-b8f8-465a926b697a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:39.249251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149bf16-8bfb-4a9c-b8f8-465a926b697a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:39.262403 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-6149bf16-8bfb-4a9c-b8f8-465a926b697a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:39.264660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:39.265813 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:39.267427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:39.267427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:39.267427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:39.267427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:39.267427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:39.267427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:39.267427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:39.267427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:39.267427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 401/799] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:39.268016 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149bf16-8bfb-4a9c-b8f8-465a926b697a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:39.285293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b77cc-964f-4d0b-9df7-0c72b66eb95c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:39.322791 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12714cc3-7300-4961-8bd0-317920f42c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:39.323733 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714cc3-7300-4961-8bd0-317920f42c0d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:39.324026 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714cc3-7300-4961-8bd0-317920f42c0d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:39.324305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12714cc3-7300-4961-8bd0-317920f42c0d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:39.324993 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12714cc3-7300-4961-8bd0-317920f42c0d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:39.326356 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12714cc3-7300-4961-8bd0-317920f42c0d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-12714cc3-7300-4961-8bd0-317920f42c0d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:39.339527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:39.340701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:39.341749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 399/800] 10.222.0.88 () {56 vars in 1129 bytes} [Thu May 19 20:50:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:39.342768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fccb4-b9ea-4dcc-8dcb-ed24db9980e8 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:39.342768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fccb4-b9ea-4dcc-8dcb-ed24db9980e8 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:39.342768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fccb4-b9ea-4dcc-8dcb-ed24db9980e8 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:39.343737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 400/801] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:39.345183 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fccb4-b9ea-4dcc-8dcb-ed24db9980e8 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:39.381613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149bf16-8bfb-4a9c-b8f8-465a926b697a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:39.383439 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:39.427313 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4a331-12c5-4f07-a0ce-3ea340cd3f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:39.428127 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4a331-12c5-4f07-a0ce-3ea340cd3f20 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:39.428513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4a331-12c5-4f07-a0ce-3ea340cd3f20 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:39.428797 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4a331-12c5-4f07-a0ce-3ea340cd3f20 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:39.437043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4a331-12c5-4f07-a0ce-3ea340cd3f20 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:39.439045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4a331-12c5-4f07-a0ce-3ea340cd3f20 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-e5c4a331-12c5-4f07-a0ce-3ea340cd3f20 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:39.454725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:39.455900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 402/802] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:50:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925587d-1ece-4b0e-b0b6-f19875c96847 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:39.457449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925587d-1ece-4b0e-b0b6-f19875c96847 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:39.458363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925587d-1ece-4b0e-b0b6-f19875c96847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:39.474975 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-c925587d-1ece-4b0e-b0b6-f19875c96847 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:39.475896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-c925587d-1ece-4b0e-b0b6-f19875c96847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:39.478523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fccb4-b9ea-4dcc-8dcb-ed24db9980e8 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:39.516880 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd4965-c650-4db3-9fa9-c662b431ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:39.517557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd4965-c650-4db3-9fa9-c662b431ee63 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:39.517843 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd4965-c650-4db3-9fa9-c662b431ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:39.518126 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd4965-c650-4db3-9fa9-c662b431ee63 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:39.518924 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd4965-c650-4db3-9fa9-c662b431ee63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:39.519361 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd4965-c650-4db3-9fa9-c662b431ee63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:39.520848 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd4965-c650-4db3-9fa9-c662b431ee63 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:39.533838 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 401/803] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:39.534669 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd4965-c650-4db3-9fa9-c662b431ee63 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:39.546144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:39.560371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 403/804] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:39.560371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925587d-1ece-4b0e-b0b6-f19875c96847 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:39.645103 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd4965-c650-4db3-9fa9-c662b431ee63 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:39.662357 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:39.671926 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925587d-1ece-4b0e-b0b6-f19875c96847 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:39.686414 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f1942-e6a4-400a-a549-faab057e0287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:39.687128 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f1942-e6a4-400a-a549-faab057e0287 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:39.687416 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f1942-e6a4-400a-a549-faab057e0287 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:39.687697 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f1942-e6a4-400a-a549-faab057e0287 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:39.688986 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f1942-e6a4-400a-a549-faab057e0287 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:39.689300 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f1942-e6a4-400a-a549-faab057e0287 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:39.690678 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f1942-e6a4-400a-a549-faab057e0287 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:39.700492 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 402/805] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:50:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:39.700492 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f1942-e6a4-400a-a549-faab057e0287 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:39.719340 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb42337-7f7c-4c85-ae1f-c00be8caf6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:39.720106 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb42337-7f7c-4c85-ae1f-c00be8caf6cc None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:39.720429 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb42337-7f7c-4c85-ae1f-c00be8caf6cc None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:39.720708 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb42337-7f7c-4c85-ae1f-c00be8caf6cc None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:39.721554 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb42337-7f7c-4c85-ae1f-c00be8caf6cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:39.721866 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb42337-7f7c-4c85-ae1f-c00be8caf6cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:39.723353 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb42337-7f7c-4c85-ae1f-c00be8caf6cc None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:39.740841 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 404/806] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:50:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:39.742087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb42337-7f7c-4c85-ae1f-c00be8caf6cc None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:39.783936 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:39.817860 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f1942-e6a4-400a-a549-faab057e0287 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:39.853802 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb42337-7f7c-4c85-ae1f-c00be8caf6cc None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:39.854273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2270ed-c1b3-4cc5-9318-a22a067132cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:39.855202 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2270ed-c1b3-4cc5-9318-a22a067132cd None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:39.855498 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2270ed-c1b3-4cc5-9318-a22a067132cd None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:39.855764 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2270ed-c1b3-4cc5-9318-a22a067132cd None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/ba6e1d31d8004f9fa956b78d8716d6e6/roles/ed68c2195de147c0a30d9150b33e47cf` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:39.857858 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2270ed-c1b3-4cc5-9318-a22a067132cd None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=ba6e1d31d8004f9fa956b78d8716d6e6, role_id=ed68c2195de147c0a30d9150b33e47cf)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:39.859298 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2270ed-c1b3-4cc5-9318-a22a067132cd None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:39.889241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d312b-44bd-46df-9a7a-8ca4120b7bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:39.890046 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d312b-44bd-46df-9a7a-8ca4120b7bb5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:39.890331 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d312b-44bd-46df-9a7a-8ca4120b7bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:39.890606 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d312b-44bd-46df-9a7a-8ca4120b7bb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:39.903858 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d312b-44bd-46df-9a7a-8ca4120b7bb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-ff9d312b-44bd-46df-9a7a-8ca4120b7bb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:39.910265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:39.911705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:39.912881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 405/807] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:39.914090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d312b-44bd-46df-9a7a-8ca4120b7bb5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:39.930010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:39.936627 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 403/808] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:50:39 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/ba6e1d31d8004f9fa956b78d8716d6e6/roles/ed68c2195de147c0a30d9150b33e47cf => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:50:39.936627 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2270ed-c1b3-4cc5-9318-a22a067132cd None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:40.053089 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d312b-44bd-46df-9a7a-8ca4120b7bb5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:40.074763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2270ed-c1b3-4cc5-9318-a22a067132cd None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:40.101730 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef2675-af58-4b2b-aedd-222d3b818ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:40.102447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef2675-af58-4b2b-aedd-222d3b818ab0 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:40.102730 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef2675-af58-4b2b-aedd-222d3b818ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:40.103008 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef2675-af58-4b2b-aedd-222d3b818ab0 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:40.103741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef2675-af58-4b2b-aedd-222d3b818ab0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:40.104051 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef2675-af58-4b2b-aedd-222d3b818ab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:40.105564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef2675-af58-4b2b-aedd-222d3b818ab0 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:40.116689 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:40.117246 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 406/809] 10.222.0.88 () {56 vars in 1124 bytes} [Thu May 19 20:50:39 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:40.118011 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef2675-af58-4b2b-aedd-222d3b818ab0 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:40.124956 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47920989-77b1-44ee-8d92-f3094e06835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:40.127617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47920989-77b1-44ee-8d92-f3094e06835c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:40.127617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47920989-77b1-44ee-8d92-f3094e06835c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:40.127617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47920989-77b1-44ee-8d92-f3094e06835c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:40.140616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47920989-77b1-44ee-8d92-f3094e06835c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:40.142773 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47920989-77b1-44ee-8d92-f3094e06835c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-47920989-77b1-44ee-8d92-f3094e06835c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:40.163243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:40.164484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:40.165579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:40.165579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:40.165579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:40.165579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:40.165579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 404/810] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:40.165579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47920989-77b1-44ee-8d92-f3094e06835c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:40.233794 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:40.264622 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef2675-af58-4b2b-aedd-222d3b818ab0 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:40.295919 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47920989-77b1-44ee-8d92-f3094e06835c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:40.303413 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fd41e-ccee-4277-9139-0276cef3631e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:40.304679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fd41e-ccee-4277-9139-0276cef3631e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:40.304981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fd41e-ccee-4277-9139-0276cef3631e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:40.305269 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fd41e-ccee-4277-9139-0276cef3631e None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:40.306069 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fd41e-ccee-4277-9139-0276cef3631e None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:40.306825 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fd41e-ccee-4277-9139-0276cef3631e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:40.308956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fd41e-ccee-4277-9139-0276cef3631e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:40.322379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 407/811] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:40.323305 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fd41e-ccee-4277-9139-0276cef3631e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:40.339132 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52113ab5-02ea-4ee2-b913-8d50d7cf1a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:40.339972 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52113ab5-02ea-4ee2-b913-8d50d7cf1a15 None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:40.340389 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52113ab5-02ea-4ee2-b913-8d50d7cf1a15 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:40.340825 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52113ab5-02ea-4ee2-b913-8d50d7cf1a15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:40.341682 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52113ab5-02ea-4ee2-b913-8d50d7cf1a15 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:40.343161 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52113ab5-02ea-4ee2-b913-8d50d7cf1a15 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:40.442288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fd41e-ccee-4277-9139-0276cef3631e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:40.469493 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 405/812] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:50:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:40.476692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52113ab5-02ea-4ee2-b913-8d50d7cf1a15 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:40.494836 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:40.513993 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20d93d-b8e3-429e-825b-2091006f5c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:40.515024 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20d93d-b8e3-429e-825b-2091006f5c5f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:40.515514 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20d93d-b8e3-429e-825b-2091006f5c5f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:40.515975 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20d93d-b8e3-429e-825b-2091006f5c5f None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:40.518216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20d93d-b8e3-429e-825b-2091006f5c5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:40.518769 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20d93d-b8e3-429e-825b-2091006f5c5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:40.520553 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20d93d-b8e3-429e-825b-2091006f5c5f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:40.538733 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 408/813] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:40.544409 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20d93d-b8e3-429e-825b-2091006f5c5f None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:40.575300 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:40.659003 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52113ab5-02ea-4ee2-b913-8d50d7cf1a15 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:40.689986 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20d93d-b8e3-429e-825b-2091006f5c5f None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:40.700796 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b8ae7-1f37-4427-bfab-dfdc4632e02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:40.701562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b8ae7-1f37-4427-bfab-dfdc4632e02c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:40.701853 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b8ae7-1f37-4427-bfab-dfdc4632e02c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:40.702136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b8ae7-1f37-4427-bfab-dfdc4632e02c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:40.714840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b8ae7-1f37-4427-bfab-dfdc4632e02c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:40.716559 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b8ae7-1f37-4427-bfab-dfdc4632e02c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-384b8ae7-1f37-4427-bfab-dfdc4632e02c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:40.737552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:40.738748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:40.739842 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:40.739842 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:40.739842 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:40.739842 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:40.739842 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 406/814] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:40.740483 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e704e3c-c09f-4199-a39d-f180755707fc None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:40.740775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e704e3c-c09f-4199-a39d-f180755707fc None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:40.741056 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e704e3c-c09f-4199-a39d-f180755707fc None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:40.743019 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32c22d-c163-425b-8c3a-605a326ef767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:40.743189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 407/815] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:40.743285 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32c22d-c163-425b-8c3a-605a326ef767 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:40.743612 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32c22d-c163-425b-8c3a-605a326ef767 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:40.743981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32c22d-c163-425b-8c3a-605a326ef767 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:40.748175 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62bdc8-bea6-4aa3-a65c-3054a16fb003 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:40.748507 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62bdc8-bea6-4aa3-a65c-3054a16fb003 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:40.748797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62bdc8-bea6-4aa3-a65c-3054a16fb003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:40.754448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32c22d-c163-425b-8c3a-605a326ef767 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:40.754448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32c22d-c163-425b-8c3a-605a326ef767 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-2a32c22d-c163-425b-8c3a-605a326ef767 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:40.769578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:40.770697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:40.772940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:40.772940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:40.772940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:40.772940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:40.772940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:40.772940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:40.772940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:40.772940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:40.772940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 409/816] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:40.772940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32c22d-c163-425b-8c3a-605a326ef767 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:40.776636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-ce62bdc8-bea6-4aa3-a65c-3054a16fb003 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:40.777865 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-ce62bdc8-bea6-4aa3-a65c-3054a16fb003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:40.835653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:40.862715 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 408/817] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:40.863599 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62bdc8-bea6-4aa3-a65c-3054a16fb003 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:40.902947 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32c22d-c163-425b-8c3a-605a326ef767 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:40.913447 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:40.945847 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c7616-1503-4376-8806-d9ec4596122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:40.946677 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c7616-1503-4376-8806-d9ec4596122e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:40.947178 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c7616-1503-4376-8806-d9ec4596122e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:40.947563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c7616-1503-4376-8806-d9ec4596122e None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:40.948432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c7616-1503-4376-8806-d9ec4596122e None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:40.948847 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c7616-1503-4376-8806-d9ec4596122e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:40.950534 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c7616-1503-4376-8806-d9ec4596122e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:40.964104 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 410/818] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:40.965348 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c7616-1503-4376-8806-d9ec4596122e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:41.000656 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62bdc8-bea6-4aa3-a65c-3054a16fb003 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:41.037062 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8416e8-4e26-46d8-b92b-8260e310133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:41.037807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8416e8-4e26-46d8-b92b-8260e310133b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:41.038092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8416e8-4e26-46d8-b92b-8260e310133b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:41.038379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8416e8-4e26-46d8-b92b-8260e310133b None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:41.039222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8416e8-4e26-46d8-b92b-8260e310133b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:41.039542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8416e8-4e26-46d8-b92b-8260e310133b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:41.041078 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8416e8-4e26-46d8-b92b-8260e310133b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:41.060147 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 409/819] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:41.061081 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8416e8-4e26-46d8-b92b-8260e310133b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:41.078365 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c7616-1503-4376-8806-d9ec4596122e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:41.108701 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:41.117684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a5728-dd56-4921-b884-61a7af8fee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:41.118626 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a5728-dd56-4921-b884-61a7af8fee9e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:41.119019 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a5728-dd56-4921-b884-61a7af8fee9e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:41.119401 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a5728-dd56-4921-b884-61a7af8fee9e None admin] PATH_INFO: `/v3/services/network` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:41.120166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a5728-dd56-4921-b884-61a7af8fee9e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:41.121753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0a5728-dd56-4921-b884-61a7af8fee9e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-1e0a5728-dd56-4921-b884-61a7af8fee9e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:41.135734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:41.137235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 19 20:50:41.138459 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:41.144453 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 411/820] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:40 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:41.145455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a5728-dd56-4921-b884-61a7af8fee9e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:41.153935 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:41.174986 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8416e8-4e26-46d8-b92b-8260e310133b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:41.213455 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322ff87-0f64-4072-b5db-4fb2cc80ac43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:41.214448 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ff87-0f64-4072-b5db-4fb2cc80ac43 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:41.214775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ff87-0f64-4072-b5db-4fb2cc80ac43 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:41.215086 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322ff87-0f64-4072-b5db-4fb2cc80ac43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:41.223096 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322ff87-0f64-4072-b5db-4fb2cc80ac43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:41.224829 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322ff87-0f64-4072-b5db-4fb2cc80ac43 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-4322ff87-0f64-4072-b5db-4fb2cc80ac43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:41.240434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:41.241628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 410/821] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:50:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:41.243127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322ff87-0f64-4072-b5db-4fb2cc80ac43 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:41.278529 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a5728-dd56-4921-b884-61a7af8fee9e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:41.325340 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95d580-0d57-4350-aafd-917dd456ee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:41.326382 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95d580-0d57-4350-aafd-917dd456ee7c None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:41.326672 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95d580-0d57-4350-aafd-917dd456ee7c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:41.326949 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95d580-0d57-4350-aafd-917dd456ee7c None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/e44797dd45ca43bcbcbb8ac732423782/roles/41728d78a94d4b7abb951f8a3cee2ce2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:41.348482 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95d580-0d57-4350-aafd-917dd456ee7c None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=e44797dd45ca43bcbcbb8ac732423782, role_id=41728d78a94d4b7abb951f8a3cee2ce2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:41.350507 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95d580-0d57-4350-aafd-917dd456ee7c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:41.355206 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322ff87-0f64-4072-b5db-4fb2cc80ac43 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:41.396922 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca76e6-b3f4-44bc-8209-b8b1f376390d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:41.397593 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca76e6-b3f4-44bc-8209-b8b1f376390d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:41.397876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca76e6-b3f4-44bc-8209-b8b1f376390d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:41.398159 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca76e6-b3f4-44bc-8209-b8b1f376390d None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:41.398868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca76e6-b3f4-44bc-8209-b8b1f376390d None admin] RBAC: Adding query filter params (name=network) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:41.399178 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca76e6-b3f4-44bc-8209-b8b1f376390d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:41.400559 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca76e6-b3f4-44bc-8209-b8b1f376390d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:41.405884 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 411/822] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:50:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:41.406501 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca76e6-b3f4-44bc-8209-b8b1f376390d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:41.415873 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:41.448736 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 412/823] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:50:41 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/e44797dd45ca43bcbcbb8ac732423782/roles/41728d78a94d4b7abb951f8a3cee2ce2 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:50:41.451087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95d580-0d57-4350-aafd-917dd456ee7c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:41.514770 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:41.583462 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca76e6-b3f4-44bc-8209-b8b1f376390d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:41.658862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb400a6-4cc9-4c72-8daa-33ecf3ba9a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:41.659579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb400a6-4cc9-4c72-8daa-33ecf3ba9a31 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:41.659951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb400a6-4cc9-4c72-8daa-33ecf3ba9a31 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:41.660234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb400a6-4cc9-4c72-8daa-33ecf3ba9a31 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:41.661129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb400a6-4cc9-4c72-8daa-33ecf3ba9a31 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:41.661450 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb400a6-4cc9-4c72-8daa-33ecf3ba9a31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:41.662974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb400a6-4cc9-4c72-8daa-33ecf3ba9a31 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:41.671080 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95d580-0d57-4350-aafd-917dd456ee7c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:41.714799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 412/824] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:50:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:41.716964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb400a6-4cc9-4c72-8daa-33ecf3ba9a31 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:41.739122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65bab1-1df5-465a-a5b8-d7f4595a6299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:41.740687 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65bab1-1df5-465a-a5b8-d7f4595a6299 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:41.741264 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65bab1-1df5-465a-a5b8-d7f4595a6299 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:41.741749 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c65bab1-1df5-465a-a5b8-d7f4595a6299 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:41.742680 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65bab1-1df5-465a-a5b8-d7f4595a6299 None admin] RBAC: Adding query filter params (type=network) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:41.743200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65bab1-1df5-465a-a5b8-d7f4595a6299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:41.745008 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c65bab1-1df5-465a-a5b8-d7f4595a6299 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:41.753502 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 413/825] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:50:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:41.761099 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65bab1-1df5-465a-a5b8-d7f4595a6299 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:41.802808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:41.850658 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb400a6-4cc9-4c72-8daa-33ecf3ba9a31 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:41.897305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf30f48-79fa-43fe-b3f0-dc5f30b06acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:41.898026 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf30f48-79fa-43fe-b3f0-dc5f30b06acc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:41.898313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf30f48-79fa-43fe-b3f0-dc5f30b06acc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:41.898592 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf30f48-79fa-43fe-b3f0-dc5f30b06acc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:41.904436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:41.924054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c65bab1-1df5-465a-a5b8-d7f4595a6299 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:41.977643 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665af372-6c25-4a3c-82be-9d140d69b0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:41.978319 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665af372-6c25-4a3c-82be-9d140d69b0f7 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:41.978605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665af372-6c25-4a3c-82be-9d140d69b0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:41.978881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665af372-6c25-4a3c-82be-9d140d69b0f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:41.980190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665af372-6c25-4a3c-82be-9d140d69b0f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:41.981629 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665af372-6c25-4a3c-82be-9d140d69b0f7 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:42.025261 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf30f48-79fa-43fe-b3f0-dc5f30b06acc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:42.026760 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf30f48-79fa-43fe-b3f0-dc5f30b06acc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:42.079644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 414/826] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:50:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:42.105824 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:42.149943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 413/827] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:50:42.155603 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665af372-6c25-4a3c-82be-9d140d69b0f7 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:42.332909 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665af372-6c25-4a3c-82be-9d140d69b0f7 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:42.370338 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cf4ef-7b65-4745-b4ab-875847230d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:42.371149 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf4ef-7b65-4745-b4ab-875847230d59 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:42.371443 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf4ef-7b65-4745-b4ab-875847230d59 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:42.371724 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf4ef-7b65-4745-b4ab-875847230d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:42.383043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cf4ef-7b65-4745-b4ab-875847230d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:42.384578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472cf4ef-7b65-4745-b4ab-875847230d59 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:42.386808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-472cf4ef-7b65-4745-b4ab-875847230d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:42.408513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:42.410447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:42.411516 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:42.411516 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:42.411516 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:42.411516 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:42.411516 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 415/828] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:42.417861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf30f48-79fa-43fe-b3f0-dc5f30b06acc None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:42.478833 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20338ed4-2d7a-4819-b2ee-0f87fc01b6ef None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:42.479215 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20338ed4-2d7a-4819-b2ee-0f87fc01b6ef None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:42.479398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20338ed4-2d7a-4819-b2ee-0f87fc01b6ef None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:42.481163 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 416/829] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:42.486729 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44c4fe-d653-4fa9-b3cd-dd9cb26ca86f None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:42.487020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44c4fe-d653-4fa9-b3cd-dd9cb26ca86f None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:42.487308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44c4fe-d653-4fa9-b3cd-dd9cb26ca86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:42.510060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-0c44c4fe-d653-4fa9-b3cd-dd9cb26ca86f None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:42.511774 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-0c44c4fe-d653-4fa9-b3cd-dd9cb26ca86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:42.547804 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf30f48-79fa-43fe-b3f0-dc5f30b06acc None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:42.592631 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df5586-aefc-4f9f-944c-57e52e2fbcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:42.593217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df5586-aefc-4f9f-944c-57e52e2fbcc5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:42.593502 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df5586-aefc-4f9f-944c-57e52e2fbcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:42.593780 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df5586-aefc-4f9f-944c-57e52e2fbcc5 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:42.594568 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df5586-aefc-4f9f-944c-57e52e2fbcc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:42.594794 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 417/830] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:42.594924 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df5586-aefc-4f9f-944c-57e52e2fbcc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:42.600489 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df5586-aefc-4f9f-944c-57e52e2fbcc5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:42.608462 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 414/831] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:42.614100 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44c4fe-d653-4fa9-b3cd-dd9cb26ca86f None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:42.625191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df5586-aefc-4f9f-944c-57e52e2fbcc5 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:42.659290 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:42.735606 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c44c4fe-d653-4fa9-b3cd-dd9cb26ca86f None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:42.744052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df5586-aefc-4f9f-944c-57e52e2fbcc5 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:42.764196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:42.777981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed425b-258f-44c3-a193-12df8aa8c77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:42.779057 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed425b-258f-44c3-a193-12df8aa8c77a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:42.779346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed425b-258f-44c3-a193-12df8aa8c77a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:42.779625 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed425b-258f-44c3-a193-12df8aa8c77a None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:42.788541 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed425b-258f-44c3-a193-12df8aa8c77a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:42.789779 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900efb8-82f9-4a2f-bd11-462e8556f90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:42.790041 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed425b-258f-44c3-a193-12df8aa8c77a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:42.791105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900efb8-82f9-4a2f-bd11-462e8556f90a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:42.791614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900efb8-82f9-4a2f-bd11-462e8556f90a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:42.792191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2900efb8-82f9-4a2f-bd11-462e8556f90a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:42.805497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2900efb8-82f9-4a2f-bd11-462e8556f90a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-5aed425b-258f-44c3-a193-12df8aa8c77a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:42.806789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:42.808060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:42.809654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:42.809654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:42.809654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:42.809654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:42.809654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:42.809654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:42.809654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:42.809654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:42.817033 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 418/832] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:42.817841 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed425b-258f-44c3-a193-12df8aa8c77a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-2900efb8-82f9-4a2f-bd11-462e8556f90a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:42.822612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:42.823708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:42.825197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 415/833] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:42.828054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900efb8-82f9-4a2f-bd11-462e8556f90a None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:42.921606 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:42.939670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2900efb8-82f9-4a2f-bd11-462e8556f90a None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:42.942143 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed425b-258f-44c3-a193-12df8aa8c77a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:42.977925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bd59f-3436-483a-a0fd-d06481bc051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:42.978773 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bd59f-3436-483a-a0fd-d06481bc051f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:42.979160 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bd59f-3436-483a-a0fd-d06481bc051f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:42.979609 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085bd59f-3436-483a-a0fd-d06481bc051f None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:42.980724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bd59f-3436-483a-a0fd-d06481bc051f None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:42.981133 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bd59f-3436-483a-a0fd-d06481bc051f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:42.982339 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0ef4b-2e0a-4c77-aa9a-870127ed904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:42.982823 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085bd59f-3436-483a-a0fd-d06481bc051f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:42.983011 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0ef4b-2e0a-4c77-aa9a-870127ed904a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:42.983304 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0ef4b-2e0a-4c77-aa9a-870127ed904a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:42.983590 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0ef4b-2e0a-4c77-aa9a-870127ed904a None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:42.984520 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0ef4b-2e0a-4c77-aa9a-870127ed904a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:42.985386 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0ef4b-2e0a-4c77-aa9a-870127ed904a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:42.986939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0ef4b-2e0a-4c77-aa9a-870127ed904a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:42.996608 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 416/834] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:42.998567 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bd59f-3436-483a-a0fd-d06481bc051f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:43.001549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 419/835] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:43.009388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0ef4b-2e0a-4c77-aa9a-870127ed904a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:43.069812 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:43.121467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085bd59f-3436-483a-a0fd-d06481bc051f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:43.127495 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0ef4b-2e0a-4c77-aa9a-870127ed904a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:43.162693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67153d8b-d359-40b6-9bc0-7d9dc9dc1309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:43.163400 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82644645-f663-4939-99c7-ed8982bac767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:43.163553 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67153d8b-d359-40b6-9bc0-7d9dc9dc1309 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:43.163733 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67153d8b-d359-40b6-9bc0-7d9dc9dc1309 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:43.163956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67153d8b-d359-40b6-9bc0-7d9dc9dc1309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:43.164194 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82644645-f663-4939-99c7-ed8982bac767 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:43.164644 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82644645-f663-4939-99c7-ed8982bac767 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:43.164834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67153d8b-d359-40b6-9bc0-7d9dc9dc1309 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=77427e1028d349b4a740efe0a985e73b) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:43.164960 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82644645-f663-4939-99c7-ed8982bac767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:43.165142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67153d8b-d359-40b6-9bc0-7d9dc9dc1309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:43.166574 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67153d8b-d359-40b6-9bc0-7d9dc9dc1309 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:43.180534 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 420/836] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:43 2022] GET /identity/v3/role_assignments?user.id=77427e1028d349b4a740efe0a985e73b&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:43.180534 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74ea0-5f1f-4e0b-958e-1ed79eec5c09 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:43.180534 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74ea0-5f1f-4e0b-958e-1ed79eec5c09 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:43.180534 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c74ea0-5f1f-4e0b-958e-1ed79eec5c09 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:43.180534 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 421/837] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:43.183635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c3749-73f8-4259-8d45-7fffaebea37d None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:43.183635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c3749-73f8-4259-8d45-7fffaebea37d None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:43.183635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c3749-73f8-4259-8d45-7fffaebea37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:43.187005 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82644645-f663-4939-99c7-ed8982bac767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:43.188570 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82644645-f663-4939-99c7-ed8982bac767 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:43.196398 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:43.201023 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-b46c3749-73f8-4259-8d45-7fffaebea37d None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:43.202253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-b46c3749-73f8-4259-8d45-7fffaebea37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-82644645-f663-4939-99c7-ed8982bac767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:43.208364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:43.218694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:43.220360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:43.220360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:43.220360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:43.220360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:43.220360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 417/838] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:43.220360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82644645-f663-4939-99c7-ed8982bac767 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:43.300931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 422/839] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:43.317109 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c3749-73f8-4259-8d45-7fffaebea37d None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:43.339392 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82644645-f663-4939-99c7-ed8982bac767 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:43.364713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:43.378622 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea739d-30a6-4767-a779-1cf445a497b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:43.384834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea739d-30a6-4767-a779-1cf445a497b5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:43.384834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea739d-30a6-4767-a779-1cf445a497b5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:43.384834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea739d-30a6-4767-a779-1cf445a497b5 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:43.384834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea739d-30a6-4767-a779-1cf445a497b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:43.384834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea739d-30a6-4767-a779-1cf445a497b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:43.384834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea739d-30a6-4767-a779-1cf445a497b5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:43.396207 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 418/840] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:43.402380 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea739d-30a6-4767-a779-1cf445a497b5 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:43.441336 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c3749-73f8-4259-8d45-7fffaebea37d None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:43.472148 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:43.487606 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e6daa-6de0-4fef-9a36-a7dc9ab85fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:43.489753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e6daa-6de0-4fef-9a36-a7dc9ab85fa5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:43.492974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e6daa-6de0-4fef-9a36-a7dc9ab85fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:43.492974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e6daa-6de0-4fef-9a36-a7dc9ab85fa5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:43.492974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e6daa-6de0-4fef-9a36-a7dc9ab85fa5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:43.492974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e6daa-6de0-4fef-9a36-a7dc9ab85fa5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-0f2e6daa-6de0-4fef-9a36-a7dc9ab85fa5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:43.506174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:43.507644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:50:43.508898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:43.511615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 423/841] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:50:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:43.518240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e6daa-6de0-4fef-9a36-a7dc9ab85fa5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:43.526084 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea739d-30a6-4767-a779-1cf445a497b5 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:43.565058 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa25cc-6e5c-42b9-8c68-8d7e1aec5da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:43.565900 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa25cc-6e5c-42b9-8c68-8d7e1aec5da5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:43.566189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa25cc-6e5c-42b9-8c68-8d7e1aec5da5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:43.566470 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa25cc-6e5c-42b9-8c68-8d7e1aec5da5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:43.575523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa25cc-6e5c-42b9-8c68-8d7e1aec5da5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:43.577154 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa25cc-6e5c-42b9-8c68-8d7e1aec5da5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-b9fa25cc-6e5c-42b9-8c68-8d7e1aec5da5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:43.596152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:43.597547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:43.598692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:43.598692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:43.598692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:43.598692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:43.598692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:43.598692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:43.598692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:43.598692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:43.598692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 19 20:50:43.598692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:43.598692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 419/842] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:50:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:43.605442 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa25cc-6e5c-42b9-8c68-8d7e1aec5da5 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:43.637168 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e6daa-6de0-4fef-9a36-a7dc9ab85fa5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:43.639197 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:43.676471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c89e0-563d-4e40-90e5-c66bdc5708db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:43.677500 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c89e0-563d-4e40-90e5-c66bdc5708db None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:43.677954 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c89e0-563d-4e40-90e5-c66bdc5708db None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:43.678416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c89e0-563d-4e40-90e5-c66bdc5708db None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:43.679441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c89e0-563d-4e40-90e5-c66bdc5708db None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:43.680059 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c89e0-563d-4e40-90e5-c66bdc5708db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:43.681867 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c89e0-563d-4e40-90e5-c66bdc5708db None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:43.688775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 424/843] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:50:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:43.694335 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c89e0-563d-4e40-90e5-c66bdc5708db None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:43.728612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa25cc-6e5c-42b9-8c68-8d7e1aec5da5 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:43.752700 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:43.766690 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d138c0-9011-4bf9-b5c2-13df510a0b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:43.767499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d138c0-9011-4bf9-b5c2-13df510a0b2e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:43.767851 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d138c0-9011-4bf9-b5c2-13df510a0b2e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:43.768383 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d138c0-9011-4bf9-b5c2-13df510a0b2e None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:43.769359 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d138c0-9011-4bf9-b5c2-13df510a0b2e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:43.769774 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d138c0-9011-4bf9-b5c2-13df510a0b2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:43.771355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d138c0-9011-4bf9-b5c2-13df510a0b2e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:43.794513 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 420/844] 10.222.0.88 () {56 vars in 1135 bytes} [Thu May 19 20:50:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:43.795598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d138c0-9011-4bf9-b5c2-13df510a0b2e None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:43.816042 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c89e0-563d-4e40-90e5-c66bdc5708db None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:43.866123 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5080e3-a08f-4a39-ab9c-5180c8945756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:43.866845 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5080e3-a08f-4a39-ab9c-5180c8945756 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:43.867202 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5080e3-a08f-4a39-ab9c-5180c8945756 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:43.867490 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5080e3-a08f-4a39-ab9c-5180c8945756 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:43.868542 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5080e3-a08f-4a39-ab9c-5180c8945756 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:43.868873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5080e3-a08f-4a39-ab9c-5180c8945756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:43.870675 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5080e3-a08f-4a39-ab9c-5180c8945756 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:43.880377 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 425/845] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:50:43 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:43.945945 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d138c0-9011-4bf9-b5c2-13df510a0b2e None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:43.987589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c804674-53b4-446c-a4d1-7600e5b2fba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:43.988736 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c804674-53b4-446c-a4d1-7600e5b2fba7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:43.989249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c804674-53b4-446c-a4d1-7600e5b2fba7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:43.989752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c804674-53b4-446c-a4d1-7600e5b2fba7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:44.044746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:44.072292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c804674-53b4-446c-a4d1-7600e5b2fba7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:44.074106 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c804674-53b4-446c-a4d1-7600e5b2fba7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:44.116106 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67e083-62c9-4d3b-a9ea-4776572f5e9a None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:44.116491 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67e083-62c9-4d3b-a9ea-4776572f5e9a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:44.116694 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67e083-62c9-4d3b-a9ea-4776572f5e9a None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:44.120965 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 426/846] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:50:44.123673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f034f1-91bb-48ea-ae2b-b91382e30774 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:44.123970 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f034f1-91bb-48ea-ae2b-b91382e30774 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:44.124261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f034f1-91bb-48ea-ae2b-b91382e30774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:44.142765 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-e2f034f1-91bb-48ea-ae2b-b91382e30774 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:44.143995 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-e2f034f1-91bb-48ea-ae2b-b91382e30774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:44.170153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:44.188993 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 421/847] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:43 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:50:44.194543 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c804674-53b4-446c-a4d1-7600e5b2fba7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:44.228491 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 427/848] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:44.245739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f034f1-91bb-48ea-ae2b-b91382e30774 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:44.280721 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:44.300982 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c804674-53b4-446c-a4d1-7600e5b2fba7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:44.339792 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24031507-58a0-4f41-9c2f-d5294574107f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:44.340912 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24031507-58a0-4f41-9c2f-d5294574107f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:44.341349 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24031507-58a0-4f41-9c2f-d5294574107f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:44.341764 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24031507-58a0-4f41-9c2f-d5294574107f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:44.353797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24031507-58a0-4f41-9c2f-d5294574107f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:44.355627 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24031507-58a0-4f41-9c2f-d5294574107f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:44.358522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f034f1-91bb-48ea-ae2b-b91382e30774 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-24031507-58a0-4f41-9c2f-d5294574107f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:44.374777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:44.382226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:44.383376 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:44.383376 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:44.383376 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:44.383376 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:44.383376 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 422/849] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:44.383376 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24031507-58a0-4f41-9c2f-d5294574107f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:44.399657 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235dbd4-521a-4eb8-99cd-d9aab193937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:44.400815 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235dbd4-521a-4eb8-99cd-d9aab193937c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:44.401329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235dbd4-521a-4eb8-99cd-d9aab193937c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:44.401820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235dbd4-521a-4eb8-99cd-d9aab193937c None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:44.413785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235dbd4-521a-4eb8-99cd-d9aab193937c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-6235dbd4-521a-4eb8-99cd-d9aab193937c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:44.422061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:44.423216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:50:44.424236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:44.431817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 428/850] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:50:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:44.433332 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235dbd4-521a-4eb8-99cd-d9aab193937c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:44.457861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:44.505687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24031507-58a0-4f41-9c2f-d5294574107f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:44.537989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:44.544108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235dbd4-521a-4eb8-99cd-d9aab193937c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:44.547722 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caca6b-86cc-4227-aa47-d249dbb00723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:44.556140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caca6b-86cc-4227-aa47-d249dbb00723 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:44.556140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caca6b-86cc-4227-aa47-d249dbb00723 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:44.556140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9caca6b-86cc-4227-aa47-d249dbb00723 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:44.556140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9caca6b-86cc-4227-aa47-d249dbb00723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:44.556140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9caca6b-86cc-4227-aa47-d249dbb00723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:44.556140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9caca6b-86cc-4227-aa47-d249dbb00723 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:44.570155 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 423/851] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:44.571379 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caca6b-86cc-4227-aa47-d249dbb00723 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:44.581290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11b2af-d4ba-447e-9d6b-1df2162b3747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:44.581977 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11b2af-d4ba-447e-9d6b-1df2162b3747 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:44.582266 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11b2af-d4ba-447e-9d6b-1df2162b3747 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:44.582550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11b2af-d4ba-447e-9d6b-1df2162b3747 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:44.583418 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11b2af-d4ba-447e-9d6b-1df2162b3747 None admin] RBAC: Adding query filter params (name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:44.583620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11b2af-d4ba-447e-9d6b-1df2162b3747 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:44.585167 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11b2af-d4ba-447e-9d6b-1df2162b3747 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:44.598996 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 429/852] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:50:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:44.600408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11b2af-d4ba-447e-9d6b-1df2162b3747 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:44.686576 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9caca6b-86cc-4227-aa47-d249dbb00723 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:44.693674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:44.714689 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11b2af-d4ba-447e-9d6b-1df2162b3747 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:44.725151 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797c0e0-69d1-4c77-b62f-db9760a35b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:44.726225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797c0e0-69d1-4c77-b62f-db9760a35b91 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:44.726738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797c0e0-69d1-4c77-b62f-db9760a35b91 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:44.727203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797c0e0-69d1-4c77-b62f-db9760a35b91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:44.734803 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797c0e0-69d1-4c77-b62f-db9760a35b91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:44.736558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797c0e0-69d1-4c77-b62f-db9760a35b91 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:44.752512 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e65065-85a7-4363-80c7-20f6aff2edfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:44.753457 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e65065-85a7-4363-80c7-20f6aff2edfd None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:44.753819 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e65065-85a7-4363-80c7-20f6aff2edfd None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:44.754173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e65065-85a7-4363-80c7-20f6aff2edfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-9797c0e0-69d1-4c77-b62f-db9760a35b91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:44.755050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:44.756134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:44.757406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:44.757406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:44.757406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:44.757406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:44.757406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:44.757406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:44.757406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:44.757406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:44.764439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 424/853] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:44.765558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797c0e0-69d1-4c77-b62f-db9760a35b91 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:44.767141 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e65065-85a7-4363-80c7-20f6aff2edfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:44.768739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e65065-85a7-4363-80c7-20f6aff2edfd None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-48e65065-85a7-4363-80c7-20f6aff2edfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:44.788173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:44.789370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:44.790445 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:44.790445 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:44.790445 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:44.790445 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:44.790445 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 430/854] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:44.798221 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e65065-85a7-4363-80c7-20f6aff2edfd None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:44.825639 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:44.886218 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797c0e0-69d1-4c77-b62f-db9760a35b91 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:44.935510 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e1008-9d5c-46b5-bd43-0b905dd7281f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:44.936242 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e1008-9d5c-46b5-bd43-0b905dd7281f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:44.936585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e1008-9d5c-46b5-bd43-0b905dd7281f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:44.936934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e1008-9d5c-46b5-bd43-0b905dd7281f None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:44.937744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e1008-9d5c-46b5-bd43-0b905dd7281f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:44.938852 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e1008-9d5c-46b5-bd43-0b905dd7281f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:44.940421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e1008-9d5c-46b5-bd43-0b905dd7281f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:44.942921 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e65065-85a7-4363-80c7-20f6aff2edfd None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:44.955149 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 425/855] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:44.961117 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e1008-9d5c-46b5-bd43-0b905dd7281f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:44.982182 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb55db-43c4-4a80-af4b-9bacd47b1285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:44.982987 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb55db-43c4-4a80-af4b-9bacd47b1285 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:44.983355 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb55db-43c4-4a80-af4b-9bacd47b1285 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:44.983713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb55db-43c4-4a80-af4b-9bacd47b1285 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:44.984543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb55db-43c4-4a80-af4b-9bacd47b1285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:44.985294 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb55db-43c4-4a80-af4b-9bacd47b1285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:44.986663 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb55db-43c4-4a80-af4b-9bacd47b1285 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:44.988399 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:45.006527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 431/856] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:45.008013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb55db-43c4-4a80-af4b-9bacd47b1285 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:45.074034 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e1008-9d5c-46b5-bd43-0b905dd7281f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:45.084827 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:45.115958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53542cc6-4d5e-4690-aa44-8e73489dd822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:45.117104 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53542cc6-4d5e-4690-aa44-8e73489dd822 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:45.117615 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53542cc6-4d5e-4690-aa44-8e73489dd822 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:45.118085 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53542cc6-4d5e-4690-aa44-8e73489dd822 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:45.119137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53542cc6-4d5e-4690-aa44-8e73489dd822 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=ba6e1d31d8004f9fa956b78d8716d6e6) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:45.119666 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53542cc6-4d5e-4690-aa44-8e73489dd822 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:45.121384 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53542cc6-4d5e-4690-aa44-8e73489dd822 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:45.129722 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 426/857] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:44 2022] GET /identity/v3/role_assignments?user.id=ba6e1d31d8004f9fa956b78d8716d6e6&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:45.147095 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb55db-43c4-4a80-af4b-9bacd47b1285 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:45.189011 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce70f19-9515-4e4c-9cf8-6c50e23ad7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:45.189866 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce70f19-9515-4e4c-9cf8-6c50e23ad7e3 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:45.190156 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce70f19-9515-4e4c-9cf8-6c50e23ad7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:45.190428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce70f19-9515-4e4c-9cf8-6c50e23ad7e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:45.199353 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce70f19-9515-4e4c-9cf8-6c50e23ad7e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:45.200919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce70f19-9515-4e4c-9cf8-6c50e23ad7e3 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-3ce70f19-9515-4e4c-9cf8-6c50e23ad7e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:45.218234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:45.220073 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:45.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:45.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:45.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:45.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:45.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:45.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:45.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:45.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:45.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:50:45.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:45.221358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 432/858] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:50:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:45.226131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53542cc6-4d5e-4690-aa44-8e73489dd822 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:45.347297 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53542cc6-4d5e-4690-aa44-8e73489dd822 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:45.383454 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d9139d-140b-4f15-b949-538f5061cd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:45.384135 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9139d-140b-4f15-b949-538f5061cd91 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:45.384455 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9139d-140b-4f15-b949-538f5061cd91 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:45.384740 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d9139d-140b-4f15-b949-538f5061cd91 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:45.385763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9139d-140b-4f15-b949-538f5061cd91 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:45.386078 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9139d-140b-4f15-b949-538f5061cd91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:45.387516 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d9139d-140b-4f15-b949-538f5061cd91 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:45.407168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 427/859] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:50:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:45.412792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce70f19-9515-4e4c-9cf8-6c50e23ad7e3 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:45.489801 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:45.538503 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce70f19-9515-4e4c-9cf8-6c50e23ad7e3 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:45.575547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4edcf-a361-4380-b2ac-ea293f96e1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:45.576207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4edcf-a361-4380-b2ac-ea293f96e1b6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:45.576524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4edcf-a361-4380-b2ac-ea293f96e1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:45.576807 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e4edcf-a361-4380-b2ac-ea293f96e1b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:45.652398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4edcf-a361-4380-b2ac-ea293f96e1b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:45.653822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e4edcf-a361-4380-b2ac-ea293f96e1b6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:45.661160 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6e6fb-5772-4cae-8e3a-9d99244887fa None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:45.661665 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6e6fb-5772-4cae-8e3a-9d99244887fa None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:45.661754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6e6fb-5772-4cae-8e3a-9d99244887fa None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:45.664069 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 428/860] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:45.669883 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac265f8f-d45c-4223-9ce0-b8918f14598b None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:45.670597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac265f8f-d45c-4223-9ce0-b8918f14598b None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:45.671288 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac265f8f-d45c-4223-9ce0-b8918f14598b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:45.693600 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-ac265f8f-d45c-4223-9ce0-b8918f14598b None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:45.695969 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-ac265f8f-d45c-4223-9ce0-b8918f14598b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:45.720254 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:45.729807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:45.785397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 429/861] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:45.789566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 433/862] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:50:45.794904 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac265f8f-d45c-4223-9ce0-b8918f14598b None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:45.812446 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4edcf-a361-4380-b2ac-ea293f96e1b6 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:45.919904 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac265f8f-d45c-4223-9ce0-b8918f14598b None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:45.954145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e4edcf-a361-4380-b2ac-ea293f96e1b6 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:45.973561 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e539e19-cd64-49b5-9de3-fc863a2dab3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:45.974497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e539e19-cd64-49b5-9de3-fc863a2dab3c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:45.975195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e539e19-cd64-49b5-9de3-fc863a2dab3c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:45.975565 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e539e19-cd64-49b5-9de3-fc863a2dab3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:45.991100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e539e19-cd64-49b5-9de3-fc863a2dab3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:45.992697 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e539e19-cd64-49b5-9de3-fc863a2dab3c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:45.994630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337ec31-3da7-427a-be29-445e70f75923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:45.995687 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337ec31-3da7-427a-be29-445e70f75923 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:45.996232 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337ec31-3da7-427a-be29-445e70f75923 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:45.996743 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337ec31-3da7-427a-be29-445e70f75923 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:45.997652 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337ec31-3da7-427a-be29-445e70f75923 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:45.999251 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8337ec31-3da7-427a-be29-445e70f75923 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-8337ec31-3da7-427a-be29-445e70f75923 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:46.013192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:46.014351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:50:46.015373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-3e539e19-cd64-49b5-9de3-fc863a2dab3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:46.017029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:46.017996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:46.019754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:46.019754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:46.019754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:46.019754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:46.019754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 430/863] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:46.020467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e539e19-cd64-49b5-9de3-fc863a2dab3c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:46.022509 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 434/864] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 20:50:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:46.030381 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337ec31-3da7-427a-be29-445e70f75923 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:46.041834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:46.056519 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:46.163534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e539e19-cd64-49b5-9de3-fc863a2dab3c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:46.178363 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337ec31-3da7-427a-be29-445e70f75923 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:46.203826 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5eab57-a5ba-4dea-b1d7-7bd0f4d39ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:46.204664 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5eab57-a5ba-4dea-b1d7-7bd0f4d39ebf None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:46.204957 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5eab57-a5ba-4dea-b1d7-7bd0f4d39ebf None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:46.205235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5eab57-a5ba-4dea-b1d7-7bd0f4d39ebf None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:46.206068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5eab57-a5ba-4dea-b1d7-7bd0f4d39ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:46.206384 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5eab57-a5ba-4dea-b1d7-7bd0f4d39ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:46.207859 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5eab57-a5ba-4dea-b1d7-7bd0f4d39ebf None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:46.218881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541da6d-cfd3-4ad5-8e26-11d162221b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:46.220549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541da6d-cfd3-4ad5-8e26-11d162221b9e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:46.221058 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541da6d-cfd3-4ad5-8e26-11d162221b9e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:46.221580 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541da6d-cfd3-4ad5-8e26-11d162221b9e None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:46.222622 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541da6d-cfd3-4ad5-8e26-11d162221b9e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:46.223371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541da6d-cfd3-4ad5-8e26-11d162221b9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:46.225721 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 431/865] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:46.226427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541da6d-cfd3-4ad5-8e26-11d162221b9e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:46.226642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5eab57-a5ba-4dea-b1d7-7bd0f4d39ebf None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:46.236463 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 435/866] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:50:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:46.238955 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541da6d-cfd3-4ad5-8e26-11d162221b9e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:46.304724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:46.331302 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:46.346151 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5eab57-a5ba-4dea-b1d7-7bd0f4d39ebf None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:46.352032 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541da6d-cfd3-4ad5-8e26-11d162221b9e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:46.386035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049e1c5-029c-4b9d-857b-1f422953ba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:46.386844 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049e1c5-029c-4b9d-857b-1f422953ba4b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:46.387133 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049e1c5-029c-4b9d-857b-1f422953ba4b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:46.387410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049e1c5-029c-4b9d-857b-1f422953ba4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:46.393086 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d4384-7674-476c-a8f6-eda9945d67df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:46.393752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d4384-7674-476c-a8f6-eda9945d67df None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:46.394035 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d4384-7674-476c-a8f6-eda9945d67df None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:46.394310 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d4384-7674-476c-a8f6-eda9945d67df None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:46.394882 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049e1c5-029c-4b9d-857b-1f422953ba4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:46.395016 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d4384-7674-476c-a8f6-eda9945d67df None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:46.395316 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d4384-7674-476c-a8f6-eda9945d67df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:46.396347 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049e1c5-029c-4b9d-857b-1f422953ba4b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:46.396760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d4384-7674-476c-a8f6-eda9945d67df None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:46.401490 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 436/867] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:50:46 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:46.410059 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378c4a5-d22c-4659-b90f-65d8556d05b3 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:46.410388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378c4a5-d22c-4659-b90f-65d8556d05b3 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:46.410838 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b378c4a5-d22c-4659-b90f-65d8556d05b3 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:46.413398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 437/868] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:46 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-0049e1c5-029c-4b9d-857b-1f422953ba4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:46.414890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:46.415974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:46.417168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:46.417168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:46.417168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:46.417168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:46.417168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:46.417168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:46.417168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:46.417168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:46.419140 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef96503-51f2-4284-9afc-b17e7e79b9b7 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:46.419562 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef96503-51f2-4284-9afc-b17e7e79b9b7 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:46.419859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef96503-51f2-4284-9afc-b17e7e79b9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:46.426165 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 432/869] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:46.427493 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049e1c5-029c-4b9d-857b-1f422953ba4b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:46.447873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-8ef96503-51f2-4284-9afc-b17e7e79b9b7 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:46.447873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-8ef96503-51f2-4284-9afc-b17e7e79b9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:46.539855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 438/870] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:46.564044 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef96503-51f2-4284-9afc-b17e7e79b9b7 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:46.571221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049e1c5-029c-4b9d-857b-1f422953ba4b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:46.609096 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:46.615302 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73380660-eb12-4ed4-acc3-db2127e8ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:46.615302 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73380660-eb12-4ed4-acc3-db2127e8ff1c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:46.615302 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73380660-eb12-4ed4-acc3-db2127e8ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:46.615679 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73380660-eb12-4ed4-acc3-db2127e8ff1c None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:46.616367 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73380660-eb12-4ed4-acc3-db2127e8ff1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:46.616714 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73380660-eb12-4ed4-acc3-db2127e8ff1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:46.618189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73380660-eb12-4ed4-acc3-db2127e8ff1c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:46.635136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 433/871] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:46.641894 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73380660-eb12-4ed4-acc3-db2127e8ff1c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:46.675262 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:46.699286 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef96503-51f2-4284-9afc-b17e7e79b9b7 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:46.744179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd0c4d-39bf-4d66-9dda-3f6ef94442e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:46.745830 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0c4d-39bf-4d66-9dda-3f6ef94442e4 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:46.746338 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0c4d-39bf-4d66-9dda-3f6ef94442e4 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:46.747003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd0c4d-39bf-4d66-9dda-3f6ef94442e4 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:46.748278 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd0c4d-39bf-4d66-9dda-3f6ef94442e4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:46.750166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd0c4d-39bf-4d66-9dda-3f6ef94442e4 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:46.751340 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73380660-eb12-4ed4-acc3-db2127e8ff1c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:46.791547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e8775-3cab-4fcf-b3dd-c40619bd35fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:46.792377 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e8775-3cab-4fcf-b3dd-c40619bd35fa None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:46.792792 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e8775-3cab-4fcf-b3dd-c40619bd35fa None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:46.793141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e8775-3cab-4fcf-b3dd-c40619bd35fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:46.794116 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e8775-3cab-4fcf-b3dd-c40619bd35fa None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=e44797dd45ca43bcbcbb8ac732423782) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:46.794550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e8775-3cab-4fcf-b3dd-c40619bd35fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:46.796120 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e8775-3cab-4fcf-b3dd-c40619bd35fa None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:46.803104 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 434/872] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:46 2022] GET /identity/v3/role_assignments?user.id=e44797dd45ca43bcbcbb8ac732423782&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:46.884471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 439/873] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:50:46 2022] POST /identity/v3/services => generated 251 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:50:47.519352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93655010-bdf3-4308-a009-c69df110e5e6 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:47.519788 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93655010-bdf3-4308-a009-c69df110e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:47.519925 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93655010-bdf3-4308-a009-c69df110e5e6 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:47.521571 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 435/874] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:47.527149 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f165b-e032-4c5a-a565-49cb48a15866 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:47.527618 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f165b-e032-4c5a-a565-49cb48a15866 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:47.527706 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f165b-e032-4c5a-a565-49cb48a15866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:47.545175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-be5f165b-e032-4c5a-a565-49cb48a15866 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:47.546432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-be5f165b-e032-4c5a-a565-49cb48a15866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:47.622776 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 440/875] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:47 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:47.646294 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93655010-bdf3-4308-a009-c69df110e5e6 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:47.710372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:47.758171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93655010-bdf3-4308-a009-c69df110e5e6 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:47.794773 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5601097-09b2-4a3e-8b6d-65ce72c28085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:47.795603 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5601097-09b2-4a3e-8b6d-65ce72c28085 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:47.795889 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5601097-09b2-4a3e-8b6d-65ce72c28085 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:47.796170 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5601097-09b2-4a3e-8b6d-65ce72c28085 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:47.796871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5601097-09b2-4a3e-8b6d-65ce72c28085 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:47.798880 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5601097-09b2-4a3e-8b6d-65ce72c28085 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-c5601097-09b2-4a3e-8b6d-65ce72c28085 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:47.811563 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:47.812783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:47.814525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 436/876] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:50:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:47.820997 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f165b-e032-4c5a-a565-49cb48a15866 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:47.884767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:47.924039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f165b-e032-4c5a-a565-49cb48a15866 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:47.959763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774850f-e3d3-442f-9c96-e5ec82b524b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:47.960546 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774850f-e3d3-442f-9c96-e5ec82b524b6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:47.960838 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774850f-e3d3-442f-9c96-e5ec82b524b6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:47.961122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774850f-e3d3-442f-9c96-e5ec82b524b6 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:47.961840 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774850f-e3d3-442f-9c96-e5ec82b524b6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:47.962153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774850f-e3d3-442f-9c96-e5ec82b524b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:47.963555 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774850f-e3d3-442f-9c96-e5ec82b524b6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:47.968723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 441/877] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:50:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:47.974921 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5601097-09b2-4a3e-8b6d-65ce72c28085 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:48.083421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5601097-09b2-4a3e-8b6d-65ce72c28085 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:48.106001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:48.121052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44162b-9a4e-4f29-8aaa-9ab34a2bbfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:48.121738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44162b-9a4e-4f29-8aaa-9ab34a2bbfcb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:48.122032 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44162b-9a4e-4f29-8aaa-9ab34a2bbfcb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:48.122312 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44162b-9a4e-4f29-8aaa-9ab34a2bbfcb None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:48.123026 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44162b-9a4e-4f29-8aaa-9ab34a2bbfcb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:48.123393 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44162b-9a4e-4f29-8aaa-9ab34a2bbfcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:48.124799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44162b-9a4e-4f29-8aaa-9ab34a2bbfcb None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:48.130553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 437/878] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:50:47 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:48.748694 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef83f1c-1839-40a8-9098-815941873491 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:48.749273 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef83f1c-1839-40a8-9098-815941873491 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:48.749273 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef83f1c-1839-40a8-9098-815941873491 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:48.751481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 442/879] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:48.756945 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f36d6-7e7a-4d30-9c39-dff6bdf62e85 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:48.757233 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f36d6-7e7a-4d30-9c39-dff6bdf62e85 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:48.757513 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f36d6-7e7a-4d30-9c39-dff6bdf62e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:48.775238 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-9b5f36d6-7e7a-4d30-9c39-dff6bdf62e85 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:48.776550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-9b5f36d6-7e7a-4d30-9c39-dff6bdf62e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:48.848366 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 438/880] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:48 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:48.870873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef83f1c-1839-40a8-9098-815941873491 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:48.987438 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef83f1c-1839-40a8-9098-815941873491 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:49.014963 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:49.023690 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb21040-0702-4ba4-8302-41f387de188b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:49.024393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21040-0702-4ba4-8302-41f387de188b None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:49.024765 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21040-0702-4ba4-8302-41f387de188b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:49.025051 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb21040-0702-4ba4-8302-41f387de188b None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:49.025799 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb21040-0702-4ba4-8302-41f387de188b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:49.027203 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb21040-0702-4ba4-8302-41f387de188b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:49.199870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 443/881] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:50:48 2022] POST /identity/v3/services => generated 269 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:49.250893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472f9d-5893-4e8d-9892-f43d1ef2d088 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:49.251244 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472f9d-5893-4e8d-9892-f43d1ef2d088 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:49.251457 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472f9d-5893-4e8d-9892-f43d1ef2d088 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:49.253130 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 439/882] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:49.257449 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfe531-2199-4c1c-bb52-cfbdae72057a None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:49.257738 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfe531-2199-4c1c-bb52-cfbdae72057a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:49.258043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfe531-2199-4c1c-bb52-cfbdae72057a None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:49.259752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 444/883] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:49.259977 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e2838-8921-4ab4-aef0-4751b61f438d None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:49.260168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e2838-8921-4ab4-aef0-4751b61f438d None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:49.260421 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e2838-8921-4ab4-aef0-4751b61f438d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:49.265418 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d499a-9fbd-4d79-b581-2b41500b9bbf None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:49.266457 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d499a-9fbd-4d79-b581-2b41500b9bbf None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:49.266760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d499a-9fbd-4d79-b581-2b41500b9bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:49.279344 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-198e2838-8921-4ab4-aef0-4751b61f438d None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:49.280670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-198e2838-8921-4ab4-aef0-4751b61f438d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:49.286590 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-827d499a-9fbd-4d79-b581-2b41500b9bbf None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:49.287976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-827d499a-9fbd-4d79-b581-2b41500b9bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:49.385125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 440/885] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:49.387694 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 445/885] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:49.408330 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e2838-8921-4ab4-aef0-4751b61f438d None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:49.408580 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d499a-9fbd-4d79-b581-2b41500b9bbf None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:49.459295 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:49.534211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d499a-9fbd-4d79-b581-2b41500b9bbf None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:49.539953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e2838-8921-4ab4-aef0-4751b61f438d None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:49.550728 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:49.575446 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50badb3e-9784-4d01-a1b2-dba722323666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:49.576861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50badb3e-9784-4d01-a1b2-dba722323666 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:49.577405 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50badb3e-9784-4d01-a1b2-dba722323666 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:49.577524 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69356123-db21-4a75-b388-af62de6420f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:49.578040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50badb3e-9784-4d01-a1b2-dba722323666 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:49.578260 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69356123-db21-4a75-b388-af62de6420f1 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:49.578555 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69356123-db21-4a75-b388-af62de6420f1 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:49.578838 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69356123-db21-4a75-b388-af62de6420f1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:49.579617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69356123-db21-4a75-b388-af62de6420f1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:49.581224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69356123-db21-4a75-b388-af62de6420f1 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:49.592522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50badb3e-9784-4d01-a1b2-dba722323666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-69356123-db21-4a75-b388-af62de6420f1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:49.593551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:49.594769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:49.595755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 441/886] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:50:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:49.601096 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69356123-db21-4a75-b388-af62de6420f1 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-50badb3e-9784-4d01-a1b2-dba722323666 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:49.607186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:49.608350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:50:49.609356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:49.612883 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 446/887] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:50:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:49.618533 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50badb3e-9784-4d01-a1b2-dba722323666 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:49.714964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69356123-db21-4a75-b388-af62de6420f1 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:49.718636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:49.752646 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ad508-560a-4eda-a3ee-42cbfc443da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:49.753598 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50badb3e-9784-4d01-a1b2-dba722323666 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:49.754321 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ad508-560a-4eda-a3ee-42cbfc443da9 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:49.754863 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ad508-560a-4eda-a3ee-42cbfc443da9 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:49.755354 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ad508-560a-4eda-a3ee-42cbfc443da9 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:49.756286 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ad508-560a-4eda-a3ee-42cbfc443da9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:49.756644 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ad508-560a-4eda-a3ee-42cbfc443da9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:49.758647 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ad508-560a-4eda-a3ee-42cbfc443da9 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:49.769725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 442/888] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:50:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:49.770819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ad508-560a-4eda-a3ee-42cbfc443da9 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:49.800138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05885b-38ca-49e8-9edd-67848e776f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:49.801312 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05885b-38ca-49e8-9edd-67848e776f77 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:49.802091 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05885b-38ca-49e8-9edd-67848e776f77 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:49.802608 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05885b-38ca-49e8-9edd-67848e776f77 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:49.804285 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05885b-38ca-49e8-9edd-67848e776f77 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:49.804845 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05885b-38ca-49e8-9edd-67848e776f77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:49.807413 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05885b-38ca-49e8-9edd-67848e776f77 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:49.823880 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 447/889] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:50:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:49.825425 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05885b-38ca-49e8-9edd-67848e776f77 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:49.878561 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:49.888142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ad508-560a-4eda-a3ee-42cbfc443da9 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:49.927313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf7660-8bec-469c-8cc3-aca796ec5fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:49.928030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf7660-8bec-469c-8cc3-aca796ec5fce None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:49.928317 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf7660-8bec-469c-8cc3-aca796ec5fce None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:49.928629 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf7660-8bec-469c-8cc3-aca796ec5fce None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:49.929349 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf7660-8bec-469c-8cc3-aca796ec5fce None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:49.929660 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf7660-8bec-469c-8cc3-aca796ec5fce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:49.931114 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf7660-8bec-469c-8cc3-aca796ec5fce None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:49.942617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 443/890] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:50:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:49.943843 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf7660-8bec-469c-8cc3-aca796ec5fce None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:49.953228 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05885b-38ca-49e8-9edd-67848e776f77 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:49.976778 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:49.993661 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace06b4d-8846-4c25-9bba-4b48b4583e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:49.996272 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace06b4d-8846-4c25-9bba-4b48b4583e1b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:49.996272 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace06b4d-8846-4c25-9bba-4b48b4583e1b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:49.996272 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace06b4d-8846-4c25-9bba-4b48b4583e1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:50.009132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace06b4d-8846-4c25-9bba-4b48b4583e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:50.010980 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace06b4d-8846-4c25-9bba-4b48b4583e1b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-ace06b4d-8846-4c25-9bba-4b48b4583e1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:50.030441 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:50.031981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:50.033111 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:50.033111 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:50.033111 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:50.033111 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:50.039897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 448/891] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:50.041315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace06b4d-8846-4c25-9bba-4b48b4583e1b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:50.059233 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf7660-8bec-469c-8cc3-aca796ec5fce None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:50.097350 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d69e5-fc99-4e58-8891-4a322d243138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:50.097986 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d69e5-fc99-4e58-8891-4a322d243138 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:50.098271 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d69e5-fc99-4e58-8891-4a322d243138 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:50.098553 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d69e5-fc99-4e58-8891-4a322d243138 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:50.099356 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d69e5-fc99-4e58-8891-4a322d243138 None admin] RBAC: Adding query filter params (interface=public, service_id=03d9271d59ae47d6a2ce8d7856aa49cc, region_id=RegionOne) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:50.099674 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d69e5-fc99-4e58-8891-4a322d243138 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:50.101453 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043d69e5-fc99-4e58-8891-4a322d243138 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:50.115156 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 444/892] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:50:49 2022] GET /identity/v3/endpoints?service_id=03d9271d59ae47d6a2ce8d7856aa49cc&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:50.116260 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d69e5-fc99-4e58-8891-4a322d243138 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:50.141033 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:50.157594 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace06b4d-8846-4c25-9bba-4b48b4583e1b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:50.195585 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbd1d4-1886-4724-abf6-6b8b03293246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:50.196767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbd1d4-1886-4724-abf6-6b8b03293246 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:50.197371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbd1d4-1886-4724-abf6-6b8b03293246 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:50.197901 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbd1d4-1886-4724-abf6-6b8b03293246 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:50.198872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbd1d4-1886-4724-abf6-6b8b03293246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:50.199432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbd1d4-1886-4724-abf6-6b8b03293246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:50.201112 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbd1d4-1886-4724-abf6-6b8b03293246 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:50.219969 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 449/893] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:50.221283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbd1d4-1886-4724-abf6-6b8b03293246 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:50.225687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:50.232490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043d69e5-fc99-4e58-8891-4a322d243138 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:50.271268 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326206f-4712-44d6-9241-9d31bc89e00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:50.271952 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326206f-4712-44d6-9241-9d31bc89e00e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:50.272239 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326206f-4712-44d6-9241-9d31bc89e00e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:50.272550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326206f-4712-44d6-9241-9d31bc89e00e None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:50.273251 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326206f-4712-44d6-9241-9d31bc89e00e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:50.274770 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326206f-4712-44d6-9241-9d31bc89e00e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:50.288709 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 445/894] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:50:50 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:50:50.352261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbd1d4-1886-4724-abf6-6b8b03293246 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:50.402091 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9625a3e-675f-445f-87bc-b0be4f82a144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:50.403358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9625a3e-675f-445f-87bc-b0be4f82a144 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:50.403884 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9625a3e-675f-445f-87bc-b0be4f82a144 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:50.404429 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9625a3e-675f-445f-87bc-b0be4f82a144 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:50.414175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9625a3e-675f-445f-87bc-b0be4f82a144 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:50.416097 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9625a3e-675f-445f-87bc-b0be4f82a144 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-c9625a3e-675f-445f-87bc-b0be4f82a144 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:50.433785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:50.435673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:50.436957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:50.436957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:50.436957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:50.436957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:50.436957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:50.436957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:50.436957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:50.436957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:50.436957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:50:50.436957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:50.440063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 450/895] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:50:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:50.446430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326206f-4712-44d6-9241-9d31bc89e00e None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:50.492095 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01ef9-2136-47dd-ad90-0b0c852a6cfe None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:50.492469 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01ef9-2136-47dd-ad90-0b0c852a6cfe None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:50.492703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f01ef9-2136-47dd-ad90-0b0c852a6cfe None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:50.494504 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 451/896] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:50.500676 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8bb6d-0c11-4bb7-a8ad-f3872fadaae3 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:50.501006 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8bb6d-0c11-4bb7-a8ad-f3872fadaae3 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:50.501252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8bb6d-0c11-4bb7-a8ad-f3872fadaae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:50.519020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:50.525724 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-72e8bb6d-0c11-4bb7-a8ad-f3872fadaae3 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:50.527066 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-72e8bb6d-0c11-4bb7-a8ad-f3872fadaae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:50.565871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326206f-4712-44d6-9241-9d31bc89e00e None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:50.605061 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dbc45-5eab-4f33-90e0-b7e1a8705173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:50.605828 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dbc45-5eab-4f33-90e0-b7e1a8705173 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:50.606215 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dbc45-5eab-4f33-90e0-b7e1a8705173 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:50.606860 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dbc45-5eab-4f33-90e0-b7e1a8705173 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:50.607584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dbc45-5eab-4f33-90e0-b7e1a8705173 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:50.608007 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dbc45-5eab-4f33-90e0-b7e1a8705173 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:50.609824 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dbc45-5eab-4f33-90e0-b7e1a8705173 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:50.632226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 446/897] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:50:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:50.637152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dbc45-5eab-4f33-90e0-b7e1a8705173 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:50.640126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 452/898] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:50.663167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:50.664023 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8bb6d-0c11-4bb7-a8ad-f3872fadaae3 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:50.754393 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dbc45-5eab-4f33-90e0-b7e1a8705173 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:50.771471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8bb6d-0c11-4bb7-a8ad-f3872fadaae3 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:50.793275 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caf6cb-3e48-48b7-bf7d-d37ca085e11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:50.794262 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caf6cb-3e48-48b7-bf7d-d37ca085e11b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:50.794790 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caf6cb-3e48-48b7-bf7d-d37ca085e11b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:50.795262 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17caf6cb-3e48-48b7-bf7d-d37ca085e11b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:50.809345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c12f1b-6d2b-460f-b108-88b0bf0ad92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:50.810164 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c12f1b-6d2b-460f-b108-88b0bf0ad92b None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:50.810510 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c12f1b-6d2b-460f-b108-88b0bf0ad92b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:50.810874 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c12f1b-6d2b-460f-b108-88b0bf0ad92b None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:50.811690 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c12f1b-6d2b-460f-b108-88b0bf0ad92b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:50.813793 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c12f1b-6d2b-460f-b108-88b0bf0ad92b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:50.817056 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:50.872742 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17caf6cb-3e48-48b7-bf7d-d37ca085e11b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:50.874108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17caf6cb-3e48-48b7-bf7d-d37ca085e11b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:50.895867 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:50.901332 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 453/899] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:50:50 2022] POST /identity/v3/services => generated 241 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:50:51.003943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 447/900] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:50:50 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:50:51.009472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c12f1b-6d2b-460f-b108-88b0bf0ad92b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:51.140524 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c12f1b-6d2b-460f-b108-88b0bf0ad92b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:51.179896 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7f4c2-cbe3-4037-b05e-b7dfe8a8f23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:51.180779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7f4c2-cbe3-4037-b05e-b7dfe8a8f23d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:51.181065 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7f4c2-cbe3-4037-b05e-b7dfe8a8f23d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:51.181342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7f4c2-cbe3-4037-b05e-b7dfe8a8f23d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:51.193686 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7f4c2-cbe3-4037-b05e-b7dfe8a8f23d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:51.195183 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7f4c2-cbe3-4037-b05e-b7dfe8a8f23d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-8ff7f4c2-cbe3-4037-b05e-b7dfe8a8f23d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:51.213172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:51.214324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:51.216031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:51.216031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:51.216031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:51.216031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:51.216031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 454/901] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:51.220413 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caf6cb-3e48-48b7-bf7d-d37ca085e11b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:51.326012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17caf6cb-3e48-48b7-bf7d-d37ca085e11b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:51.344878 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:51.361807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb0f74-5aef-4a35-b00f-288df6690726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:51.362481 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb0f74-5aef-4a35-b00f-288df6690726 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:51.362763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb0f74-5aef-4a35-b00f-288df6690726 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:51.363041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb0f74-5aef-4a35-b00f-288df6690726 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:51.363771 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb0f74-5aef-4a35-b00f-288df6690726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:51.364077 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb0f74-5aef-4a35-b00f-288df6690726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:51.365615 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb0f74-5aef-4a35-b00f-288df6690726 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:51.378503 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 448/902] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:51.385685 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7f4c2-cbe3-4037-b05e-b7dfe8a8f23d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:51.435569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:51.496624 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7f4c2-cbe3-4037-b05e-b7dfe8a8f23d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:51.537675 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32b814-efc0-413f-bdc1-d8a4867358e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:51.538473 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32b814-efc0-413f-bdc1-d8a4867358e9 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:51.538760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32b814-efc0-413f-bdc1-d8a4867358e9 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:51.539035 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32b814-efc0-413f-bdc1-d8a4867358e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:51.546285 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32b814-efc0-413f-bdc1-d8a4867358e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:51.547817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32b814-efc0-413f-bdc1-d8a4867358e9 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-df32b814-efc0-413f-bdc1-d8a4867358e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:51.563593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:51.564855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:51.565937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:51.565937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:51.565937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:51.565937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:51.565937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:51.565937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:51.565937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:51.565937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:51.565937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 455/903] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:51.573187 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb0f74-5aef-4a35-b00f-288df6690726 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:51.674011 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933d100-c29d-472f-981a-f32a6833ce79 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:51.674388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933d100-c29d-472f-981a-f32a6833ce79 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:51.674613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933d100-c29d-472f-981a-f32a6833ce79 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:51.677491 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 456/904] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:50:51.681591 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9d614-9cf5-4ccc-8b17-530c220d3d67 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:51.681883 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9d614-9cf5-4ccc-8b17-530c220d3d67 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:51.682022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb0f74-5aef-4a35-b00f-288df6690726 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:51.682204 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c9d614-9cf5-4ccc-8b17-530c220d3d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:51.700944 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-b5c9d614-9cf5-4ccc-8b17-530c220d3d67 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:51.702071 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-b5c9d614-9cf5-4ccc-8b17-530c220d3d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:51.725702 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583ad2c-eab2-4fcc-8fd2-9b9a213744d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:51.726840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583ad2c-eab2-4fcc-8fd2-9b9a213744d7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:51.727185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583ad2c-eab2-4fcc-8fd2-9b9a213744d7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:51.727752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583ad2c-eab2-4fcc-8fd2-9b9a213744d7 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:51.728627 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583ad2c-eab2-4fcc-8fd2-9b9a213744d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:51.728939 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583ad2c-eab2-4fcc-8fd2-9b9a213744d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:51.731134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8583ad2c-eab2-4fcc-8fd2-9b9a213744d7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:51.746477 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 449/905] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:51.752987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583ad2c-eab2-4fcc-8fd2-9b9a213744d7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:51.784454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 457/906] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:51.801326 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:51.802212 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9d614-9cf5-4ccc-8b17-530c220d3d67 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:51.847460 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:51.881409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8583ad2c-eab2-4fcc-8fd2-9b9a213744d7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:51.933816 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c9d614-9cf5-4ccc-8b17-530c220d3d67 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:51.941011 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0b19f-572b-47c4-9a68-4d03c240bf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:51.941768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0b19f-572b-47c4-9a68-4d03c240bf68 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:51.942129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0b19f-572b-47c4-9a68-4d03c240bf68 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:51.942481 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0b19f-572b-47c4-9a68-4d03c240bf68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:51.943974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0b19f-572b-47c4-9a68-4d03c240bf68 None admin] RBAC: Adding query filter params (role.id=5699d7a934b247b28e1e80867f31e61e, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=e44797dd45ca43bcbcbb8ac732423782) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:51.944380 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0b19f-572b-47c4-9a68-4d03c240bf68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:51.946369 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0b19f-572b-47c4-9a68-4d03c240bf68 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:51.954871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 450/907] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:50:51 2022] GET /identity/v3/role_assignments?user.id=e44797dd45ca43bcbcbb8ac732423782&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=5699d7a934b247b28e1e80867f31e61e => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:51.978052 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa11643-3b5d-4edb-852c-082495003f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:51.978899 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa11643-3b5d-4edb-852c-082495003f48 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:51.979183 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa11643-3b5d-4edb-852c-082495003f48 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:51.979461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa11643-3b5d-4edb-852c-082495003f48 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:51.980200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa11643-3b5d-4edb-852c-082495003f48 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:51.981703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa11643-3b5d-4edb-852c-082495003f48 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-3aa11643-3b5d-4edb-852c-082495003f48 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:52.001207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:52.003753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:52.005450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 458/908] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 20:50:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:52.009510 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0b19f-572b-47c4-9a68-4d03c240bf68 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:52.126099 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0b19f-572b-47c4-9a68-4d03c240bf68 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:52.127828 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:52.164195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862604df-d36a-4181-8137-258afb4f408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:52.164957 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862604df-d36a-4181-8137-258afb4f408f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:52.172767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862604df-d36a-4181-8137-258afb4f408f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:52.172767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862604df-d36a-4181-8137-258afb4f408f None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:52.172767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862604df-d36a-4181-8137-258afb4f408f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:52.172767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862604df-d36a-4181-8137-258afb4f408f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:52.172767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862604df-d36a-4181-8137-258afb4f408f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:52.174004 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 451/909] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:50:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:52.179447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa11643-3b5d-4edb-852c-082495003f48 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:52.294383 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:52.299219 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa11643-3b5d-4edb-852c-082495003f48 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:52.337335 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31758-1e03-4d80-b4a2-69af55f4d4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:52.338063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31758-1e03-4d80-b4a2-69af55f4d4c3 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:52.338358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31758-1e03-4d80-b4a2-69af55f4d4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:52.338643 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31758-1e03-4d80-b4a2-69af55f4d4c3 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:52.339693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a31758-1e03-4d80-b4a2-69af55f4d4c3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:52.340432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a31758-1e03-4d80-b4a2-69af55f4d4c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:52.341519 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a31758-1e03-4d80-b4a2-69af55f4d4c3 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:52.348094 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 459/910] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:50:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:52.354022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862604df-d36a-4181-8137-258afb4f408f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:52.460371 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862604df-d36a-4181-8137-258afb4f408f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:52.501694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32099954-e373-41db-80cf-c380de3a16ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:52.502391 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32099954-e373-41db-80cf-c380de3a16ab None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:52.502673 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32099954-e373-41db-80cf-c380de3a16ab None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:52.502954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32099954-e373-41db-80cf-c380de3a16ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:52.503775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32099954-e373-41db-80cf-c380de3a16ab None admin] RBAC: Adding query filter params (interface=public, service_id=1bdf3e09fca048c2a38f26ef478373de, region_id=RegionOne) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:52.504090 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32099954-e373-41db-80cf-c380de3a16ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:52.505576 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32099954-e373-41db-80cf-c380de3a16ab None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:52.511903 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 452/911] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:50:52 2022] GET /identity/v3/endpoints?service_id=1bdf3e09fca048c2a38f26ef478373de&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:52.517508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31758-1e03-4d80-b4a2-69af55f4d4c3 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:52.627520 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a31758-1e03-4d80-b4a2-69af55f4d4c3 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:52.665810 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f33b8e-0ad0-40bf-9287-873ed25de89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:52.666493 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f33b8e-0ad0-40bf-9287-873ed25de89b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:52.666785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f33b8e-0ad0-40bf-9287-873ed25de89b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:52.667068 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f33b8e-0ad0-40bf-9287-873ed25de89b None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:52.667768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f33b8e-0ad0-40bf-9287-873ed25de89b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:52.669207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f33b8e-0ad0-40bf-9287-873ed25de89b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:52.677017 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 460/912] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:50:52 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:50:52.739162 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b765-f84c-4e4e-a408-cb88bc504ee7 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:52.739531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b765-f84c-4e4e-a408-cb88bc504ee7 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:52.739710 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b765-f84c-4e4e-a408-cb88bc504ee7 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:52.741519 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 453/913] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:52.747566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344d1a1-cdf7-423d-851d-eb00cf96b518 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:52.747925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344d1a1-cdf7-423d-851d-eb00cf96b518 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:52.748147 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344d1a1-cdf7-423d-851d-eb00cf96b518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:52.767291 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-3344d1a1-cdf7-423d-851d-eb00cf96b518 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:52.768547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-3344d1a1-cdf7-423d-851d-eb00cf96b518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:52.797428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:52.852868 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 461/914] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:52.875204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b765-f84c-4e4e-a408-cb88bc504ee7 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:52.912750 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:53.008786 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b765-f84c-4e4e-a408-cb88bc504ee7 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:53.045766 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1ae82-23c1-43b1-a77e-4f4ccb378676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:53.046581 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1ae82-23c1-43b1-a77e-4f4ccb378676 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:53.046869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1ae82-23c1-43b1-a77e-4f4ccb378676 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:53.047156 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1ae82-23c1-43b1-a77e-4f4ccb378676 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:53.047818 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1ae82-23c1-43b1-a77e-4f4ccb378676 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:53.049224 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1ae82-23c1-43b1-a77e-4f4ccb378676 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-baa1ae82-23c1-43b1-a77e-4f4ccb378676 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:53.061822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:53.063655 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:53.064833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 454/915] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:50:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:53.072722 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344d1a1-cdf7-423d-851d-eb00cf96b518 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:53.180046 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344d1a1-cdf7-423d-851d-eb00cf96b518 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:53.220937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06ba30-c09a-4cae-a8ea-11c0dd077722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:53.221958 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06ba30-c09a-4cae-a8ea-11c0dd077722 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:53.222423 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06ba30-c09a-4cae-a8ea-11c0dd077722 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:53.222905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06ba30-c09a-4cae-a8ea-11c0dd077722 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:53.223857 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06ba30-c09a-4cae-a8ea-11c0dd077722 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:53.224370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06ba30-c09a-4cae-a8ea-11c0dd077722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:53.226153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06ba30-c09a-4cae-a8ea-11c0dd077722 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:53.232885 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 462/916] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:50:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:53.241729 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1ae82-23c1-43b1-a77e-4f4ccb378676 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:53.345326 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:53.352920 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1ae82-23c1-43b1-a77e-4f4ccb378676 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:53.393461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d02d00-66f5-46c4-80b4-eeb19b1b0ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:53.394157 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d02d00-66f5-46c4-80b4-eeb19b1b0ede None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:53.394450 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d02d00-66f5-46c4-80b4-eeb19b1b0ede None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:53.394738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d02d00-66f5-46c4-80b4-eeb19b1b0ede None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:53.395461 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d02d00-66f5-46c4-80b4-eeb19b1b0ede None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:53.395775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d02d00-66f5-46c4-80b4-eeb19b1b0ede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:53.397206 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d02d00-66f5-46c4-80b4-eeb19b1b0ede None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:53.403106 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 455/917] 10.222.0.88 () {56 vars in 1114 bytes} [Thu May 19 20:50:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:53.408839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06ba30-c09a-4cae-a8ea-11c0dd077722 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:53.410299 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e498ff-7e5b-4129-9546-339cc39441f0 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:53.410593 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e498ff-7e5b-4129-9546-339cc39441f0 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:53.410875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e498ff-7e5b-4129-9546-339cc39441f0 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:53.412524 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 456/918] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:53.417888 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ccd8c-c8e3-487e-9e63-30aedb7220ff None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:53.418182 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ccd8c-c8e3-487e-9e63-30aedb7220ff None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:53.418466 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ccd8c-c8e3-487e-9e63-30aedb7220ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:53.433671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:53.436531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-a80ccd8c-c8e3-487e-9e63-30aedb7220ff None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:53.438332 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-a80ccd8c-c8e3-487e-9e63-30aedb7220ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:53.521200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06ba30-c09a-4cae-a8ea-11c0dd077722 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:53.525439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 457/919] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:53.548889 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ccd8c-c8e3-487e-9e63-30aedb7220ff None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:53.563445 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54e37a-7ccf-4444-8f1f-ceb3552db5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:53.564511 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54e37a-7ccf-4444-8f1f-ceb3552db5b1 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:53.565013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54e37a-7ccf-4444-8f1f-ceb3552db5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:53.565551 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54e37a-7ccf-4444-8f1f-ceb3552db5b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:53.566509 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54e37a-7ccf-4444-8f1f-ceb3552db5b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:53.568153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d54e37a-7ccf-4444-8f1f-ceb3552db5b1 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:53.642492 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:53.660367 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ccd8c-c8e3-487e-9e63-30aedb7220ff None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:53.660616 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 463/920] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:50:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:53.708703 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c781972-e29a-4a97-b994-585e51238a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:53.709522 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c781972-e29a-4a97-b994-585e51238a30 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:53.709812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c781972-e29a-4a97-b994-585e51238a30 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:53.710092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c781972-e29a-4a97-b994-585e51238a30 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:53.710763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c781972-e29a-4a97-b994-585e51238a30 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:53.712152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c781972-e29a-4a97-b994-585e51238a30 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-2c781972-e29a-4a97-b994-585e51238a30 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:53.726204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:53.727540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:50:53.729049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:53.732082 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 458/921] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:50:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:53.734426 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54e37a-7ccf-4444-8f1f-ceb3552db5b1 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:53.822505 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:53.855215 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54e37a-7ccf-4444-8f1f-ceb3552db5b1 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:53.901112 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f763490-b8bb-420d-a784-581620d898c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:53.901811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f763490-b8bb-420d-a784-581620d898c0 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:53.902103 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f763490-b8bb-420d-a784-581620d898c0 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:53.902389 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f763490-b8bb-420d-a784-581620d898c0 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:53.903120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f763490-b8bb-420d-a784-581620d898c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:53.903433 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f763490-b8bb-420d-a784-581620d898c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:53.904885 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f763490-b8bb-420d-a784-581620d898c0 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:53.910920 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 464/922] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:50:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:53.916922 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c781972-e29a-4a97-b994-585e51238a30 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:54.041698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c781972-e29a-4a97-b994-585e51238a30 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:54.079123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ac9dd-ce07-42fb-acde-d8b0f602dd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:54.080040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ac9dd-ce07-42fb-acde-d8b0f602dd46 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:54.081116 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ac9dd-ce07-42fb-acde-d8b0f602dd46 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:54.081413 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ac9dd-ce07-42fb-acde-d8b0f602dd46 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:54.082129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ac9dd-ce07-42fb-acde-d8b0f602dd46 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:54.082444 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ac9dd-ce07-42fb-acde-d8b0f602dd46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:54.083838 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ac9dd-ce07-42fb-acde-d8b0f602dd46 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:54.089981 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 459/923] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:50:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:54.096428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f763490-b8bb-420d-a784-581620d898c0 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:54.206427 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f763490-b8bb-420d-a784-581620d898c0 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:54.245604 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f7653-1289-4cdb-b586-547629715da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:54.246286 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f7653-1289-4cdb-b586-547629715da4 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:54.246570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f7653-1289-4cdb-b586-547629715da4 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:54.246848 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f7653-1289-4cdb-b586-547629715da4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:54.247662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f7653-1289-4cdb-b586-547629715da4 None admin] RBAC: Adding query filter params (interface=public, service_id=e20d855b43d143a789bb9629f7828731, region_id=RegionOne) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:54.247973 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f7653-1289-4cdb-b586-547629715da4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:54.249428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f7653-1289-4cdb-b586-547629715da4 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:54.255669 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 465/924] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:50:54 2022] GET /identity/v3/endpoints?service_id=e20d855b43d143a789bb9629f7828731&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:54.261219 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ac9dd-ce07-42fb-acde-d8b0f602dd46 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:54.278671 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:54.376870 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ac9dd-ce07-42fb-acde-d8b0f602dd46 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:54.415743 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ee4bc-6e74-482f-a677-a9ec60f6d20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:54.416750 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ee4bc-6e74-482f-a677-a9ec60f6d20f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:54.417198 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ee4bc-6e74-482f-a677-a9ec60f6d20f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:54.417669 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ee4bc-6e74-482f-a677-a9ec60f6d20f None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:54.418533 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ee4bc-6e74-482f-a677-a9ec60f6d20f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:54.420130 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ee4bc-6e74-482f-a677-a9ec60f6d20f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:54.427680 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 460/925] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:50:54 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:50:54.431660 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a143795-1ff6-400b-9bee-beca04edbd67 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:54.432199 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a143795-1ff6-400b-9bee-beca04edbd67 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:54.432677 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a143795-1ff6-400b-9bee-beca04edbd67 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:54.434583 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 466/926] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:54.443025 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a5cbe-5f0e-4086-98e3-8b4cff2480e4 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:54.443586 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a5cbe-5f0e-4086-98e3-8b4cff2480e4 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:54.444053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a5cbe-5f0e-4086-98e3-8b4cff2480e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:54.464222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-aa2a5cbe-5f0e-4086-98e3-8b4cff2480e4 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:54.465750 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-aa2a5cbe-5f0e-4086-98e3-8b4cff2480e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:54.560480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 461/927] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:54.583916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a143795-1ff6-400b-9bee-beca04edbd67 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:54.589468 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:54.699919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a143795-1ff6-400b-9bee-beca04edbd67 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:54.737131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334cec0-0b58-45c0-a37a-50023e25e979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:54.737947 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334cec0-0b58-45c0-a37a-50023e25e979 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:54.738230 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334cec0-0b58-45c0-a37a-50023e25e979 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:54.738503 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334cec0-0b58-45c0-a37a-50023e25e979 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:54.750824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334cec0-0b58-45c0-a37a-50023e25e979 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-8334cec0-0b58-45c0-a37a-50023e25e979 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:54.759186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:54.760550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:54.761703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 467/928] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:50:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:54.769666 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a5cbe-5f0e-4086-98e3-8b4cff2480e4 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:54.778165 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:54.878181 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a5cbe-5f0e-4086-98e3-8b4cff2480e4 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:54.918835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1512b2a-af99-41b4-9700-601da9293424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:54.919980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1512b2a-af99-41b4-9700-601da9293424 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:54.920555 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1512b2a-af99-41b4-9700-601da9293424 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:54.921081 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1512b2a-af99-41b4-9700-601da9293424 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:54.922638 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1512b2a-af99-41b4-9700-601da9293424 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:54.923180 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1512b2a-af99-41b4-9700-601da9293424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:54.925358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1512b2a-af99-41b4-9700-601da9293424 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:54.940452 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 462/929] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:50:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:54.942419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334cec0-0b58-45c0-a37a-50023e25e979 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:55.014075 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:55.062222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334cec0-0b58-45c0-a37a-50023e25e979 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:55.095544 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d92e37-d7ca-4689-8bc4-7fbf38494a35 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:55.096078 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d92e37-d7ca-4689-8bc4-7fbf38494a35 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:55.096357 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d92e37-d7ca-4689-8bc4-7fbf38494a35 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:55.098807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 463/930] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:55.104272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a978-d7be-4a3d-b2e2-0a20b304530b None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:55.104605 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a978-d7be-4a3d-b2e2-0a20b304530b None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:55.104859 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37a978-d7be-4a3d-b2e2-0a20b304530b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:55.105855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec035c94-2628-4ebc-aaf9-de0d319902a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:55.106756 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec035c94-2628-4ebc-aaf9-de0d319902a5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:55.107049 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec035c94-2628-4ebc-aaf9-de0d319902a5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:55.107436 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec035c94-2628-4ebc-aaf9-de0d319902a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:55.121110 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec035c94-2628-4ebc-aaf9-de0d319902a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:55.122677 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec035c94-2628-4ebc-aaf9-de0d319902a5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:55.123775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-0e37a978-d7be-4a3d-b2e2-0a20b304530b None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:55.124998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-0e37a978-d7be-4a3d-b2e2-0a20b304530b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-ec035c94-2628-4ebc-aaf9-de0d319902a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:55.145942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:55.147430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:55.148781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:55.148781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:55.148781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:55.148781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:55.152412 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 468/931] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:55.154026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec035c94-2628-4ebc-aaf9-de0d319902a5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:55.196750 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:55.208416 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 464/932] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:50:55.226094 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37a978-d7be-4a3d-b2e2-0a20b304530b None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:55.279775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec035c94-2628-4ebc-aaf9-de0d319902a5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:55.298406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:55.318300 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ff15d-1bd4-4c36-9b5b-e24313531076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:55.319327 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff15d-1bd4-4c36-9b5b-e24313531076 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:55.319921 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff15d-1bd4-4c36-9b5b-e24313531076 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:55.320560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ff15d-1bd4-4c36-9b5b-e24313531076 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:55.321545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ff15d-1bd4-4c36-9b5b-e24313531076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:55.322120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ff15d-1bd4-4c36-9b5b-e24313531076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:55.323955 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ff15d-1bd4-4c36-9b5b-e24313531076 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:55.337943 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 469/933] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:55.344620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ff15d-1bd4-4c36-9b5b-e24313531076 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:55.352134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37a978-d7be-4a3d-b2e2-0a20b304530b None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:55.389697 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f3b8d-aa7f-4f89-9397-3837b1009a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:55.391132 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f3b8d-aa7f-4f89-9397-3837b1009a16 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:55.391650 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f3b8d-aa7f-4f89-9397-3837b1009a16 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:55.392171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f3b8d-aa7f-4f89-9397-3837b1009a16 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:55.393076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f3b8d-aa7f-4f89-9397-3837b1009a16 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:55.394664 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f3b8d-aa7f-4f89-9397-3837b1009a16 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-335f3b8d-aa7f-4f89-9397-3837b1009a16 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:55.413562 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:55.414727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 19 20:50:55.415711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:55.424726 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 465/934] 10.222.0.88 () {56 vars in 1132 bytes} [Thu May 19 20:50:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:55.426134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f3b8d-aa7f-4f89-9397-3837b1009a16 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:55.465297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ff15d-1bd4-4c36-9b5b-e24313531076 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:55.503822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3a00b-3fb2-4102-af6d-d549a0245529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:55.505089 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3a00b-3fb2-4102-af6d-d549a0245529 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:55.505591 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3a00b-3fb2-4102-af6d-d549a0245529 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:55.506077 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3a00b-3fb2-4102-af6d-d549a0245529 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:55.512709 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:55.514232 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3a00b-3fb2-4102-af6d-d549a0245529 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:55.516052 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca3a00b-3fb2-4102-af6d-d549a0245529 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-2ca3a00b-3fb2-4102-af6d-d549a0245529 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:50:55.533166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:55.534393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:55.535522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:55.535522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:55.535522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:50:55.535522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:50:55.535522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:50:55.535522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:50:55.535522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:50:55.535522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:50:55.535522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:50:55.535522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:55.542806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 470/935] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:50:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:55.543940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3a00b-3fb2-4102-af6d-d549a0245529 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:55.560115 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f3b8d-aa7f-4f89-9397-3837b1009a16 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:55.584930 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:55.612022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceba52c-beb9-453d-baeb-5ef2ab6399f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:55.613579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceba52c-beb9-453d-baeb-5ef2ab6399f1 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:55.614308 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceba52c-beb9-453d-baeb-5ef2ab6399f1 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:55.614922 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceba52c-beb9-453d-baeb-5ef2ab6399f1 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:55.615993 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceba52c-beb9-453d-baeb-5ef2ab6399f1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:55.616537 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceba52c-beb9-453d-baeb-5ef2ab6399f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:55.618654 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceba52c-beb9-453d-baeb-5ef2ab6399f1 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:55.627524 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 466/936] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:50:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:55.632790 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceba52c-beb9-453d-baeb-5ef2ab6399f1 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:55.668594 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca3a00b-3fb2-4102-af6d-d549a0245529 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:55.720886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e9f8b-0461-4811-9ca0-4cb9376ed581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:55.721564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e9f8b-0461-4811-9ca0-4cb9376ed581 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:55.721843 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e9f8b-0461-4811-9ca0-4cb9376ed581 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:55.722113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e9f8b-0461-4811-9ca0-4cb9376ed581 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:55.722964 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e9f8b-0461-4811-9ca0-4cb9376ed581 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:55.724153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e9f8b-0461-4811-9ca0-4cb9376ed581 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:55.725640 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e9f8b-0461-4811-9ca0-4cb9376ed581 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:55.745775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 471/937] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:50:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:55.751758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e9f8b-0461-4811-9ca0-4cb9376ed581 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:55.755459 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceba52c-beb9-453d-baeb-5ef2ab6399f1 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:55.795102 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:55.807067 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775d52a-6915-4b24-bae5-b0a7ed09bc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:55.807757 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775d52a-6915-4b24-bae5-b0a7ed09bc1e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:55.808212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775d52a-6915-4b24-bae5-b0a7ed09bc1e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:55.811179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9775d52a-6915-4b24-bae5-b0a7ed09bc1e None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:55.811179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775d52a-6915-4b24-bae5-b0a7ed09bc1e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:55.811179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775d52a-6915-4b24-bae5-b0a7ed09bc1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:55.812217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9775d52a-6915-4b24-bae5-b0a7ed09bc1e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:55.823766 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 467/938] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:50:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:50:55.825348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775d52a-6915-4b24-bae5-b0a7ed09bc1e None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:55.867285 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:55.875313 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e9f8b-0461-4811-9ca0-4cb9376ed581 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:55.919902 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46355c-7f3e-443d-9797-2dea64c7b94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:55.921186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46355c-7f3e-443d-9797-2dea64c7b94e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:55.921802 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46355c-7f3e-443d-9797-2dea64c7b94e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:55.922382 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46355c-7f3e-443d-9797-2dea64c7b94e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:55.935761 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46355c-7f3e-443d-9797-2dea64c7b94e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:55.938253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a46355c-7f3e-443d-9797-2dea64c7b94e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-1a46355c-7f3e-443d-9797-2dea64c7b94e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:50:55.960045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:55.961620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:55.962895 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:50:55.962895 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:50:55.962895 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:50:55.962895 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:55.964901 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9775d52a-6915-4b24-bae5-b0a7ed09bc1e None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:55.970090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 472/939] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:55.971400 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46355c-7f3e-443d-9797-2dea64c7b94e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:56.010210 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf426989-fb7e-4049-9c9c-177aee098eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:56.011979 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426989-fb7e-4049-9c9c-177aee098eab None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.012392 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426989-fb7e-4049-9c9c-177aee098eab None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.012777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf426989-fb7e-4049-9c9c-177aee098eab None admin] PATH_INFO: `/v3/endpoints` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.013738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf426989-fb7e-4049-9c9c-177aee098eab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:56.015226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf426989-fb7e-4049-9c9c-177aee098eab None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:56.090705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a46355c-7f3e-443d-9797-2dea64c7b94e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:56.127400 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099bf873-246f-4b35-b922-4c0273502477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:56.128087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bf873-246f-4b35-b922-4c0273502477 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.128370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bf873-246f-4b35-b922-4c0273502477 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.128650 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099bf873-246f-4b35-b922-4c0273502477 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.129383 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099bf873-246f-4b35-b922-4c0273502477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:56.129691 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099bf873-246f-4b35-b922-4c0273502477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:56.131119 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099bf873-246f-4b35-b922-4c0273502477 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:56.131903 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 468/940] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:50:55 2022] POST /identity/v3/endpoints => generated 347 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:56.133242 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ec398-5898-43b6-8912-930fc2d889cb None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.133551 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ec398-5898-43b6-8912-930fc2d889cb None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.133901 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ec398-5898-43b6-8912-930fc2d889cb None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.139522 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 469/941] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:56.141729 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeb9be-b666-4cec-b0ad-ad9ee6f9ead9 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.142024 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeb9be-b666-4cec-b0ad-ad9ee6f9ead9 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.142307 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaeb9be-b666-4cec-b0ad-ad9ee6f9ead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.157510 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 473/942] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:56.166360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-cdaeb9be-b666-4cec-b0ad-ad9ee6f9ead9 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:56.166902 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099bf873-246f-4b35-b922-4c0273502477 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:56.167596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-cdaeb9be-b666-4cec-b0ad-ad9ee6f9ead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:56.179667 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:56.223170 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:56.261868 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 470/943] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:56.290503 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaeb9be-b666-4cec-b0ad-ad9ee6f9ead9 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:56.326569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099bf873-246f-4b35-b922-4c0273502477 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:56.374364 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86912fe4-7f86-474b-8445-96a97c09631f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:56.375156 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86912fe4-7f86-474b-8445-96a97c09631f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.376085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86912fe4-7f86-474b-8445-96a97c09631f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.376397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86912fe4-7f86-474b-8445-96a97c09631f None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.389270 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86912fe4-7f86-474b-8445-96a97c09631f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:56.392824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86912fe4-7f86-474b-8445-96a97c09631f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-86912fe4-7f86-474b-8445-96a97c09631f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:56.407849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:56.412142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:56.413455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:56.413455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:56.413455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:56.413455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:56.413455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:56.413455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:56.413455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:56.413455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:56.413455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 474/944] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:56.415393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86912fe4-7f86-474b-8445-96a97c09631f None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:56.424537 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaeb9be-b666-4cec-b0ad-ad9ee6f9ead9 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:56.473839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15f03e-7bad-4ed3-a540-1ae98e49327c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:56.474578 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15f03e-7bad-4ed3-a540-1ae98e49327c None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.474869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15f03e-7bad-4ed3-a540-1ae98e49327c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.475154 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15f03e-7bad-4ed3-a540-1ae98e49327c None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.476413 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15f03e-7bad-4ed3-a540-1ae98e49327c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:56.478520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15f03e-7bad-4ed3-a540-1ae98e49327c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:56.479936 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: WARNING keystone.server.flask.common [None req-fd15f03e-7bad-4ed3-a540-1ae98e49327c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:50:56.504724 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:56.533913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.sql.core [None req-fd15f03e-7bad-4ed3-a540-1ae98e49327c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:50:56.533913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:50:56.533913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [parameters: {'id': 'e0a24c9c28b24a46bc16c76ae863e8ef', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:50:56.533913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=278914) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:50:56.536917 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86912fe4-7f86-474b-8445-96a97c09631f None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-fd15f03e-7bad-4ed3-a540-1ae98e49327c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e0a24c9c28b24a46bc16c76ae863e8ef.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e0a24c9c28b24a46bc16c76ae863e8ef. May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application conn.query(q) May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:50:56.544508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application result.read() May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 19 20:50:56.545499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return project_ref.to_dict() May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application next(self.gen) May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application yield resource May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application next(self.gen) May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise self.value May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application session.commit() May 19 20:50:56.546745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self.session.flush() May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application compat.raise_( May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:50:56.547869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application c = connection._execute_20( May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:50:56.551229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application conn.query(q) May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application result.read() May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:50:56.553958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [parameters: {'id': 'e0a24c9c28b24a46bc16c76ae863e8ef', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:56.557068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:56.560636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:56.564887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:56.564887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 19 20:50:56.564887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 19 20:50:56.564887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:56.564887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:56.564887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 19 20:50:56.564887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.Conflict( May 19 20:50:56.564887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e0a24c9c28b24a46bc16c76ae863e8ef. May 19 20:50:56.564887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:56.566706 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 471/945] 10.222.0.88 () {60 vars in 1140 bytes} [Thu May 19 20:50:56 2022] POST /identity/v3/projects => generated 262 bytes in 277 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 19 20:50:56.569631 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15f03e-7bad-4ed3-a540-1ae98e49327c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:56.580952 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf8500-2cf0-47a4-9605-55d7b3e84254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:56.581750 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf8500-2cf0-47a4-9605-55d7b3e84254 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.582100 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf8500-2cf0-47a4-9605-55d7b3e84254 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.582942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf8500-2cf0-47a4-9605-55d7b3e84254 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.583861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf8500-2cf0-47a4-9605-55d7b3e84254 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:56.584289 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf8500-2cf0-47a4-9605-55d7b3e84254 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:56.586364 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf8500-2cf0-47a4-9605-55d7b3e84254 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:56.606180 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 475/946] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:50:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:56.608937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:56.611006 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf8500-2cf0-47a4-9605-55d7b3e84254 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:56.694645 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15f03e-7bad-4ed3-a540-1ae98e49327c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:56.733769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c9b0e-14f3-468d-b017-e1a7e83ba36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:56.734755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c9b0e-14f3-468d-b017-e1a7e83ba36f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.735042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c9b0e-14f3-468d-b017-e1a7e83ba36f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.735365 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c9b0e-14f3-468d-b017-e1a7e83ba36f None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.738317 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf8500-2cf0-47a4-9605-55d7b3e84254 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:56.743293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c9b0e-14f3-468d-b017-e1a7e83ba36f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:56.745220 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c9b0e-14f3-468d-b017-e1a7e83ba36f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-932c9b0e-14f3-468d-b017-e1a7e83ba36f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:50:56.761406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:56.762534 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:56.764184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:50:56.764184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:50:56.764184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:50:56.764184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:50:56.764184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:50:56.764184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:50:56.764184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:50:56.764184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:56.764184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 472/947] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:56.769327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c9b0e-14f3-468d-b017-e1a7e83ba36f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:56.771473 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:56.781973 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d971dd1-f82a-4e9f-a239-3e67efdd57c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:56.783129 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971dd1-f82a-4e9f-a239-3e67efdd57c1 None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.783240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971dd1-f82a-4e9f-a239-3e67efdd57c1 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.783464 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d971dd1-f82a-4e9f-a239-3e67efdd57c1 None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/e44797dd45ca43bcbcbb8ac732423782/roles/5699d7a934b247b28e1e80867f31e61e` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.786662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d971dd1-f82a-4e9f-a239-3e67efdd57c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=e44797dd45ca43bcbcbb8ac732423782, role_id=5699d7a934b247b28e1e80867f31e61e)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:56.788561 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d971dd1-f82a-4e9f-a239-3e67efdd57c1 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:56.871053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:56.882758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 476/948] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:50:56 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/e44797dd45ca43bcbcbb8ac732423782/roles/5699d7a934b247b28e1e80867f31e61e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 19 20:50:56.884228 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973ef69-2fa0-4d1c-b38b-bb88371e0f52 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.884616 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973ef69-2fa0-4d1c-b38b-bb88371e0f52 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.885067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973ef69-2fa0-4d1c-b38b-bb88371e0f52 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.885308 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c9b0e-14f3-468d-b017-e1a7e83ba36f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:56.893665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 477/949] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:56.895275 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a266eb6-df98-462f-94bd-05938f352e08 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.895685 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a266eb6-df98-462f-94bd-05938f352e08 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.896047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a266eb6-df98-462f-94bd-05938f352e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.915575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-4a266eb6-df98-462f-94bd-05938f352e08 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:56.917072 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-4a266eb6-df98-462f-94bd-05938f352e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:56.953263 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905de897-0289-4bbf-9fad-4f99e680fc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:56.954448 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905de897-0289-4bbf-9fad-4f99e680fc05 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:56.954736 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905de897-0289-4bbf-9fad-4f99e680fc05 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:56.955022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905de897-0289-4bbf-9fad-4f99e680fc05 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:56.955860 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905de897-0289-4bbf-9fad-4f99e680fc05 None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:56.956118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905de897-0289-4bbf-9fad-4f99e680fc05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:56.957879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905de897-0289-4bbf-9fad-4f99e680fc05 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:56.981588 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 473/950] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:50:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:57.021859 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 478/951] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:57.046273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905de897-0289-4bbf-9fad-4f99e680fc05 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:57.170076 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905de897-0289-4bbf-9fad-4f99e680fc05 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:57.208857 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe20762-3d81-40d6-95d2-2e568010af0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:57.209584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe20762-3d81-40d6-95d2-2e568010af0a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:57.209867 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe20762-3d81-40d6-95d2-2e568010af0a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:57.210156 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe20762-3d81-40d6-95d2-2e568010af0a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:57.210809 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe20762-3d81-40d6-95d2-2e568010af0a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:57.212181 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe20762-3d81-40d6-95d2-2e568010af0a None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-9fe20762-3d81-40d6-95d2-2e568010af0a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:57.224772 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:57.225980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:57.227137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 474/952] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:50:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:57.235240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a266eb6-df98-462f-94bd-05938f352e08 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:57.345933 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a266eb6-df98-462f-94bd-05938f352e08 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:57.384714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:57.386912 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7da3b-3b90-4fb9-a04f-da6f64628bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:57.387591 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7da3b-3b90-4fb9-a04f-da6f64628bf3 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:57.391280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7da3b-3b90-4fb9-a04f-da6f64628bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:57.391280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7da3b-3b90-4fb9-a04f-da6f64628bf3 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:57.391280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7da3b-3b90-4fb9-a04f-da6f64628bf3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:57.391280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7da3b-3b90-4fb9-a04f-da6f64628bf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:57.391280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7da3b-3b90-4fb9-a04f-da6f64628bf3 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:57.396789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 479/953] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:50:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:57.402683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe20762-3d81-40d6-95d2-2e568010af0a None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:57.420014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:57.515209 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe20762-3d81-40d6-95d2-2e568010af0a None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:57.552300 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f8a92-daf2-46a4-b6c8-6b7110257854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:57.553001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f8a92-daf2-46a4-b6c8-6b7110257854 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:57.553285 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f8a92-daf2-46a4-b6c8-6b7110257854 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:57.553566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f8a92-daf2-46a4-b6c8-6b7110257854 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:57.554284 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f8a92-daf2-46a4-b6c8-6b7110257854 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:57.554598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f8a92-daf2-46a4-b6c8-6b7110257854 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:57.556135 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f8a92-daf2-46a4-b6c8-6b7110257854 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:57.568141 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7da3b-3b90-4fb9-a04f-da6f64628bf3 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:57.569434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 475/954] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:50:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:57.675889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7da3b-3b90-4fb9-a04f-da6f64628bf3 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:57.711164 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ba037-4997-4a27-8b0f-7b2b517e976f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:57.711832 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ba037-4997-4a27-8b0f-7b2b517e976f None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:57.712115 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ba037-4997-4a27-8b0f-7b2b517e976f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:57.712408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ba037-4997-4a27-8b0f-7b2b517e976f None admin] PATH_INFO: `/v3/endpoints` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:57.713309 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ba037-4997-4a27-8b0f-7b2b517e976f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:57.714706 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ba037-4997-4a27-8b0f-7b2b517e976f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:57.794081 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 480/955] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:50:57 2022] POST /identity/v3/endpoints => generated 349 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:50:58.518462 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d3e0-c593-4bf6-bbec-561e547f9d08 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:58.519084 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d3e0-c593-4bf6-bbec-561e547f9d08 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:58.519084 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d3e0-c593-4bf6-bbec-561e547f9d08 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:58.520719 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 476/956] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:58.526059 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917951e7-13ce-48aa-93dd-19ae9a9a08d3 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:58.526482 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917951e7-13ce-48aa-93dd-19ae9a9a08d3 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:58.526662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917951e7-13ce-48aa-93dd-19ae9a9a08d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:58.544247 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-917951e7-13ce-48aa-93dd-19ae9a9a08d3 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:58.545557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-917951e7-13ce-48aa-93dd-19ae9a9a08d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:58.575570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:58.622698 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 481/957] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:58.645212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d3e0-c593-4bf6-bbec-561e547f9d08 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:58.731621 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:58.755863 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d3e0-c593-4bf6-bbec-561e547f9d08 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:58.793435 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d710a-15c1-41c0-874c-5443e2c73803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:58.794247 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d710a-15c1-41c0-874c-5443e2c73803 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:58.794591 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d710a-15c1-41c0-874c-5443e2c73803 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:58.794871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397d710a-15c1-41c0-874c-5443e2c73803 None admin] PATH_INFO: `/v3/services/compute` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:58.795579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d710a-15c1-41c0-874c-5443e2c73803 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:58.797054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397d710a-15c1-41c0-874c-5443e2c73803 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-397d710a-15c1-41c0-874c-5443e2c73803 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:58.809364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:58.810614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:58.811947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 477/958] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:50:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:58.816941 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917951e7-13ce-48aa-93dd-19ae9a9a08d3 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:58.920756 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917951e7-13ce-48aa-93dd-19ae9a9a08d3 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:58.958634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1b914-f80e-408f-9a5a-138e227b444c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:58.959314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1b914-f80e-408f-9a5a-138e227b444c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:58.959595 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1b914-f80e-408f-9a5a-138e227b444c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:58.959962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1b914-f80e-408f-9a5a-138e227b444c None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:58.961093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1b914-f80e-408f-9a5a-138e227b444c None admin] RBAC: Adding query filter params (name=compute) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:58.961406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1b914-f80e-408f-9a5a-138e227b444c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:58.963190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1b914-f80e-408f-9a5a-138e227b444c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:58.968839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 482/959] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:50:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:58.974141 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d710a-15c1-41c0-874c-5443e2c73803 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:59.083569 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397d710a-15c1-41c0-874c-5443e2c73803 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:59.126016 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2e920-b731-43d3-8116-a7cf2cd115bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:59.126712 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2e920-b731-43d3-8116-a7cf2cd115bc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:59.126998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2e920-b731-43d3-8116-a7cf2cd115bc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:59.127278 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2e920-b731-43d3-8116-a7cf2cd115bc None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:59.128011 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2e920-b731-43d3-8116-a7cf2cd115bc None admin] RBAC: Adding query filter params (type=compute) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:59.128323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2e920-b731-43d3-8116-a7cf2cd115bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:59.129821 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2e920-b731-43d3-8116-a7cf2cd115bc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:59.136007 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 478/960] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:50:58 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:59.309905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1327f-7a99-4e22-93c1-30a00f9716ad None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:59.310298 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1327f-7a99-4e22-93c1-30a00f9716ad None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:59.310474 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b1327f-7a99-4e22-93c1-30a00f9716ad None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:59.312177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 483/961] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:59.318581 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c2ee8-1418-4e8c-bd16-be665463f9ff None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:59.318881 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c2ee8-1418-4e8c-bd16-be665463f9ff None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:59.319144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c2ee8-1418-4e8c-bd16-be665463f9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:59.325699 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:59.344838 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-ed6c2ee8-1418-4e8c-bd16-be665463f9ff None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:59.344838 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-ed6c2ee8-1418-4e8c-bd16-be665463f9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:59.413055 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 479/962] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:59.432023 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9350cbc-f17d-4da7-a132-13d49dca22ac None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:59.432383 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9350cbc-f17d-4da7-a132-13d49dca22ac None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:59.432635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9350cbc-f17d-4da7-a132-13d49dca22ac None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:59.434454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 484/963] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:50:59.437519 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c2ee8-1418-4e8c-bd16-be665463f9ff None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:59.440539 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734d046-9161-4dff-a9b0-a8a981d91858 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:59.440836 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734d046-9161-4dff-a9b0-a8a981d91858 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:59.441123 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734d046-9161-4dff-a9b0-a8a981d91858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:59.459659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-7734d046-9161-4dff-a9b0-a8a981d91858 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:50:59.461014 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-7734d046-9161-4dff-a9b0-a8a981d91858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:50:59.484288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:59.538781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 485/964] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:50:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:50:59.544365 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c2ee8-1418-4e8c-bd16-be665463f9ff None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:59.565108 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734d046-9161-4dff-a9b0-a8a981d91858 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:59.581456 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9408c-0a97-4dab-ad41-dc630b95ab7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:59.582289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9408c-0a97-4dab-ad41-dc630b95ab7d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:59.582575 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9408c-0a97-4dab-ad41-dc630b95ab7d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:59.582857 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9408c-0a97-4dab-ad41-dc630b95ab7d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:59.584847 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:59.594261 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9408c-0a97-4dab-ad41-dc630b95ab7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-3bb9408c-0a97-4dab-ad41-dc630b95ab7d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:50:59.602560 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:50:59.603871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:50:59.605014 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 480/965] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:50:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:59.613352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9408c-0a97-4dab-ad41-dc630b95ab7d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:59.683684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7734d046-9161-4dff-a9b0-a8a981d91858 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:59.722338 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9408c-0a97-4dab-ad41-dc630b95ab7d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:59.726023 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bf55e-5615-4516-83c1-3f76a70ba898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:59.726983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf55e-5615-4516-83c1-3f76a70ba898 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:59.727460 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf55e-5615-4516-83c1-3f76a70ba898 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:59.727937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bf55e-5615-4516-83c1-3f76a70ba898 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:59.728865 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bf55e-5615-4516-83c1-3f76a70ba898 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:59.730625 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bf55e-5615-4516-83c1-3f76a70ba898 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:59.732282 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: WARNING keystone.server.flask.common [None req-101bf55e-5615-4516-83c1-3f76a70ba898 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 19 20:50:59.761525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a0bde-8031-42a8-945a-1c9782518d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:59.762211 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a0bde-8031-42a8-945a-1c9782518d0f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:59.762499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a0bde-8031-42a8-945a-1c9782518d0f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:59.762782 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a0bde-8031-42a8-945a-1c9782518d0f None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:59.763550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a0bde-8031-42a8-945a-1c9782518d0f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:50:59.763854 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a0bde-8031-42a8-945a-1c9782518d0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:59.765264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a0bde-8031-42a8-945a-1c9782518d0f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:59.774531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 481/966] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:50:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:50:59.779132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.sql.core [None req-101bf55e-5615-4516-83c1-3f76a70ba898 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:50:59.779132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:50:59.779132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [parameters: {'user_id': '543c5c8d054a4af8bb2e44ed08d1e3a0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:50:59.779132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=278915) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 19 20:50:59.779917 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a0bde-8031-42a8-945a-1c9782518d0f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-101bf55e-5615-4516-83c1-3f76a70ba898 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application conn.query(q) May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application result.read() May 19 20:50:59.801752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:50:59.808318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application next(self.gen) May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application yield resource May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application next(self.gen) May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self.session.rollback() May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self.force_reraise() May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise self.value May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application session.commit() May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 19 20:50:59.809767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._prepare_impl() May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self.session.flush() May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._flush(objects) May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application compat.raise_( May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application flush_context.execute() May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rec.execute(self) May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 19 20:50:59.810826 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application _emit_insert_statements( May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application result = connection._execute_20( May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ret = self._execute_context( May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 19 20:50:59.811906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self.dialect.do_execute( May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application result = self._query(query) May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application conn.query(q) May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application result.read() May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 19 20:50:59.814613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application packet.raise_for_error() May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [parameters: {'user_id': '543c5c8d054a4af8bb2e44ed08d1e3a0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:50:59.817983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 19 20:50:59.820216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 19 20:50:59.823475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 19 20:50:59.823475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 19 20:50:59.823475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 19 20:50:59.823475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 19 20:50:59.823475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 19 20:50:59.823475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:50:59.823475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 486/967] 10.222.0.88 () {60 vars in 1131 bytes} [Thu May 19 20:50:59 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 19 20:50:59.823475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bf55e-5615-4516-83c1-3f76a70ba898 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:50:59.838533 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:59.871402 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:50:59.938869 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bf55e-5615-4516-83c1-3f76a70ba898 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:59.940809 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a0bde-8031-42a8-945a-1c9782518d0f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:50:59.978534 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff82b9-1931-44eb-b409-c6174b27a56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:59.979346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff82b9-1931-44eb-b409-c6174b27a56b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:59.979630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff82b9-1931-44eb-b409-c6174b27a56b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:59.979908 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff82b9-1931-44eb-b409-c6174b27a56b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:50:59.988620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff82b9-1931-44eb-b409-c6174b27a56b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:50:59.990297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff82b9-1931-44eb-b409-c6174b27a56b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:50:59.994059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73367-892f-45e0-8b21-9b10cd57c177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:50:59.995145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73367-892f-45e0-8b21-9b10cd57c177 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:50:59.995667 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73367-892f-45e0-8b21-9b10cd57c177 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:50:59.996183 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a73367-892f-45e0-8b21-9b10cd57c177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-61ff82b9-1931-44eb-b409-c6174b27a56b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:51:00.006758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:00.008036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:00.009188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:00.009188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:00.009188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:51:00.009188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:51:00.009188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:51:00.009188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:51:00.009188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:51:00.009188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:51:00.009188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 19 20:51:00.009188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:00.012981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 487/968] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:50:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:00.014393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff82b9-1931-44eb-b409-c6174b27a56b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:00.016294 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a73367-892f-45e0-8b21-9b10cd57c177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:00.018381 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a73367-892f-45e0-8b21-9b10cd57c177 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-d8a73367-892f-45e0-8b21-9b10cd57c177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:51:00.045518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:00.046683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:00.048318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:51:00.048318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:51:00.048318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:00.048318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:00.050197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 482/969] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:00.055599 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73367-892f-45e0-8b21-9b10cd57c177 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:00.124873 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:00.156109 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff82b9-1931-44eb-b409-c6174b27a56b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:00.178157 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a73367-892f-45e0-8b21-9b10cd57c177 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:00.190222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:00.209351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944268bb-0efb-48c5-a5fe-b886527acb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:00.210187 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944268bb-0efb-48c5-a5fe-b886527acb31 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:00.210476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944268bb-0efb-48c5-a5fe-b886527acb31 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:00.210758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944268bb-0efb-48c5-a5fe-b886527acb31 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:00.211674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944268bb-0efb-48c5-a5fe-b886527acb31 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:00.212114 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944268bb-0efb-48c5-a5fe-b886527acb31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:00.214274 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944268bb-0efb-48c5-a5fe-b886527acb31 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:00.221184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3eab4-c1d5-4686-9cdd-13d622c7459e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:00.222155 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3eab4-c1d5-4686-9cdd-13d622c7459e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:00.222685 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3eab4-c1d5-4686-9cdd-13d622c7459e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:00.223354 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3eab4-c1d5-4686-9cdd-13d622c7459e None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:00.224359 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3eab4-c1d5-4686-9cdd-13d622c7459e None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:00.224906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3eab4-c1d5-4686-9cdd-13d622c7459e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:00.226574 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3eab4-c1d5-4686-9cdd-13d622c7459e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:00.243817 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 483/970] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:00.243817 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25249f14-92f4-420b-bd7f-a3d69c0e2ba1 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:00.243817 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25249f14-92f4-420b-bd7f-a3d69c0e2ba1 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:00.243817 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25249f14-92f4-420b-bd7f-a3d69c0e2ba1 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:00.248700 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 484/971] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:00.249704 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25249f14-92f4-420b-bd7f-a3d69c0e2ba1 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:00.258350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 488/972] 10.222.0.88 () {56 vars in 1101 bytes} [Thu May 19 20:51:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:00.261351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8fa6c-c8a5-48c8-95bc-4abf1b5238c1 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:00.262632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8fa6c-c8a5-48c8-95bc-4abf1b5238c1 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:00.263121 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa8fa6c-c8a5-48c8-95bc-4abf1b5238c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:00.288764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-4fa8fa6c-c8a5-48c8-95bc-4abf1b5238c1 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:00.291391 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-4fa8fa6c-c8a5-48c8-95bc-4abf1b5238c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:00.389953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25249f14-92f4-420b-bd7f-a3d69c0e2ba1 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:00.390193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 489/973] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:00.409469 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8fa6c-c8a5-48c8-95bc-4abf1b5238c1 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:00.437144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f8b99-c008-465c-9c85-c52070690829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:00.439528 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8b99-c008-465c-9c85-c52070690829 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:00.440021 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8b99-c008-465c-9c85-c52070690829 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:00.440527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f8b99-c008-465c-9c85-c52070690829 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:00.451705 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f8b99-c008-465c-9c85-c52070690829 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:00.454108 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f8b99-c008-465c-9c85-c52070690829 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-a37f8b99-c008-465c-9c85-c52070690829 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:51:00.471006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:00.472191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:00.473395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:00.473395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:00.473395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:51:00.473395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:51:00.473395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:51:00.473395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:51:00.473395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:51:00.473395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:51:00.473395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 19 20:51:00.473395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:00.475472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 485/974] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:51:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:00.481048 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f8b99-c008-465c-9c85-c52070690829 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:00.492940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:00.527230 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:00.535685 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa8fa6c-c8a5-48c8-95bc-4abf1b5238c1 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:00.585133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d6206-19aa-460a-9a96-6280620371a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:00.586250 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d6206-19aa-460a-9a96-6280620371a0 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:00.586546 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d6206-19aa-460a-9a96-6280620371a0 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:00.586838 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d6206-19aa-460a-9a96-6280620371a0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:00.587666 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d6206-19aa-460a-9a96-6280620371a0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:00.589649 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d6206-19aa-460a-9a96-6280620371a0 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-391d6206-19aa-460a-9a96-6280620371a0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:00.607481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:00.608889 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:00.610919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 490/975] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:51:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:00.615489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d6206-19aa-460a-9a96-6280620371a0 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:00.630249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f8b99-c008-465c-9c85-c52070690829 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:00.670766 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6b4db-eaac-435b-a2e1-0e4841f01838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:00.671363 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6b4db-eaac-435b-a2e1-0e4841f01838 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:00.671648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6b4db-eaac-435b-a2e1-0e4841f01838 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:00.672523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6b4db-eaac-435b-a2e1-0e4841f01838 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:00.673374 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6b4db-eaac-435b-a2e1-0e4841f01838 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:00.673685 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6b4db-eaac-435b-a2e1-0e4841f01838 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:00.675270 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6b4db-eaac-435b-a2e1-0e4841f01838 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:00.694625 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 486/976] 10.222.0.88 () {56 vars in 1149 bytes} [Thu May 19 20:51:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:00.700303 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6b4db-eaac-435b-a2e1-0e4841f01838 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:00.725713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d6206-19aa-460a-9a96-6280620371a0 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:00.756925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:00.763665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb652b24-c127-4ef3-ad2d-2c893405cbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:00.764705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb652b24-c127-4ef3-ad2d-2c893405cbe3 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:00.765228 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb652b24-c127-4ef3-ad2d-2c893405cbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:00.765767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb652b24-c127-4ef3-ad2d-2c893405cbe3 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:00.766661 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb652b24-c127-4ef3-ad2d-2c893405cbe3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:00.770514 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb652b24-c127-4ef3-ad2d-2c893405cbe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:00.770514 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb652b24-c127-4ef3-ad2d-2c893405cbe3 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:00.774489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 491/977] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:51:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:00.779705 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb652b24-c127-4ef3-ad2d-2c893405cbe3 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:00.798094 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:00.813446 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6b4db-eaac-435b-a2e1-0e4841f01838 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:00.849079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b329b01-a447-4eb0-a5a3-dbe9f6b9ddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:00.849808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b329b01-a447-4eb0-a5a3-dbe9f6b9ddcc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:00.850095 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b329b01-a447-4eb0-a5a3-dbe9f6b9ddcc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:00.850374 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b329b01-a447-4eb0-a5a3-dbe9f6b9ddcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:00.901584 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb652b24-c127-4ef3-ad2d-2c893405cbe3 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:00.928431 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b329b01-a447-4eb0-a5a3-dbe9f6b9ddcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:00.929804 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b329b01-a447-4eb0-a5a3-dbe9f6b9ddcc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:00.943957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1348f25-678c-4db9-b1e3-0db814cccb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:00.944681 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1348f25-678c-4db9-b1e3-0db814cccb56 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:00.944976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1348f25-678c-4db9-b1e3-0db814cccb56 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:00.945264 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1348f25-678c-4db9-b1e3-0db814cccb56 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:00.945995 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1348f25-678c-4db9-b1e3-0db814cccb56 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:00.946312 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1348f25-678c-4db9-b1e3-0db814cccb56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:00.947720 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1348f25-678c-4db9-b1e3-0db814cccb56 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:00.952870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 492/978] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:51:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:00.959279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1348f25-678c-4db9-b1e3-0db814cccb56 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:01.023560 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:01.027943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:01.048969 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 487/979] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:51:00 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:51:01.050090 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b329b01-a447-4eb0-a5a3-dbe9f6b9ddcc None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:01.077334 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1348f25-678c-4db9-b1e3-0db814cccb56 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:01.115165 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca02528-4659-49bd-9d61-0f38e423cfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:01.115838 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca02528-4659-49bd-9d61-0f38e423cfd5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:01.116128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca02528-4659-49bd-9d61-0f38e423cfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:01.116669 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca02528-4659-49bd-9d61-0f38e423cfd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:01.117275 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca02528-4659-49bd-9d61-0f38e423cfd5 None admin] RBAC: Adding query filter params (interface=admin, service_id=e20d855b43d143a789bb9629f7828731, region_id=RegionOne) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:01.117626 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca02528-4659-49bd-9d61-0f38e423cfd5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:01.118994 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca02528-4659-49bd-9d61-0f38e423cfd5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:01.128738 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 493/980] 10.222.0.88 () {56 vars in 1249 bytes} [Thu May 19 20:51:00 2022] GET /identity/v3/endpoints?service_id=e20d855b43d143a789bb9629f7828731&interface=admin®ion_id=RegionOne => generated 193 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:01.129773 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca02528-4659-49bd-9d61-0f38e423cfd5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:01.165212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b329b01-a447-4eb0-a5a3-dbe9f6b9ddcc None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:01.203858 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bb1f2-922e-4eb0-b08d-ba806a7a1825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:01.204604 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bb1f2-922e-4eb0-b08d-ba806a7a1825 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:01.204891 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bb1f2-922e-4eb0-b08d-ba806a7a1825 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:01.205168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bb1f2-922e-4eb0-b08d-ba806a7a1825 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:01.216603 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bb1f2-922e-4eb0-b08d-ba806a7a1825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:01.218249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bb1f2-922e-4eb0-b08d-ba806a7a1825 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:01.233943 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca02528-4659-49bd-9d61-0f38e423cfd5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-426bb1f2-922e-4eb0-b08d-ba806a7a1825 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:51:01.238079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:01.239131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:01.244904 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:51:01.244904 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:51:01.244904 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:01.244904 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:01.244904 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 488/981] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:01.244904 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bb1f2-922e-4eb0-b08d-ba806a7a1825 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:01.264276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:01.277280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72df41-fca8-43b3-bc4f-cabe3daf7e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:01.283039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72df41-fca8-43b3-bc4f-cabe3daf7e55 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:01.283039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72df41-fca8-43b3-bc4f-cabe3daf7e55 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:01.283039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72df41-fca8-43b3-bc4f-cabe3daf7e55 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:01.283039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72df41-fca8-43b3-bc4f-cabe3daf7e55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:01.283039 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72df41-fca8-43b3-bc4f-cabe3daf7e55 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:01.287444 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:01.300375 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 494/982] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:51:01 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 19 20:51:01.359536 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bb1f2-922e-4eb0-b08d-ba806a7a1825 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:01.400764 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865f773-5619-4da1-a33f-f3f2b691fe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:01.401462 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865f773-5619-4da1-a33f-f3f2b691fe2b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:01.401749 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865f773-5619-4da1-a33f-f3f2b691fe2b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:01.402029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865f773-5619-4da1-a33f-f3f2b691fe2b None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:01.402932 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865f773-5619-4da1-a33f-f3f2b691fe2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:01.403249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865f773-5619-4da1-a33f-f3f2b691fe2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:01.404768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865f773-5619-4da1-a33f-f3f2b691fe2b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:01.421221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 489/983] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:01.428492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72df41-fca8-43b3-bc4f-cabe3daf7e55 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:01.543157 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72df41-fca8-43b3-bc4f-cabe3daf7e55 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:01.546202 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad755b-7353-4932-a304-1c681f32c118 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:01.546509 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad755b-7353-4932-a304-1c681f32c118 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:01.546785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad755b-7353-4932-a304-1c681f32c118 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:01.549012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 490/984] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:51:01.554184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a5f5f-747f-432d-8f05-bdb61dad0039 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:01.554547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a5f5f-747f-432d-8f05-bdb61dad0039 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:01.554893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a5f5f-747f-432d-8f05-bdb61dad0039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:01.573725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-d91a5f5f-747f-432d-8f05-bdb61dad0039 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:01.575041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-d91a5f5f-747f-432d-8f05-bdb61dad0039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:01.583139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e323817-3621-40b2-9485-7e55331fdced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:01.583955 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e323817-3621-40b2-9485-7e55331fdced None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:01.584307 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e323817-3621-40b2-9485-7e55331fdced None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:01.584619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e323817-3621-40b2-9485-7e55331fdced None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:01.591956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e323817-3621-40b2-9485-7e55331fdced None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:01.593476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e323817-3621-40b2-9485-7e55331fdced None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-1e323817-3621-40b2-9485-7e55331fdced None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:51:01.608545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:01.610194 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:01.611279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:51:01.611279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:51:01.611279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:51:01.611279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:51:01.611279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:51:01.611279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:51:01.611279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:51:01.611279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:01.611279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 495/985] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:51:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:01.616047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e323817-3621-40b2-9485-7e55331fdced None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:01.662743 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 491/986] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:51:01.666435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:01.687140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a5f5f-747f-432d-8f05-bdb61dad0039 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:01.721189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:01.730620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e323817-3621-40b2-9485-7e55331fdced None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:01.767756 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfe3a2-dcef-44c2-80c5-73c2a6202f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:01.769741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfe3a2-dcef-44c2-80c5-73c2a6202f68 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:01.769741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfe3a2-dcef-44c2-80c5-73c2a6202f68 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:01.769741 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cfe3a2-dcef-44c2-80c5-73c2a6202f68 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:01.770110 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfe3a2-dcef-44c2-80c5-73c2a6202f68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:01.770276 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfe3a2-dcef-44c2-80c5-73c2a6202f68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:01.771732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cfe3a2-dcef-44c2-80c5-73c2a6202f68 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:01.783995 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 496/987] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:51:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:01.789872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfe3a2-dcef-44c2-80c5-73c2a6202f68 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:01.810154 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a5f5f-747f-432d-8f05-bdb61dad0039 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:01.849732 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da1696-bf9f-43a6-94b6-2800005a9ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:01.850413 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1696-bf9f-43a6-94b6-2800005a9ec1 None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:01.850701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1696-bf9f-43a6-94b6-2800005a9ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:01.850982 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1696-bf9f-43a6-94b6-2800005a9ec1 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:01.851736 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da1696-bf9f-43a6-94b6-2800005a9ec1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:01.853168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da1696-bf9f-43a6-94b6-2800005a9ec1 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:01.904664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cfe3a2-dcef-44c2-80c5-73c2a6202f68 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:01.947211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:01.947211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ff70b-d7f8-4907-958a-553545fc1c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:01.947211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ff70b-d7f8-4907-958a-553545fc1c8a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:01.947211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ff70b-d7f8-4907-958a-553545fc1c8a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:01.948279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ff70b-d7f8-4907-958a-553545fc1c8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:01.948279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ff70b-d7f8-4907-958a-553545fc1c8a None admin] RBAC: Adding query filter params (role.id=5699d7a934b247b28e1e80867f31e61e, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=e44797dd45ca43bcbcbb8ac732423782) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:01.948279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ff70b-d7f8-4907-958a-553545fc1c8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:01.948651 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ff70b-d7f8-4907-958a-553545fc1c8a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:01.954815 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 497/988] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:51:01 2022] GET /identity/v3/role_assignments?user.id=e44797dd45ca43bcbcbb8ac732423782&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=5699d7a934b247b28e1e80867f31e61e => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:02.003718 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 492/989] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:51:01 2022] POST /identity/v3/services => generated 242 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:51:03.568454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1968d-1135-4265-9a43-b78f458629da None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:03.569035 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1968d-1135-4265-9a43-b78f458629da None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:03.569035 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1968d-1135-4265-9a43-b78f458629da None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:03.570466 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 498/990] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:03.575970 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acbd28-1545-4f69-866a-c8e2015855b4 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:03.576427 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acbd28-1545-4f69-866a-c8e2015855b4 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:03.576636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acbd28-1545-4f69-866a-c8e2015855b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:03.594176 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-42acbd28-1545-4f69-866a-c8e2015855b4 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:03.595370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-42acbd28-1545-4f69-866a-c8e2015855b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:03.672813 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 493/991] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:03.696481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1968d-1135-4265-9a43-b78f458629da None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:03.807633 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1968d-1135-4265-9a43-b78f458629da None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:03.844625 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc97b4c-ade0-4afa-ae02-a6006f08fec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:03.845345 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc97b4c-ade0-4afa-ae02-a6006f08fec8 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:03.845628 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc97b4c-ade0-4afa-ae02-a6006f08fec8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:03.845899 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc97b4c-ade0-4afa-ae02-a6006f08fec8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:03.846557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc97b4c-ade0-4afa-ae02-a6006f08fec8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:03.847959 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc97b4c-ade0-4afa-ae02-a6006f08fec8 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-adc97b4c-ade0-4afa-ae02-a6006f08fec8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:03.860359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:03.861641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 19 20:51:03.862861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:03.866617 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 499/992] 10.222.0.88 () {56 vars in 1126 bytes} [Thu May 19 20:51:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:03.867936 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acbd28-1545-4f69-866a-c8e2015855b4 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:03.884095 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:03.978390 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acbd28-1545-4f69-866a-c8e2015855b4 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:04.013109 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c9cc4-6e9b-4949-a2d1-0674f20f0e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:04.013787 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c9cc4-6e9b-4949-a2d1-0674f20f0e0f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.014073 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c9cc4-6e9b-4949-a2d1-0674f20f0e0f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.014350 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c9cc4-6e9b-4949-a2d1-0674f20f0e0f None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.015054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c9cc4-6e9b-4949-a2d1-0674f20f0e0f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:04.015364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c9cc4-6e9b-4949-a2d1-0674f20f0e0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:04.017306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c9cc4-6e9b-4949-a2d1-0674f20f0e0f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:04.022514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 494/993] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:51:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:04.028148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc97b4c-ade0-4afa-ae02-a6006f08fec8 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:04.068751 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:04.149164 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc97b4c-ade0-4afa-ae02-a6006f08fec8 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:04.189358 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea80cd-38f5-4046-8b6c-fb1cf0b12b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:04.190035 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea80cd-38f5-4046-8b6c-fb1cf0b12b63 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.190322 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea80cd-38f5-4046-8b6c-fb1cf0b12b63 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.190602 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea80cd-38f5-4046-8b6c-fb1cf0b12b63 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.191327 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea80cd-38f5-4046-8b6c-fb1cf0b12b63 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:04.191641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea80cd-38f5-4046-8b6c-fb1cf0b12b63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:04.193085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea80cd-38f5-4046-8b6c-fb1cf0b12b63 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:04.198500 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 500/994] 10.222.0.88 () {56 vars in 1122 bytes} [Thu May 19 20:51:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:04.204503 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c9cc4-6e9b-4949-a2d1-0674f20f0e0f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:04.261759 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae8ce8-0d77-40cd-aa7e-cbcd1450a3d0 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.262126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae8ce8-0d77-40cd-aa7e-cbcd1450a3d0 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.262323 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae8ce8-0d77-40cd-aa7e-cbcd1450a3d0 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.263855 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 501/995] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:51:04.269184 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791a48-41f4-4dd7-b3d7-549e158d2b40 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.269478 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791a48-41f4-4dd7-b3d7-549e158d2b40 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.269763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f791a48-41f4-4dd7-b3d7-549e158d2b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.288752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-2f791a48-41f4-4dd7-b3d7-549e158d2b40 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:04.289861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:04.290279 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-2f791a48-41f4-4dd7-b3d7-549e158d2b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:04.311404 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c9cc4-6e9b-4949-a2d1-0674f20f0e0f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:04.346831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2237f9-01ec-4df0-a046-b9e15a52a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:04.347515 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2237f9-01ec-4df0-a046-b9e15a52a064 None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.348212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2237f9-01ec-4df0-a046-b9e15a52a064 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.348592 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2237f9-01ec-4df0-a046-b9e15a52a064 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.349347 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2237f9-01ec-4df0-a046-b9e15a52a064 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:04.350756 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2237f9-01ec-4df0-a046-b9e15a52a064 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:04.359531 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 502/996] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:04 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:51:04.361043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406f5f6-6b9c-40a3-963b-bd403c5c3a7a None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.361336 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406f5f6-6b9c-40a3-963b-bd403c5c3a7a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.361612 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406f5f6-6b9c-40a3-963b-bd403c5c3a7a None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.362891 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 503/997] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:04.368463 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed312b-2350-4729-bb40-0bb98e29b5f1 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.373548 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed312b-2350-4729-bb40-0bb98e29b5f1 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.373548 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed312b-2350-4729-bb40-0bb98e29b5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.386398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-98ed312b-2350-4729-bb40-0bb98e29b5f1 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:04.387593 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-98ed312b-2350-4729-bb40-0bb98e29b5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:04.412284 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:04.435188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 495/998] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:51:04 2022] POST /identity/v3/endpoints => generated 325 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:51:04.449171 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2237f9-01ec-4df0-a046-b9e15a52a064 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:04.469362 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 504/999] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:04 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:51:04.495432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed312b-2350-4729-bb40-0bb98e29b5f1 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:04.589659 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2237f9-01ec-4df0-a046-b9e15a52a064 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:04.616256 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed312b-2350-4729-bb40-0bb98e29b5f1 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:04.630911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d2c98-9b38-4a3b-8142-20e8fb2cabf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:04.631825 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d2c98-9b38-4a3b-8142-20e8fb2cabf0 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.633334 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d2c98-9b38-4a3b-8142-20e8fb2cabf0 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.633334 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d2c98-9b38-4a3b-8142-20e8fb2cabf0 None admin] PATH_INFO: `/v3/services/image` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.633570 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d2c98-9b38-4a3b-8142-20e8fb2cabf0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:04.634946 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d2c98-9b38-4a3b-8142-20e8fb2cabf0 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-407d2c98-9b38-4a3b-8142-20e8fb2cabf0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:04.648594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:04.649879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:04.650902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 496/1000] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 20:51:04 2022] GET /identity/v3/services/image => generated 86 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:04.657320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d2c98-9b38-4a3b-8142-20e8fb2cabf0 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:04.660819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:04.662455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2779975-80a2-4483-a86e-0329529fafd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:04.663200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2779975-80a2-4483-a86e-0329529fafd8 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.663486 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2779975-80a2-4483-a86e-0329529fafd8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.663761 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2779975-80a2-4483-a86e-0329529fafd8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.664504 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2779975-80a2-4483-a86e-0329529fafd8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:04.665909 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2779975-80a2-4483-a86e-0329529fafd8 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-d2779975-80a2-4483-a86e-0329529fafd8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:04.679484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:04.680824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:04.682209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 505/1001] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:51:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:04.687139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2779975-80a2-4483-a86e-0329529fafd8 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:04.708248 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:04.779862 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d2c98-9b38-4a3b-8142-20e8fb2cabf0 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:04.796042 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2779975-80a2-4483-a86e-0329529fafd8 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:04.820539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f27b42-908b-4dcb-8800-62e9409032f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:04.821229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f27b42-908b-4dcb-8800-62e9409032f1 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.821491 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f27b42-908b-4dcb-8800-62e9409032f1 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.821766 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f27b42-908b-4dcb-8800-62e9409032f1 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.822472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f27b42-908b-4dcb-8800-62e9409032f1 None admin] RBAC: Adding query filter params (name=image) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:04.822777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f27b42-908b-4dcb-8800-62e9409032f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:04.824139 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f27b42-908b-4dcb-8800-62e9409032f1 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:04.834668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 497/1002] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:04.835774 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f27b42-908b-4dcb-8800-62e9409032f1 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:04.835914 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657502c-78b0-4e84-8d1e-dc01de79825c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:04.836480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657502c-78b0-4e84-8d1e-dc01de79825c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.836772 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657502c-78b0-4e84-8d1e-dc01de79825c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.837056 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657502c-78b0-4e84-8d1e-dc01de79825c None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.837777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657502c-78b0-4e84-8d1e-dc01de79825c None admin] RBAC: Adding query filter params (name=compute) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:04.838088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657502c-78b0-4e84-8d1e-dc01de79825c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:04.840618 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657502c-78b0-4e84-8d1e-dc01de79825c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:04.851405 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 506/1003] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:51:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:04.853319 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657502c-78b0-4e84-8d1e-dc01de79825c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:04.907649 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:04.945265 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f27b42-908b-4dcb-8800-62e9409032f1 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:04.951614 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:04.971647 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657502c-78b0-4e84-8d1e-dc01de79825c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:04.983121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67659020-85ce-47c3-97ab-153f85df871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:04.983805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67659020-85ce-47c3-97ab-153f85df871d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:04.984152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67659020-85ce-47c3-97ab-153f85df871d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:04.984495 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67659020-85ce-47c3-97ab-153f85df871d None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:04.985222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67659020-85ce-47c3-97ab-153f85df871d None admin] RBAC: Adding query filter params (type=image) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:04.985526 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67659020-85ce-47c3-97ab-153f85df871d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:04.986905 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67659020-85ce-47c3-97ab-153f85df871d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:04.992050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 498/1004] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:05.009228 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ab0c4-8802-49e1-a413-eb4a08c986f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:05.009924 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab0c4-8802-49e1-a413-eb4a08c986f4 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:05.010209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab0c4-8802-49e1-a413-eb4a08c986f4 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:05.010483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ab0c4-8802-49e1-a413-eb4a08c986f4 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:05.011217 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ab0c4-8802-49e1-a413-eb4a08c986f4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:05.011549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ab0c4-8802-49e1-a413-eb4a08c986f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:05.013027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ab0c4-8802-49e1-a413-eb4a08c986f4 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:05.019778 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 507/1005] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:51:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:05.025873 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67659020-85ce-47c3-97ab-153f85df871d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:05.142039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67659020-85ce-47c3-97ab-153f85df871d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:05.173146 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:05.182901 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde7ce0-914c-4585-b13a-618d53c42956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:05.183503 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde7ce0-914c-4585-b13a-618d53c42956 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:05.184539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde7ce0-914c-4585-b13a-618d53c42956 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:05.184539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde7ce0-914c-4585-b13a-618d53c42956 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:05.185146 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde7ce0-914c-4585-b13a-618d53c42956 None admin] RBAC: Adding query filter params (interface=public, service_id=62b38a6be3b142f285378d154bdbd405, region_id=RegionOne) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:05.185503 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde7ce0-914c-4585-b13a-618d53c42956 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:05.186837 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde7ce0-914c-4585-b13a-618d53c42956 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:05.193842 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 499/1006] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:51:05 2022] GET /identity/v3/endpoints?service_id=62b38a6be3b142f285378d154bdbd405&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:05.200975 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ab0c4-8802-49e1-a413-eb4a08c986f4 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:05.306757 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ab0c4-8802-49e1-a413-eb4a08c986f4 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:05.341635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11501b-6fa5-4a0c-8fba-cdfae610188e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:05.342306 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11501b-6fa5-4a0c-8fba-cdfae610188e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:05.342590 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11501b-6fa5-4a0c-8fba-cdfae610188e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:05.342867 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11501b-6fa5-4a0c-8fba-cdfae610188e None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:05.343552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11501b-6fa5-4a0c-8fba-cdfae610188e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:05.345002 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11501b-6fa5-4a0c-8fba-cdfae610188e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:05.351530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 508/1007] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:51:05 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:51:06.743406 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b815c-0f75-4403-a2fe-b4d10de39c1a None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:06.743979 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b815c-0f75-4403-a2fe-b4d10de39c1a None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:06.743979 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304b815c-0f75-4403-a2fe-b4d10de39c1a None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:06.745431 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 500/1008] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:06.750789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c4f39-82e6-4e23-bfbc-515b747c6266 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:06.751242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c4f39-82e6-4e23-bfbc-515b747c6266 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:06.751342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c4f39-82e6-4e23-bfbc-515b747c6266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:06.768434 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-437c4f39-82e6-4e23-bfbc-515b747c6266 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:06.769633 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-437c4f39-82e6-4e23-bfbc-515b747c6266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:06.772550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:06.841916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 509/1009] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:06 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:06.865730 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b815c-0f75-4403-a2fe-b4d10de39c1a None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:06.987793 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304b815c-0f75-4403-a2fe-b4d10de39c1a None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:07.025395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97bb070-c4f6-4c57-a4a4-44f922649725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:07.026105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bb070-c4f6-4c57-a4a4-44f922649725 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:07.026392 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bb070-c4f6-4c57-a4a4-44f922649725 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:07.026669 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97bb070-c4f6-4c57-a4a4-44f922649725 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:07.027906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bb070-c4f6-4c57-a4a4-44f922649725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:07.029370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97bb070-c4f6-4c57-a4a4-44f922649725 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:07.036202 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 501/1010] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:07.037462 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c4f39-82e6-4e23-bfbc-515b747c6266 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:07.149313 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c4f39-82e6-4e23-bfbc-515b747c6266 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:07.186408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d17506-afec-4ecf-9404-bf1a070e45ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:07.187088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d17506-afec-4ecf-9404-bf1a070e45ad None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:07.187973 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d17506-afec-4ecf-9404-bf1a070e45ad None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:07.188352 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d17506-afec-4ecf-9404-bf1a070e45ad None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:07.189087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d17506-afec-4ecf-9404-bf1a070e45ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:07.190556 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d17506-afec-4ecf-9404-bf1a070e45ad None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:07.201007 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:07.252042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f037a-8b04-48bb-aa42-d798e2d21bd2 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:07.252355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f037a-8b04-48bb-aa42-d798e2d21bd2 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:07.252673 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f037a-8b04-48bb-aa42-d798e2d21bd2 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:07.254075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 502/1011] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:51:07.259228 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7e2b5-b996-44a9-bf42-d571b42cbe3a None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:07.259544 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7e2b5-b996-44a9-bf42-d571b42cbe3a None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:07.259852 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c7e2b5-b996-44a9-bf42-d571b42cbe3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:07.277438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-21c7e2b5-b996-44a9-bf42-d571b42cbe3a None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:07.278632 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-21c7e2b5-b996-44a9-bf42-d571b42cbe3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:07.336428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 510/1012] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:51:07 2022] POST /identity/v3/projects => generated 311 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:51:07.352605 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:07.364229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 503/1013] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:07 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:51:07.387701 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d17506-afec-4ecf-9404-bf1a070e45ad None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:07.508870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d17506-afec-4ecf-9404-bf1a070e45ad None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:07.550695 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3245252-86c4-43ea-8c13-c8f3c2d6f5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:07.551419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3245252-86c4-43ea-8c13-c8f3c2d6f5ad None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:07.551690 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3245252-86c4-43ea-8c13-c8f3c2d6f5ad None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:07.551970 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3245252-86c4-43ea-8c13-c8f3c2d6f5ad None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:07.552865 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3245252-86c4-43ea-8c13-c8f3c2d6f5ad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:07.554252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3245252-86c4-43ea-8c13-c8f3c2d6f5ad None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:07.617100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b0928-9ea1-496d-bb88-14b1b30d07e2 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:07.617535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b0928-9ea1-496d-bb88-14b1b30d07e2 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:07.618358 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b0928-9ea1-496d-bb88-14b1b30d07e2 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:07.620365 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 504/1014] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:51:07.626496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461e7f0-de78-4680-ba41-ecfae7e28857 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:07.628109 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461e7f0-de78-4680-ba41-ecfae7e28857 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:07.628109 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461e7f0-de78-4680-ba41-ecfae7e28857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:07.643797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-6461e7f0-de78-4680-ba41-ecfae7e28857 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:07.645009 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-6461e7f0-de78-4680-ba41-ecfae7e28857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:07.656113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 511/1015] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:51:07 2022] POST /identity/v3/services => generated 242 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:51:07.734847 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 505/1016] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:07 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:51:07.758378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3245252-86c4-43ea-8c13-c8f3c2d6f5ad None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:07.825315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:07.882435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3245252-86c4-43ea-8c13-c8f3c2d6f5ad None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:07.922448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000b4c5-480e-437b-90fb-c80703f3cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:07.923553 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b4c5-480e-437b-90fb-c80703f3cb8e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:07.924011 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b4c5-480e-437b-90fb-c80703f3cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:07.924484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b4c5-480e-437b-90fb-c80703f3cb8e None admin] PATH_INFO: `/v3/services/compute` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:07.925394 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000b4c5-480e-437b-90fb-c80703f3cb8e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:07.926952 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000b4c5-480e-437b-90fb-c80703f3cb8e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-6000b4c5-480e-437b-90fb-c80703f3cb8e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:07.940402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:07.941632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:07.942671 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 512/1017] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:51:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:07.947987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461e7f0-de78-4680-ba41-ecfae7e28857 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:08.054289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461e7f0-de78-4680-ba41-ecfae7e28857 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:08.073125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:08.090618 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5293d41-39c8-4478-8735-ecd481ce5b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:08.091215 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5293d41-39c8-4478-8735-ecd481ce5b3a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:08.091497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5293d41-39c8-4478-8735-ecd481ce5b3a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:08.091776 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5293d41-39c8-4478-8735-ecd481ce5b3a None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:08.092569 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5293d41-39c8-4478-8735-ecd481ce5b3a None admin] RBAC: Adding query filter params (name=compute) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:08.092880 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5293d41-39c8-4478-8735-ecd481ce5b3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:08.094286 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5293d41-39c8-4478-8735-ecd481ce5b3a None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:08.104195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 506/1018] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:51:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:08.105509 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000b4c5-480e-437b-90fb-c80703f3cb8e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:08.237906 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000b4c5-480e-437b-90fb-c80703f3cb8e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:08.255714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:08.274280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8862d-fbc0-4485-9766-d755445e00e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:08.275236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8862d-fbc0-4485-9766-d755445e00e6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:08.275739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8862d-fbc0-4485-9766-d755445e00e6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:08.276216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8862d-fbc0-4485-9766-d755445e00e6 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:08.277154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8862d-fbc0-4485-9766-d755445e00e6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:08.277669 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8862d-fbc0-4485-9766-d755445e00e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:08.279247 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8862d-fbc0-4485-9766-d755445e00e6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:08.284664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 513/1019] 10.222.0.88 () {56 vars in 1112 bytes} [Thu May 19 20:51:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:08.290166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5293d41-39c8-4478-8735-ecd481ce5b3a None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:08.395751 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5293d41-39c8-4478-8735-ecd481ce5b3a None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:08.431637 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565433b7-e553-42ba-ae26-76b9289b0fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:08.432304 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565433b7-e553-42ba-ae26-76b9289b0fd6 None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:08.432622 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565433b7-e553-42ba-ae26-76b9289b0fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:08.432897 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565433b7-e553-42ba-ae26-76b9289b0fd6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:08.433641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565433b7-e553-42ba-ae26-76b9289b0fd6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:08.435269 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565433b7-e553-42ba-ae26-76b9289b0fd6 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:08.513479 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 507/1020] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:51:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:51:09.662510 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75251d5-f660-4125-a870-e21a4632bb57 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:09.663092 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75251d5-f660-4125-a870-e21a4632bb57 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:09.663092 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75251d5-f660-4125-a870-e21a4632bb57 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:09.664563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 514/1021] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:09.669827 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a43c17-026c-4eff-9600-85505852c420 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:09.670263 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a43c17-026c-4eff-9600-85505852c420 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:09.670447 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a43c17-026c-4eff-9600-85505852c420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:09.689136 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-24a43c17-026c-4eff-9600-85505852c420 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:09.690344 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-24a43c17-026c-4eff-9600-85505852c420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:09.722117 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:09.772382 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 508/1022] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:09.796132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75251d5-f660-4125-a870-e21a4632bb57 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:09.917355 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75251d5-f660-4125-a870-e21a4632bb57 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:09.953764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73169998-af84-485d-82d4-7bbc77a07311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:09.954561 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73169998-af84-485d-82d4-7bbc77a07311 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:09.954843 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73169998-af84-485d-82d4-7bbc77a07311 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:09.955117 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73169998-af84-485d-82d4-7bbc77a07311 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:09.956305 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73169998-af84-485d-82d4-7bbc77a07311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:09.957789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73169998-af84-485d-82d4-7bbc77a07311 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:09.959758 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 515/1023] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:09.962895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd56f6-e4d9-4ca1-b2c8-d7e6aa5bf7cc None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:09.963194 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd56f6-e4d9-4ca1-b2c8-d7e6aa5bf7cc None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:09.963482 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd56f6-e4d9-4ca1-b2c8-d7e6aa5bf7cc None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:09.965091 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73169998-af84-485d-82d4-7bbc77a07311 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:09.969439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 509/1024] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:09.971103 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec921703-717b-46d7-a758-3d0ea54381ac None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:09.971395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec921703-717b-46d7-a758-3d0ea54381ac None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:09.971679 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec921703-717b-46d7-a758-3d0ea54381ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:09.991992 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-ec921703-717b-46d7-a758-3d0ea54381ac None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:09.993230 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-ec921703-717b-46d7-a758-3d0ea54381ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:10.018722 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:10.080522 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 510/1025] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:10.080760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73169998-af84-485d-82d4-7bbc77a07311 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:10.093417 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec921703-717b-46d7-a758-3d0ea54381ac None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:10.117805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ec848-fef4-4f23-a6d9-811e50044c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:10.118562 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ec848-fef4-4f23-a6d9-811e50044c5e None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:10.118798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ec848-fef4-4f23-a6d9-811e50044c5e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:10.119044 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ec848-fef4-4f23-a6d9-811e50044c5e None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:10.119757 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ec848-fef4-4f23-a6d9-811e50044c5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:10.121289 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ec848-fef4-4f23-a6d9-811e50044c5e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:10.192110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:10.198411 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec921703-717b-46d7-a758-3d0ea54381ac None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:10.216552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 516/1026] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:51:09 2022] POST /identity/v3/users => generated 285 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 19 20:51:10.235276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538bfad-bf24-4073-a77d-a1e8f05e8301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:10.236138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538bfad-bf24-4073-a77d-a1e8f05e8301 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:10.236486 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538bfad-bf24-4073-a77d-a1e8f05e8301 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:10.236766 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d538bfad-bf24-4073-a77d-a1e8f05e8301 None admin] PATH_INFO: `/v3/services/image` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:10.237442 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538bfad-bf24-4073-a77d-a1e8f05e8301 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:10.239078 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d538bfad-bf24-4073-a77d-a1e8f05e8301 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-d538bfad-bf24-4073-a77d-a1e8f05e8301 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:10.258531 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:10.265314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:10.267012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 511/1027] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 20:51:10 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:10.270300 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ec848-fef4-4f23-a6d9-811e50044c5e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:10.398389 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ec848-fef4-4f23-a6d9-811e50044c5e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:10.402525 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:10.449545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59882fcc-4367-42f2-8ef3-fae761aa4aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:10.450150 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59882fcc-4367-42f2-8ef3-fae761aa4aa6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:10.450433 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59882fcc-4367-42f2-8ef3-fae761aa4aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:10.450711 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59882fcc-4367-42f2-8ef3-fae761aa4aa6 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:10.451435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59882fcc-4367-42f2-8ef3-fae761aa4aa6 None admin] RBAC: Adding query filter params (name=image) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:10.451745 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59882fcc-4367-42f2-8ef3-fae761aa4aa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:10.453763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59882fcc-4367-42f2-8ef3-fae761aa4aa6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:10.459264 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 517/1028] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:10.466134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538bfad-bf24-4073-a77d-a1e8f05e8301 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:10.577829 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d538bfad-bf24-4073-a77d-a1e8f05e8301 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:10.621733 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe78ea-74a4-4d93-9e2d-094eef4c1946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:10.622907 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe78ea-74a4-4d93-9e2d-094eef4c1946 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:10.623426 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe78ea-74a4-4d93-9e2d-094eef4c1946 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:10.623724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe78ea-74a4-4d93-9e2d-094eef4c1946 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:10.624480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe78ea-74a4-4d93-9e2d-094eef4c1946 None admin] RBAC: Adding query filter params (type=image) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:10.625359 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe78ea-74a4-4d93-9e2d-094eef4c1946 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:10.626756 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe78ea-74a4-4d93-9e2d-094eef4c1946 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:10.632394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 512/1029] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:10.638526 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59882fcc-4367-42f2-8ef3-fae761aa4aa6 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:10.754849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59882fcc-4367-42f2-8ef3-fae761aa4aa6 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:10.793675 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57786c-8b3b-43ba-8e20-774b3e84d906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:10.794340 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57786c-8b3b-43ba-8e20-774b3e84d906 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:10.794623 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57786c-8b3b-43ba-8e20-774b3e84d906 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:10.794898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57786c-8b3b-43ba-8e20-774b3e84d906 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:10.795700 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57786c-8b3b-43ba-8e20-774b3e84d906 None admin] RBAC: Adding query filter params (interface=public, service_id=5787f01f38b54708b9e3b1b2f4c3d919, region_id=RegionOne) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:10.796012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57786c-8b3b-43ba-8e20-774b3e84d906 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:10.798009 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57786c-8b3b-43ba-8e20-774b3e84d906 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:10.803629 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 518/1030] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:51:10 2022] GET /identity/v3/endpoints?service_id=5787f01f38b54708b9e3b1b2f4c3d919&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:10.809068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe78ea-74a4-4d93-9e2d-094eef4c1946 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:10.838043 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:10.919193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb7bb5-9325-412d-bdc1-bcdf0dfd8357 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:10.919513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb7bb5-9325-412d-bdc1-bcdf0dfd8357 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:10.919753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb7bb5-9325-412d-bdc1-bcdf0dfd8357 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:10.921171 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 519/1031] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:51:10.927164 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa3d94-e2ed-42a7-8835-57d7d0841463 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:10.927767 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa3d94-e2ed-42a7-8835-57d7d0841463 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:10.928292 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa3d94-e2ed-42a7-8835-57d7d0841463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:10.929279 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe78ea-74a4-4d93-9e2d-094eef4c1946 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:10.949773 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-1baa3d94-e2ed-42a7-8835-57d7d0841463 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:10.951079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-1baa3d94-e2ed-42a7-8835-57d7d0841463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:10.967994 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f889d-4bee-4164-965c-88637e7bf6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:10.969422 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f889d-4bee-4164-965c-88637e7bf6f0 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:10.969718 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f889d-4bee-4164-965c-88637e7bf6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:10.969999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f889d-4bee-4164-965c-88637e7bf6f0 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:10.970690 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f889d-4bee-4164-965c-88637e7bf6f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:10.972102 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f889d-4bee-4164-965c-88637e7bf6f0 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:10.979926 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 513/1032] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:51:10 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:51:10.983089 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:11.035204 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 520/1033] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:51:11.061324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f889d-4bee-4164-965c-88637e7bf6f0 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:11.161339 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:11.178998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f889d-4bee-4164-965c-88637e7bf6f0 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:11.216255 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2465cf-ef62-4129-b895-a43cd4c3c121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:11.217173 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2465cf-ef62-4129-b895-a43cd4c3c121 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:11.217446 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2465cf-ef62-4129-b895-a43cd4c3c121 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:11.217725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2465cf-ef62-4129-b895-a43cd4c3c121 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:11.228810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2465cf-ef62-4129-b895-a43cd4c3c121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-4b2465cf-ef62-4129-b895-a43cd4c3c121 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:51:11.237737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:11.238989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:11.240028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 514/1034] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:51:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:11.246543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa3d94-e2ed-42a7-8835-57d7d0841463 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:11.350393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa3d94-e2ed-42a7-8835-57d7d0841463 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:11.385668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671564c-ecec-4b2e-9e74-4b935b8d3f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:11.386350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671564c-ecec-4b2e-9e74-4b935b8d3f61 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:11.386638 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671564c-ecec-4b2e-9e74-4b935b8d3f61 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:11.386918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671564c-ecec-4b2e-9e74-4b935b8d3f61 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:11.387677 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671564c-ecec-4b2e-9e74-4b935b8d3f61 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:11.387988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671564c-ecec-4b2e-9e74-4b935b8d3f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:11.389420 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671564c-ecec-4b2e-9e74-4b935b8d3f61 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:11.398964 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 521/1035] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:51:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:11.404450 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2465cf-ef62-4129-b895-a43cd4c3c121 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:11.515848 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2465cf-ef62-4129-b895-a43cd4c3c121 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:11.552947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971796a-85aa-4d5b-8ce3-a071a4b7e3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:11.553746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971796a-85aa-4d5b-8ce3-a071a4b7e3e2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:11.554027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971796a-85aa-4d5b-8ce3-a071a4b7e3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:11.554301 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971796a-85aa-4d5b-8ce3-a071a4b7e3e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:11.566169 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971796a-85aa-4d5b-8ce3-a071a4b7e3e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:11.567618 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971796a-85aa-4d5b-8ce3-a071a4b7e3e2 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:11.574352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-5971796a-85aa-4d5b-8ce3-a071a4b7e3e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:51:11.585937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:11.587080 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:11.593044 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:51:11.593044 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:51:11.593044 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:11.593044 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:11.593044 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 515/1036] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:11.594775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671564c-ecec-4b2e-9e74-4b935b8d3f61 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:11.622334 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:11.699898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671564c-ecec-4b2e-9e74-4b935b8d3f61 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:11.738026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7d8a0-1e6f-4561-ad8c-758936204a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:11.738708 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7d8a0-1e6f-4561-ad8c-758936204a8a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:11.738999 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7d8a0-1e6f-4561-ad8c-758936204a8a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:11.739283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7d8a0-1e6f-4561-ad8c-758936204a8a None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:11.740018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7d8a0-1e6f-4561-ad8c-758936204a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:11.740350 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7d8a0-1e6f-4561-ad8c-758936204a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:11.741768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7d8a0-1e6f-4561-ad8c-758936204a8a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:11.758466 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 522/1037] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:11.759697 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971796a-85aa-4d5b-8ce3-a071a4b7e3e2 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:11.863833 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971796a-85aa-4d5b-8ce3-a071a4b7e3e2 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:11.902900 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79143e-2f19-4bf1-b24e-bb741a8cb246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:11.903627 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79143e-2f19-4bf1-b24e-bb741a8cb246 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:11.903916 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79143e-2f19-4bf1-b24e-bb741a8cb246 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:11.904197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb79143e-2f19-4bf1-b24e-bb741a8cb246 None admin] PATH_INFO: `/v3/users/nova` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:11.912672 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79143e-2f19-4bf1-b24e-bb741a8cb246 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:11.914116 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb79143e-2f19-4bf1-b24e-bb741a8cb246 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-cb79143e-2f19-4bf1-b24e-bb741a8cb246 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:51:11.929075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:11.930315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:11.936594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:11.936594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:11.936594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:51:11.936594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:51:11.936594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:51:11.936594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:51:11.936594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:51:11.936594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:51:11.936594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:51:11.936594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:11.936594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 516/1038] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:51:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:11.938175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7d8a0-1e6f-4561-ad8c-758936204a8a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:12.027019 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:12.045881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7d8a0-1e6f-4561-ad8c-758936204a8a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:12.082423 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef3ae4-72b6-4397-8691-add2c1073168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:12.083101 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef3ae4-72b6-4397-8691-add2c1073168 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:12.083392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef3ae4-72b6-4397-8691-add2c1073168 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:12.083675 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef3ae4-72b6-4397-8691-add2c1073168 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:12.084633 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef3ae4-72b6-4397-8691-add2c1073168 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:12.084952 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef3ae4-72b6-4397-8691-add2c1073168 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:12.086488 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef3ae4-72b6-4397-8691-add2c1073168 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:12.109528 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 523/1039] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:51:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:12.111133 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79143e-2f19-4bf1-b24e-bb741a8cb246 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:12.175006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:12.220432 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb79143e-2f19-4bf1-b24e-bb741a8cb246 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:12.255516 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41986913-40ee-4a28-8771-6361eef516e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:12.256175 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41986913-40ee-4a28-8771-6361eef516e2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:12.256489 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41986913-40ee-4a28-8771-6361eef516e2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:12.256764 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41986913-40ee-4a28-8771-6361eef516e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:12.327404 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41986913-40ee-4a28-8771-6361eef516e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:12.329511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41986913-40ee-4a28-8771-6361eef516e2 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:12.390238 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:12.438740 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 517/1040] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:51:12 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:51:12.444118 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef3ae4-72b6-4397-8691-add2c1073168 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:12.545780 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef3ae4-72b6-4397-8691-add2c1073168 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:12.577234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895cb0f-9675-43f8-bfbe-e63c609ec428 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:12.577567 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895cb0f-9675-43f8-bfbe-e63c609ec428 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:12.577812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895cb0f-9675-43f8-bfbe-e63c609ec428 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:12.579998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 518/1041] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:12.585088 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05968-9082-42d9-a9c9-455eba1d368e None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:12.585321 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f529fb3-f613-4826-9e3d-3995daa14238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:12.585439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05968-9082-42d9-a9c9-455eba1d368e None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:12.585741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da05968-9082-42d9-a9c9-455eba1d368e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:12.586010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529fb3-f613-4826-9e3d-3995daa14238 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:12.586300 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529fb3-f613-4826-9e3d-3995daa14238 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:12.586579 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f529fb3-f613-4826-9e3d-3995daa14238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:12.598420 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f529fb3-f613-4826-9e3d-3995daa14238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:12.599918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f529fb3-f613-4826-9e3d-3995daa14238 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:12.601918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:12.603345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-8da05968-9082-42d9-a9c9-455eba1d368e None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:12.604579 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-8da05968-9082-42d9-a9c9-455eba1d368e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-1f529fb3-f613-4826-9e3d-3995daa14238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:51:12.620645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:12.621835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:12.623013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:51:12.623013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:51:12.623013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:12.623013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:12.623013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 524/1042] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:12.631089 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f529fb3-f613-4826-9e3d-3995daa14238 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:12.688388 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 519/1043] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:51:12.711670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da05968-9082-42d9-a9c9-455eba1d368e None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:12.742998 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f529fb3-f613-4826-9e3d-3995daa14238 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:12.783644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24cba9-6866-42c4-8462-252bc0e79f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:12.784955 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cba9-6866-42c4-8462-252bc0e79f75 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:12.785244 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cba9-6866-42c4-8462-252bc0e79f75 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:12.785522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24cba9-6866-42c4-8462-252bc0e79f75 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:12.786261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24cba9-6866-42c4-8462-252bc0e79f75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:12.786571 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24cba9-6866-42c4-8462-252bc0e79f75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:12.788134 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24cba9-6866-42c4-8462-252bc0e79f75 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:12.801544 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 525/1044] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:12.806921 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24cba9-6866-42c4-8462-252bc0e79f75 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:12.813816 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:12.827883 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da05968-9082-42d9-a9c9-455eba1d368e None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:12.865827 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:12.866596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe04ef-71cd-4666-8e5c-9fd4c25b4e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:12.867399 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe04ef-71cd-4666-8e5c-9fd4c25b4e60 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:12.867676 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe04ef-71cd-4666-8e5c-9fd4c25b4e60 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:12.867948 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe04ef-71cd-4666-8e5c-9fd4c25b4e60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:12.879594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe04ef-71cd-4666-8e5c-9fd4c25b4e60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-9afe04ef-71cd-4666-8e5c-9fd4c25b4e60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:51:12.887254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:12.888769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:12.889810 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:12.893923 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 520/1045] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:51:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:12.894838 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe04ef-71cd-4666-8e5c-9fd4c25b4e60 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:12.921027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24cba9-6866-42c4-8462-252bc0e79f75 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:12.960079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed92ed2-3611-4306-a09b-3d03fe621448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:12.961093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed92ed2-3611-4306-a09b-3d03fe621448 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:12.961474 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed92ed2-3611-4306-a09b-3d03fe621448 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:12.961933 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed92ed2-3611-4306-a09b-3d03fe621448 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:12.970433 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed92ed2-3611-4306-a09b-3d03fe621448 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:12.972053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed92ed2-3611-4306-a09b-3d03fe621448 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-4ed92ed2-3611-4306-a09b-3d03fe621448 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:51:12.987471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:12.988897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:12.990196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:51:12.990196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:51:12.990196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:51:12.990196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:51:12.990196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:51:12.990196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:51:12.990196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:51:12.990196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:12.991061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 526/1046] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:51:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:12.996070 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed92ed2-3611-4306-a09b-3d03fe621448 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:13.008325 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe04ef-71cd-4666-8e5c-9fd4c25b4e60 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:13.045203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4220b0-4d81-4623-9cef-1f204c5d11d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:13.045889 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4220b0-4d81-4623-9cef-1f204c5d11d2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.046169 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4220b0-4d81-4623-9cef-1f204c5d11d2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.046446 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4220b0-4d81-4623-9cef-1f204c5d11d2 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.047747 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4220b0-4d81-4623-9cef-1f204c5d11d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:13.048054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4220b0-4d81-4623-9cef-1f204c5d11d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:13.049473 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4220b0-4d81-4623-9cef-1f204c5d11d2 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:13.064124 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 521/1047] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:51:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:13.065980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4220b0-4d81-4623-9cef-1f204c5d11d2 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:13.079950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:13.118984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed92ed2-3611-4306-a09b-3d03fe621448 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:13.136742 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:13.158824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abac631-533f-43ca-a350-78bdb27b3d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:13.159501 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abac631-533f-43ca-a350-78bdb27b3d99 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.159988 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abac631-533f-43ca-a350-78bdb27b3d99 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.160127 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abac631-533f-43ca-a350-78bdb27b3d99 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.160883 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abac631-533f-43ca-a350-78bdb27b3d99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:13.161202 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abac631-533f-43ca-a350-78bdb27b3d99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:13.162799 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abac631-533f-43ca-a350-78bdb27b3d99 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:13.182556 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 527/1048] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:51:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:13.184551 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abac631-533f-43ca-a350-78bdb27b3d99 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:13.185538 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4220b0-4d81-4623-9cef-1f204c5d11d2 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:13.225990 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e046c5-301a-4ada-9338-a865ced027ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:13.226793 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e046c5-301a-4ada-9338-a865ced027ee None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.227083 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e046c5-301a-4ada-9338-a865ced027ee None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.227362 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e046c5-301a-4ada-9338-a865ced027ee None admin] PATH_INFO: `/v3/users/a849036c316e47bc945b3184960627fb` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.243585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e046c5-301a-4ada-9338-a865ced027ee None admin] RBAC: Authorizing `identity:get_user(user_id=a849036c316e47bc945b3184960627fb)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:13.245071 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e046c5-301a-4ada-9338-a865ced027ee None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:13.246809 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 522/1049] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:13 2022] GET /identity/v3/users/a849036c316e47bc945b3184960627fb => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:13.253813 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e046c5-301a-4ada-9338-a865ced027ee None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:13.301357 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abac631-533f-43ca-a350-78bdb27b3d99 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:13.323046 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:13.336654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222a042-7b40-43fe-bd00-7bc63488c28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:13.337438 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222a042-7b40-43fe-bd00-7bc63488c28c None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.337772 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222a042-7b40-43fe-bd00-7bc63488c28c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.338012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222a042-7b40-43fe-bd00-7bc63488c28c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.338914 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222a042-7b40-43fe-bd00-7bc63488c28c None admin] RBAC: Adding query filter params (role.id=5699d7a934b247b28e1e80867f31e61e, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=77427e1028d349b4a740efe0a985e73b) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:13.339230 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222a042-7b40-43fe-bd00-7bc63488c28c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:13.340731 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8222a042-7b40-43fe-bd00-7bc63488c28c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:13.346984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 528/1050] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:51:13 2022] GET /identity/v3/role_assignments?user.id=77427e1028d349b4a740efe0a985e73b&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=5699d7a934b247b28e1e80867f31e61e => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:13.348577 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e80957-24dc-4aaf-9533-6a481a65c4f7 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.349018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e80957-24dc-4aaf-9533-6a481a65c4f7 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.349300 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e80957-24dc-4aaf-9533-6a481a65c4f7 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.350822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 529/1051] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:13.358307 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53074eba-567d-49da-b831-ccfdf6aeae75 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.358554 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e046c5-301a-4ada-9338-a865ced027ee None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:13.358655 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53074eba-567d-49da-b831-ccfdf6aeae75 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.358881 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53074eba-567d-49da-b831-ccfdf6aeae75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.377792 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-53074eba-567d-49da-b831-ccfdf6aeae75 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:13.379051 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-53074eba-567d-49da-b831-ccfdf6aeae75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:13.400975 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae611d-db42-4ef0-aab0-244f1f10d18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:13.401652 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae611d-db42-4ef0-aab0-244f1f10d18c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.401943 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae611d-db42-4ef0-aab0-244f1f10d18c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.402227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae611d-db42-4ef0-aab0-244f1f10d18c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.414399 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:13.461172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 530/1052] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:51:13.484751 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53074eba-567d-49da-b831-ccfdf6aeae75 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:13.492454 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae611d-db42-4ef0-aab0-244f1f10d18c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:13.494525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae611d-db42-4ef0-aab0-244f1f10d18c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:13.579715 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:13.597110 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53074eba-567d-49da-b831-ccfdf6aeae75 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:13.617220 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 523/1053] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:51:13 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:51:13.619676 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae611d-db42-4ef0-aab0-244f1f10d18c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:13.640185 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283a630-b79d-43ab-9cbf-e6cfe3f7bc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:13.641060 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283a630-b79d-43ab-9cbf-e6cfe3f7bc62 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.641349 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283a630-b79d-43ab-9cbf-e6cfe3f7bc62 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.641630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a283a630-b79d-43ab-9cbf-e6cfe3f7bc62 None admin] PATH_INFO: `/v3/services/image` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.642298 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283a630-b79d-43ab-9cbf-e6cfe3f7bc62 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:13.643724 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a283a630-b79d-43ab-9cbf-e6cfe3f7bc62 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-a283a630-b79d-43ab-9cbf-e6cfe3f7bc62 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:13.655846 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:13.657196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:13.658226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 531/1054] 10.222.0.88 () {56 vars in 1105 bytes} [Thu May 19 20:51:13 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:13.663058 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283a630-b79d-43ab-9cbf-e6cfe3f7bc62 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:13.698438 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:13.728705 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae611d-db42-4ef0-aab0-244f1f10d18c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:13.765413 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab9c9c-2014-4430-99f5-99e2ced2d6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:13.766660 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab9c9c-2014-4430-99f5-99e2ced2d6eb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.767189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab9c9c-2014-4430-99f5-99e2ced2d6eb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.767774 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab9c9c-2014-4430-99f5-99e2ced2d6eb None admin] PATH_INFO: `/v3/projects/a1365787d8534a4c8d69df1fc719689d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.769212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab9c9c-2014-4430-99f5-99e2ced2d6eb None admin] RBAC: Authorizing `identity:get_project(project_id=a1365787d8534a4c8d69df1fc719689d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:13.771074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab9c9c-2014-4430-99f5-99e2ced2d6eb None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:13.773097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 524/1055] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:51:13 2022] GET /identity/v3/projects/a1365787d8534a4c8d69df1fc719689d => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:13.774258 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a283a630-b79d-43ab-9cbf-e6cfe3f7bc62 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:13.779029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab9c9c-2014-4430-99f5-99e2ced2d6eb None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:13.812151 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b07d5c-83f5-45e6-9360-2fbf6a9ba6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:13.813628 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b07d5c-83f5-45e6-9360-2fbf6a9ba6e7 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.813773 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b07d5c-83f5-45e6-9360-2fbf6a9ba6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.813991 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b07d5c-83f5-45e6-9360-2fbf6a9ba6e7 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.814680 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b07d5c-83f5-45e6-9360-2fbf6a9ba6e7 None admin] RBAC: Adding query filter params (name=image) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:13.814992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b07d5c-83f5-45e6-9360-2fbf6a9ba6e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:13.816417 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b07d5c-83f5-45e6-9360-2fbf6a9ba6e7 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:13.818564 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:13.826068 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 532/1056] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:13.827169 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b07d5c-83f5-45e6-9360-2fbf6a9ba6e7 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:13.897882 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab9c9c-2014-4430-99f5-99e2ced2d6eb None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:13.937090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:13.939203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102719f-3dfc-48fd-ba43-5853db3db1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:13.939876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102719f-3dfc-48fd-ba43-5853db3db1f4 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.940265 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102719f-3dfc-48fd-ba43-5853db3db1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.940491 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102719f-3dfc-48fd-ba43-5853db3db1f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.941281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102719f-3dfc-48fd-ba43-5853db3db1f4 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=a1365787d8534a4c8d69df1fc719689d, user.id=a849036c316e47bc945b3184960627fb) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:13.941591 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102719f-3dfc-48fd-ba43-5853db3db1f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:13.943010 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102719f-3dfc-48fd-ba43-5853db3db1f4 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:13.943898 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b07d5c-83f5-45e6-9360-2fbf6a9ba6e7 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:13.960651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 525/1057] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:51:13 2022] GET /identity/v3/role_assignments?user.id=a849036c316e47bc945b3184960627fb&scope.project.id=a1365787d8534a4c8d69df1fc719689d&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:13.982956 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195f9ce-f5af-4c7d-8b8a-757aaf4fc7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:13.983648 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195f9ce-f5af-4c7d-8b8a-757aaf4fc7b8 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:13.983937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195f9ce-f5af-4c7d-8b8a-757aaf4fc7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:13.984326 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195f9ce-f5af-4c7d-8b8a-757aaf4fc7b8 None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:13.985109 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195f9ce-f5af-4c7d-8b8a-757aaf4fc7b8 None admin] RBAC: Adding query filter params (type=image) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:13.985467 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195f9ce-f5af-4c7d-8b8a-757aaf4fc7b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:13.986970 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195f9ce-f5af-4c7d-8b8a-757aaf4fc7b8 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:13.997957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 533/1058] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:13.999310 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102719f-3dfc-48fd-ba43-5853db3db1f4 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:14.112494 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:14.116057 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102719f-3dfc-48fd-ba43-5853db3db1f4 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:14.154909 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b179881-dba6-4c72-845a-2762ba7210cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:14.155581 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b179881-dba6-4c72-845a-2762ba7210cd None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:14.155881 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b179881-dba6-4c72-845a-2762ba7210cd None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:14.156158 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b179881-dba6-4c72-845a-2762ba7210cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:14.156991 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b179881-dba6-4c72-845a-2762ba7210cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:14.158326 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b179881-dba6-4c72-845a-2762ba7210cd None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:14.279805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 526/1059] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:51:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:51:15.600043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26c50-5c33-4682-8cb5-434e36b989fd None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:15.600732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26c50-5c33-4682-8cb5-434e36b989fd None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:15.600732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a26c50-5c33-4682-8cb5-434e36b989fd None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:15.602224 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 534/1060] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:15.607347 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc335a-5715-4a18-a1fd-3e95c05750c8 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:15.607763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc335a-5715-4a18-a1fd-3e95c05750c8 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:15.607940 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc335a-5715-4a18-a1fd-3e95c05750c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:15.625402 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-c0dc335a-5715-4a18-a1fd-3e95c05750c8 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:15.626590 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-c0dc335a-5715-4a18-a1fd-3e95c05750c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:15.702910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 527/1061] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:15.725868 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a26c50-5c33-4682-8cb5-434e36b989fd None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:15.835687 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a26c50-5c33-4682-8cb5-434e36b989fd None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:15.870968 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed84810-19bf-4016-989b-438b7e9dcbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:15.871706 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed84810-19bf-4016-989b-438b7e9dcbde None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:15.871990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed84810-19bf-4016-989b-438b7e9dcbde None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:15.872263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed84810-19bf-4016-989b-438b7e9dcbde None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:15.884018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed84810-19bf-4016-989b-438b7e9dcbde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-9ed84810-19bf-4016-989b-438b7e9dcbde None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:51:15.892981 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:15.894186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:51:15.895713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:15.899527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 535/1062] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:51:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:15.900203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc335a-5715-4a18-a1fd-3e95c05750c8 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:15.992631 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:16.014167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc335a-5715-4a18-a1fd-3e95c05750c8 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:16.052824 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff2537-32f1-4a3c-a3d2-dca75dd6018e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:16.053684 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff2537-32f1-4a3c-a3d2-dca75dd6018e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.053998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff2537-32f1-4a3c-a3d2-dca75dd6018e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.054294 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff2537-32f1-4a3c-a3d2-dca75dd6018e None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.055431 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff2537-32f1-4a3c-a3d2-dca75dd6018e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:16.056069 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff2537-32f1-4a3c-a3d2-dca75dd6018e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:16.057713 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff2537-32f1-4a3c-a3d2-dca75dd6018e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:16.071101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 528/1063] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:51:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:16.072945 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed84810-19bf-4016-989b-438b7e9dcbde None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:16.097677 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:16.179119 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed84810-19bf-4016-989b-438b7e9dcbde None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:16.194231 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4e782-6cfb-4c23-9021-47b1cfaa498c None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.194637 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4e782-6cfb-4c23-9021-47b1cfaa498c None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.194805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4e782-6cfb-4c23-9021-47b1cfaa498c None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.196487 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 529/1064] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:51:16.201661 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d46e8-94b2-46fe-8adc-abe4997a11bc None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.201956 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d46e8-94b2-46fe-8adc-abe4997a11bc None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.202299 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d46e8-94b2-46fe-8adc-abe4997a11bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.220728 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-229d46e8-94b2-46fe-8adc-abe4997a11bc None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:16.220931 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28858dd2-fc1f-41cd-bf26-5f6e9941d67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:16.221545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28858dd2-fc1f-41cd-bf26-5f6e9941d67b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.221839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28858dd2-fc1f-41cd-bf26-5f6e9941d67b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.221964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-229d46e8-94b2-46fe-8adc-abe4997a11bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:16.222139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28858dd2-fc1f-41cd-bf26-5f6e9941d67b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.232756 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28858dd2-fc1f-41cd-bf26-5f6e9941d67b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:16.234284 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28858dd2-fc1f-41cd-bf26-5f6e9941d67b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-28858dd2-fc1f-41cd-bf26-5f6e9941d67b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:51:16.253195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:16.254557 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:16.256373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:51:16.256373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:51:16.256373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:16.256373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:16.256373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 536/1065] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:16.260615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28858dd2-fc1f-41cd-bf26-5f6e9941d67b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:16.301727 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 530/1066] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:51:16.325390 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d46e8-94b2-46fe-8adc-abe4997a11bc None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:16.341789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:16.374229 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28858dd2-fc1f-41cd-bf26-5f6e9941d67b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:16.408305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:16.416007 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa05642-ef22-4772-9b23-67508f051e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:16.416766 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa05642-ef22-4772-9b23-67508f051e57 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.417103 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa05642-ef22-4772-9b23-67508f051e57 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.417335 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa05642-ef22-4772-9b23-67508f051e57 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.418080 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa05642-ef22-4772-9b23-67508f051e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:16.418391 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa05642-ef22-4772-9b23-67508f051e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:16.420068 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa05642-ef22-4772-9b23-67508f051e57 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:16.435295 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 537/1067] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:16.439768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d46e8-94b2-46fe-8adc-abe4997a11bc None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:16.442314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa05642-ef22-4772-9b23-67508f051e57 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:16.479713 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea143cd-934b-4a5a-9eee-cbece80e4d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:16.480602 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea143cd-934b-4a5a-9eee-cbece80e4d2e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.480888 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea143cd-934b-4a5a-9eee-cbece80e4d2e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.481185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea143cd-934b-4a5a-9eee-cbece80e4d2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.492034 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea143cd-934b-4a5a-9eee-cbece80e4d2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-eea143cd-934b-4a5a-9eee-cbece80e4d2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:51:16.499511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:16.501331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:16.503234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 531/1068] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:51:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:16.507152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea143cd-934b-4a5a-9eee-cbece80e4d2e None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:16.574786 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa05642-ef22-4772-9b23-67508f051e57 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:16.609643 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:16.615490 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0cc0cf-89e8-4b7c-b899-90f3c1a65cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:16.616299 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cc0cf-89e8-4b7c-b899-90f3c1a65cc1 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.616617 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cc0cf-89e8-4b7c-b899-90f3c1a65cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.616899 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0cc0cf-89e8-4b7c-b899-90f3c1a65cc1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.621773 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea143cd-934b-4a5a-9eee-cbece80e4d2e None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:16.625366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cc0cf-89e8-4b7c-b899-90f3c1a65cc1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:16.626848 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0cc0cf-89e8-4b7c-b899-90f3c1a65cc1 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-eb0cc0cf-89e8-4b7c-b899-90f3c1a65cc1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:51:16.641796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:16.643177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 538/1069] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:51:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:16.644569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ec209-2208-4d1e-a6a0-58708980c1f4 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.645523 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ec209-2208-4d1e-a6a0-58708980c1f4 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.645523 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ec209-2208-4d1e-a6a0-58708980c1f4 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.646488 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 539/1070] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:16.651309 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ec209-2208-4d1e-a6a0-58708980c1f4 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:16.660228 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be349d4-d0ef-4cc5-9777-135b6125dd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:16.660942 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be349d4-d0ef-4cc5-9777-135b6125dd78 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.661233 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be349d4-d0ef-4cc5-9777-135b6125dd78 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.661514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be349d4-d0ef-4cc5-9777-135b6125dd78 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.662270 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be349d4-d0ef-4cc5-9777-135b6125dd78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:16.662589 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be349d4-d0ef-4cc5-9777-135b6125dd78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:16.663967 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be349d4-d0ef-4cc5-9777-135b6125dd78 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:16.665401 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:16.673761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 532/1071] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:51:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:16.675705 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32820a-be49-403f-bdad-794c5ffa9df7 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.675705 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32820a-be49-403f-bdad-794c5ffa9df7 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.675705 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32820a-be49-403f-bdad-794c5ffa9df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.701120 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-6c32820a-be49-403f-bdad-794c5ffa9df7 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:16.702289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-6c32820a-be49-403f-bdad-794c5ffa9df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:16.760940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ec209-2208-4d1e-a6a0-58708980c1f4 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:16.770594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 533/1072] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:16.771443 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32820a-be49-403f-bdad-794c5ffa9df7 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:16.801046 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed90780-4511-48db-821e-0090d15ee99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:16.801726 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed90780-4511-48db-821e-0090d15ee99e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.802014 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed90780-4511-48db-821e-0090d15ee99e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.802294 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed90780-4511-48db-821e-0090d15ee99e None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.803148 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed90780-4511-48db-821e-0090d15ee99e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:16.803465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed90780-4511-48db-821e-0090d15ee99e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:16.806232 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed90780-4511-48db-821e-0090d15ee99e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:16.824477 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 540/1073] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:51:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:16.825234 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed90780-4511-48db-821e-0090d15ee99e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:16.877092 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:16.886225 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32820a-be49-403f-bdad-794c5ffa9df7 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:16.916194 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:16.925777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9133c-6892-4cb9-b672-9c6eb2df8156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:16.926677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9133c-6892-4cb9-b672-9c6eb2df8156 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.926966 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9133c-6892-4cb9-b672-9c6eb2df8156 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.927251 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9133c-6892-4cb9-b672-9c6eb2df8156 None admin] PATH_INFO: `/v3/users/a849036c316e47bc945b3184960627fb` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:16.927555 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed90780-4511-48db-821e-0090d15ee99e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:16.928535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9133c-6892-4cb9-b672-9c6eb2df8156 None admin] RBAC: Authorizing `identity:get_user(user_id=a849036c316e47bc945b3184960627fb)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:16.930060 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9133c-6892-4cb9-b672-9c6eb2df8156 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:16.931902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 534/1074] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:16 2022] GET /identity/v3/users/a849036c316e47bc945b3184960627fb => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:16.932751 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9133c-6892-4cb9-b672-9c6eb2df8156 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:16.963809 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d20c94-1730-4ed1-9683-123cce854b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:16.964419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d20c94-1730-4ed1-9683-123cce854b38 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:16.964713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d20c94-1730-4ed1-9683-123cce854b38 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:16.964995 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d20c94-1730-4ed1-9683-123cce854b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:17.032402 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d20c94-1730-4ed1-9683-123cce854b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:17.033874 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d20c94-1730-4ed1-9683-123cce854b38 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:17.034102 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9133c-6892-4cb9-b672-9c6eb2df8156 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:17.071474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b6bf1-c72d-4fd7-aeea-4f5c9d74c1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:17.072292 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b6bf1-c72d-4fd7-aeea-4f5c9d74c1c2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:17.072609 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b6bf1-c72d-4fd7-aeea-4f5c9d74c1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:17.072891 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b6bf1-c72d-4fd7-aeea-4f5c9d74c1c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:17.084187 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b6bf1-c72d-4fd7-aeea-4f5c9d74c1c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:17.085683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5b6bf1-c72d-4fd7-aeea-4f5c9d74c1c2 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:17.087369 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-ad5b6bf1-c72d-4fd7-aeea-4f5c9d74c1c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:51:17.102997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:17.104054 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:17.105271 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:51:17.105271 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:51:17.105271 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:17.105271 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:17.105271 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 535/1075] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:17.105271 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b6bf1-c72d-4fd7-aeea-4f5c9d74c1c2 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:17.145589 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 541/1076] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:51:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:51:17.146446 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d20c94-1730-4ed1-9683-123cce854b38 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:17.164416 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:17.219788 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b6bf1-c72d-4fd7-aeea-4f5c9d74c1c2 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:17.257796 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c71e5-542b-4f98-92fd-5ff6836d609e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:17.258803 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c71e5-542b-4f98-92fd-5ff6836d609e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:17.259184 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c71e5-542b-4f98-92fd-5ff6836d609e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:17.259474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c71e5-542b-4f98-92fd-5ff6836d609e None admin] PATH_INFO: `/v3/projects/a1365787d8534a4c8d69df1fc719689d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:17.260723 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c71e5-542b-4f98-92fd-5ff6836d609e None admin] RBAC: Authorizing `identity:get_project(project_id=a1365787d8534a4c8d69df1fc719689d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:17.261657 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d20c94-1730-4ed1-9683-123cce854b38 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:17.262499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15c71e5-542b-4f98-92fd-5ff6836d609e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:17.263916 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 536/1077] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:51:17 2022] GET /identity/v3/projects/a1365787d8534a4c8d69df1fc719689d => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:17.264890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c71e5-542b-4f98-92fd-5ff6836d609e None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:17.298987 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469c85c-9f2c-4a0e-8d07-9dc768309562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:17.299661 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469c85c-9f2c-4a0e-8d07-9dc768309562 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:17.299943 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469c85c-9f2c-4a0e-8d07-9dc768309562 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:17.300220 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469c85c-9f2c-4a0e-8d07-9dc768309562 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:17.300982 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469c85c-9f2c-4a0e-8d07-9dc768309562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:17.301296 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469c85c-9f2c-4a0e-8d07-9dc768309562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:17.302715 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9469c85c-9f2c-4a0e-8d07-9dc768309562 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:17.315027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 542/1078] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:17.315827 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469c85c-9f2c-4a0e-8d07-9dc768309562 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:17.329726 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:17.370820 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c71e5-542b-4f98-92fd-5ff6836d609e None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:17.385391 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:17.405929 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35426f-3f8b-4826-896b-81a754985644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:17.406790 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35426f-3f8b-4826-896b-81a754985644 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:17.407074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35426f-3f8b-4826-896b-81a754985644 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:17.407411 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35426f-3f8b-4826-896b-81a754985644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:17.418123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35426f-3f8b-4826-896b-81a754985644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:17.419565 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469c85c-9f2c-4a0e-8d07-9dc768309562 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:17.419691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35426f-3f8b-4826-896b-81a754985644 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-5b35426f-3f8b-4826-896b-81a754985644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:51:17.438959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:17.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:17.442557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:51:17.442557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:51:17.442557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:17.442557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:17.442557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 537/1079] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:17.442557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35426f-3f8b-4826-896b-81a754985644 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:17.459850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6d361-7633-4aee-901d-20614a87b64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:17.460771 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6d361-7633-4aee-901d-20614a87b64c None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:17.461056 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6d361-7633-4aee-901d-20614a87b64c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:17.461331 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6d361-7633-4aee-901d-20614a87b64c None admin] PATH_INFO: `/v3/projects/a1365787d8534a4c8d69df1fc719689d/users/a849036c316e47bc945b3184960627fb/roles/ed68c2195de147c0a30d9150b33e47cf` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:17.463323 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6d361-7633-4aee-901d-20614a87b64c None admin] RBAC: Authorizing `identity:create_grant(project_id=a1365787d8534a4c8d69df1fc719689d, user_id=a849036c316e47bc945b3184960627fb, role_id=ed68c2195de147c0a30d9150b33e47cf)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:17.464817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6d361-7633-4aee-901d-20614a87b64c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:17.545355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35426f-3f8b-4826-896b-81a754985644 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:17.551340 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 543/1080] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:51:17 2022] PUT /identity/v3/projects/a1365787d8534a4c8d69df1fc719689d/users/a849036c316e47bc945b3184960627fb/roles/ed68c2195de147c0a30d9150b33e47cf => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:51:17.552193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6d361-7633-4aee-901d-20614a87b64c None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:17.597449 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9fc0d-abfa-4ad8-86ee-2eeeeb934b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:17.598273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9fc0d-abfa-4ad8-86ee-2eeeeb934b46 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:17.598557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9fc0d-abfa-4ad8-86ee-2eeeeb934b46 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:17.598832 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9fc0d-abfa-4ad8-86ee-2eeeeb934b46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:17.606673 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9fc0d-abfa-4ad8-86ee-2eeeeb934b46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:17.608179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9fc0d-abfa-4ad8-86ee-2eeeeb934b46 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-26b9fc0d-abfa-4ad8-86ee-2eeeeb934b46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:51:17.624105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:17.626370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:17.627441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:51:17.627441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:51:17.627441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:51:17.627441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:51:17.627441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:51:17.627441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:51:17.627441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:51:17.627441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:17.627441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 538/1081] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:17.634649 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9fc0d-abfa-4ad8-86ee-2eeeeb934b46 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:17.651269 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:17.666977 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:17.689959 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6d361-7633-4aee-901d-20614a87b64c None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:17.730822 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92c803-341a-4393-b817-85158ca9c246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:17.731914 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92c803-341a-4393-b817-85158ca9c246 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:17.732371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92c803-341a-4393-b817-85158ca9c246 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:17.732789 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92c803-341a-4393-b817-85158ca9c246 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:17.733547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92c803-341a-4393-b817-85158ca9c246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:17.735022 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92c803-341a-4393-b817-85158ca9c246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:17.736139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c92c803-341a-4393-b817-85158ca9c246 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:17.751928 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 544/1082] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:17.756036 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9fc0d-abfa-4ad8-86ee-2eeeeb934b46 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:17.799653 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a5ac8-8c37-4bfc-889c-dd6dccdb3286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:17.800259 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a5ac8-8c37-4bfc-889c-dd6dccdb3286 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:17.800587 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a5ac8-8c37-4bfc-889c-dd6dccdb3286 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:17.800872 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a5ac8-8c37-4bfc-889c-dd6dccdb3286 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:17.801678 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a5ac8-8c37-4bfc-889c-dd6dccdb3286 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:17.801994 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a5ac8-8c37-4bfc-889c-dd6dccdb3286 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:17.803451 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a5ac8-8c37-4bfc-889c-dd6dccdb3286 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:17.822788 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 539/1083] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:51:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:17.824145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92c803-341a-4393-b817-85158ca9c246 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:17.949811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92c803-341a-4393-b817-85158ca9c246 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:17.985785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:17.991237 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe0f94-3792-4f59-a22b-c46effd2b68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:17.992269 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0f94-3792-4f59-a22b-c46effd2b68c None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:17.992691 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0f94-3792-4f59-a22b-c46effd2b68c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:17.993079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe0f94-3792-4f59-a22b-c46effd2b68c None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/77427e1028d349b4a740efe0a985e73b/roles/5699d7a934b247b28e1e80867f31e61e` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:17.995480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe0f94-3792-4f59-a22b-c46effd2b68c None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=77427e1028d349b4a740efe0a985e73b, role_id=5699d7a934b247b28e1e80867f31e61e)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:17.997130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbe0f94-3792-4f59-a22b-c46effd2b68c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:18.139405 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 545/1084] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:51:17 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/77427e1028d349b4a740efe0a985e73b/roles/5699d7a934b247b28e1e80867f31e61e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:51:19.795988 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a3d08-9cd2-46e5-b819-ee697df98e9c None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:19.797119 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a3d08-9cd2-46e5-b819-ee697df98e9c None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:19.797223 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5a3d08-9cd2-46e5-b819-ee697df98e9c None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:19.798584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 540/1085] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:19.804200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481ae1d-d707-4b78-86d0-b4ce960b605a None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:19.804679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481ae1d-d707-4b78-86d0-b4ce960b605a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:19.804936 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481ae1d-d707-4b78-86d0-b4ce960b605a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:19.822803 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-8481ae1d-d707-4b78-86d0-b4ce960b605a None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:19.824028 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-8481ae1d-d707-4b78-86d0-b4ce960b605a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:19.897568 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 546/1086] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:19.920815 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a3d08-9cd2-46e5-b819-ee697df98e9c None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:20.019367 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:20.035847 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5a3d08-9cd2-46e5-b819-ee697df98e9c None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:20.072452 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12460a85-253f-4d19-8120-fda82f7272ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:20.073245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460a85-253f-4d19-8120-fda82f7272ee None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:20.073530 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460a85-253f-4d19-8120-fda82f7272ee None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:20.073809 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12460a85-253f-4d19-8120-fda82f7272ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:20.085272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12460a85-253f-4d19-8120-fda82f7272ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-12460a85-253f-4d19-8120-fda82f7272ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:51:20.093110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:20.094542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:20.095642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 541/1087] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:51:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:20.100546 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481ae1d-d707-4b78-86d0-b4ce960b605a None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:20.205322 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481ae1d-d707-4b78-86d0-b4ce960b605a None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:20.225694 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:20.239561 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd1bfb-7a45-4db2-9b52-638231cef5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:20.240240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd1bfb-7a45-4db2-9b52-638231cef5f6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:20.240556 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd1bfb-7a45-4db2-9b52-638231cef5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:20.240837 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd1bfb-7a45-4db2-9b52-638231cef5f6 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:20.241592 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd1bfb-7a45-4db2-9b52-638231cef5f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:20.241902 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd1bfb-7a45-4db2-9b52-638231cef5f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:20.243334 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd1bfb-7a45-4db2-9b52-638231cef5f6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:20.251978 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 547/1088] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:51:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:20.257938 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12460a85-253f-4d19-8120-fda82f7272ee None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:20.359998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12460a85-253f-4d19-8120-fda82f7272ee None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:20.362263 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f069436-bf34-47d4-8d85-521df8a79e10 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:20.362554 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f069436-bf34-47d4-8d85-521df8a79e10 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:20.362836 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f069436-bf34-47d4-8d85-521df8a79e10 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:20.364227 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 548/1089] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:20.369177 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e07a10-be63-493e-881f-dc6d54341dcf None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:20.369464 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e07a10-be63-493e-881f-dc6d54341dcf None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:20.369743 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e07a10-be63-493e-881f-dc6d54341dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:20.386947 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-27e07a10-be63-493e-881f-dc6d54341dcf None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:20.388169 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-27e07a10-be63-493e-881f-dc6d54341dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:20.395308 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b689e-ee54-4615-b65e-c687c9c7a76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:20.396181 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b689e-ee54-4615-b65e-c687c9c7a76a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:20.396484 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b689e-ee54-4615-b65e-c687c9c7a76a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:20.396800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b689e-ee54-4615-b65e-c687c9c7a76a None admin] PATH_INFO: `/v3/users/a849036c316e47bc945b3184960627fb` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:20.398008 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b689e-ee54-4615-b65e-c687c9c7a76a None admin] RBAC: Authorizing `identity:get_user(user_id=a849036c316e47bc945b3184960627fb)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:20.399427 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b689e-ee54-4615-b65e-c687c9c7a76a None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:20.401327 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 542/1090] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:20 2022] GET /identity/v3/users/a849036c316e47bc945b3184960627fb => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:20.406827 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b689e-ee54-4615-b65e-c687c9c7a76a None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:20.423696 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:20.459589 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 549/1091] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 19 20:51:20.482897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e07a10-be63-493e-881f-dc6d54341dcf None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:20.512388 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b689e-ee54-4615-b65e-c687c9c7a76a None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:20.547216 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489362ec-6757-4494-889e-339132970164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:20.547876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489362ec-6757-4494-889e-339132970164 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:20.548160 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489362ec-6757-4494-889e-339132970164 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:20.548481 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489362ec-6757-4494-889e-339132970164 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:20.583894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e07a10-be63-493e-881f-dc6d54341dcf None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:20.597185 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:20.617963 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489362ec-6757-4494-889e-339132970164 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:20.618238 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bad99d-6b17-4140-a768-aa924e5e66fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:20.619052 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad99d-6b17-4140-a768-aa924e5e66fa None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:20.619337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad99d-6b17-4140-a768-aa924e5e66fa None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:20.619460 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489362ec-6757-4494-889e-339132970164 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:20.619614 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad99d-6b17-4140-a768-aa924e5e66fa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:20.630170 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bad99d-6b17-4140-a768-aa924e5e66fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:51:20.635274 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-a9bad99d-6b17-4140-a768-aa924e5e66fa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:51:20.638916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:20.640054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 19 20:51:20.641122 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:20.641996 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 550/1092] 10.222.0.88 () {56 vars in 1120 bytes} [Thu May 19 20:51:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:20.647658 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bad99d-6b17-4140-a768-aa924e5e66fa None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:20.720908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 543/1093] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:51:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 19 20:51:20.730173 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489362ec-6757-4494-889e-339132970164 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:20.749020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bad99d-6b17-4140-a768-aa924e5e66fa None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:20.783829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbc560-ff34-479b-8fc5-d9b0fd9e641b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:20.784540 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbc560-ff34-479b-8fc5-d9b0fd9e641b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:20.784827 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbc560-ff34-479b-8fc5-d9b0fd9e641b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:20.785116 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbc560-ff34-479b-8fc5-d9b0fd9e641b None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:20.785862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbc560-ff34-479b-8fc5-d9b0fd9e641b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:20.786170 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbc560-ff34-479b-8fc5-d9b0fd9e641b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:20.787581 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbc560-ff34-479b-8fc5-d9b0fd9e641b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:20.795757 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 551/1094] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:51:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:20.801875 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbc560-ff34-479b-8fc5-d9b0fd9e641b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:20.834285 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489362ec-6757-4494-889e-339132970164 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:20.834948 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:20.849008 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:20.869617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b4b77-e3b1-4eb3-beba-c95b70a7a852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:20.870428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b4b77-e3b1-4eb3-beba-c95b70a7a852 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:20.870707 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b4b77-e3b1-4eb3-beba-c95b70a7a852 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:20.870985 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b4b77-e3b1-4eb3-beba-c95b70a7a852 None admin] PATH_INFO: `/v3/projects/a1365787d8534a4c8d69df1fc719689d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:20.872166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b4b77-e3b1-4eb3-beba-c95b70a7a852 None admin] RBAC: Authorizing `identity:get_project(project_id=a1365787d8534a4c8d69df1fc719689d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:20.873637 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b4b77-e3b1-4eb3-beba-c95b70a7a852 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:20.879004 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 544/1095] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:51:20 2022] GET /identity/v3/projects/a1365787d8534a4c8d69df1fc719689d => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:20.881494 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b4b77-e3b1-4eb3-beba-c95b70a7a852 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:20.903433 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbc560-ff34-479b-8fc5-d9b0fd9e641b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:20.938446 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcc60c-b0ce-4811-885c-789e07fb48e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:20.939239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcc60c-b0ce-4811-885c-789e07fb48e6 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:20.939521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcc60c-b0ce-4811-885c-789e07fb48e6 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:20.939799 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcc60c-b0ce-4811-885c-789e07fb48e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:20.950224 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcc60c-b0ce-4811-885c-789e07fb48e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:20.951690 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcc60c-b0ce-4811-885c-789e07fb48e6 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-13bcc60c-b0ce-4811-885c-789e07fb48e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:51:20.970031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:20.971324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:20.972643 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:51:20.972643 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:51:20.972643 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:20.972643 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:20.972643 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 552/1096] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:51:20.977135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcc60c-b0ce-4811-885c-789e07fb48e6 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:20.986575 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b4b77-e3b1-4eb3-beba-c95b70a7a852 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:21.021757 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb7a7f-b3d4-4a6b-b473-45fae2f1722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:21.022430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb7a7f-b3d4-4a6b-b473-45fae2f1722d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:21.022718 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb7a7f-b3d4-4a6b-b473-45fae2f1722d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:21.022995 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb7a7f-b3d4-4a6b-b473-45fae2f1722d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:21.023914 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb7a7f-b3d4-4a6b-b473-45fae2f1722d None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=a1365787d8534a4c8d69df1fc719689d, user.id=a849036c316e47bc945b3184960627fb) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:21.024235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb7a7f-b3d4-4a6b-b473-45fae2f1722d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:21.025691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb7a7f-b3d4-4a6b-b473-45fae2f1722d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:21.032101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 545/1097] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:51:20 2022] GET /identity/v3/role_assignments?user.id=a849036c316e47bc945b3184960627fb&scope.project.id=a1365787d8534a4c8d69df1fc719689d&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:21.072195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:21.088213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcc60c-b0ce-4811-885c-789e07fb48e6 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:21.126089 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d6c8d-7c54-4572-978f-d27eae682c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:21.126677 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6c8d-7c54-4572-978f-d27eae682c71 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:21.126960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6c8d-7c54-4572-978f-d27eae682c71 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:21.127235 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6c8d-7c54-4572-978f-d27eae682c71 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:21.127970 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d6c8d-7c54-4572-978f-d27eae682c71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:21.128278 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d6c8d-7c54-4572-978f-d27eae682c71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:21.129760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d6c8d-7c54-4572-978f-d27eae682c71 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:21.143401 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 553/1098] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:21.149278 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb7a7f-b3d4-4a6b-b473-45fae2f1722d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:21.199173 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:21.255045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb7a7f-b3d4-4a6b-b473-45fae2f1722d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:21.295770 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210c0bd-c1d5-4d6f-b513-334179d2b6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:21.296651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210c0bd-c1d5-4d6f-b513-334179d2b6f4 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:21.296933 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210c0bd-c1d5-4d6f-b513-334179d2b6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:21.297208 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210c0bd-c1d5-4d6f-b513-334179d2b6f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:21.305197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210c0bd-c1d5-4d6f-b513-334179d2b6f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:21.306633 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210c0bd-c1d5-4d6f-b513-334179d2b6f4 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-3210c0bd-c1d5-4d6f-b513-334179d2b6f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:51:21.324596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:21.326045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:21.327264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:21.327264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:21.327264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:51:21.327264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:51:21.327264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:51:21.327264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:51:21.327264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:51:21.327264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:51:21.327264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 19 20:51:21.327264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:21.327264 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 546/1099] 10.222.0.88 () {56 vars in 1093 bytes} [Thu May 19 20:51:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:21.332314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d6c8d-7c54-4572-978f-d27eae682c71 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:21.438283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d6c8d-7c54-4572-978f-d27eae682c71 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:21.473079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d85f83-a7da-4fc0-b7ad-f0991114698f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:21.473797 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d85f83-a7da-4fc0-b7ad-f0991114698f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:21.474085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d85f83-a7da-4fc0-b7ad-f0991114698f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:21.474356 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d85f83-a7da-4fc0-b7ad-f0991114698f None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:21.475351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d85f83-a7da-4fc0-b7ad-f0991114698f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:21.475657 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d85f83-a7da-4fc0-b7ad-f0991114698f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:21.477149 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d85f83-a7da-4fc0-b7ad-f0991114698f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:21.494548 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 554/1100] 10.222.0.88 () {56 vars in 1133 bytes} [Thu May 19 20:51:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:21.501236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210c0bd-c1d5-4d6f-b513-334179d2b6f4 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:21.605449 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210c0bd-c1d5-4d6f-b513-334179d2b6f4 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:21.613306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:21.641115 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e8ebf-785a-47fd-867d-95970e2a398b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:21.641780 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e8ebf-785a-47fd-867d-95970e2a398b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:21.642127 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e8ebf-785a-47fd-867d-95970e2a398b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:21.642410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e8ebf-785a-47fd-867d-95970e2a398b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:21.714884 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e8ebf-785a-47fd-867d-95970e2a398b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:21.716253 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e8ebf-785a-47fd-867d-95970e2a398b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:21.817145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 547/1101] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:51:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:51:21.822897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d85f83-a7da-4fc0-b7ad-f0991114698f None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:21.826990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:21.927609 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d85f83-a7da-4fc0-b7ad-f0991114698f None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:21.963721 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccdf76-f92a-4faf-b71e-dd13677c432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:21.964556 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccdf76-f92a-4faf-b71e-dd13677c432d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:21.964849 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccdf76-f92a-4faf-b71e-dd13677c432d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:21.965185 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccdf76-f92a-4faf-b71e-dd13677c432d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:21.976098 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccdf76-f92a-4faf-b71e-dd13677c432d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:21.977634 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccdf76-f92a-4faf-b71e-dd13677c432d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-25ccdf76-f92a-4faf-b71e-dd13677c432d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:51:21.996222 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:22.002105 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:22.003277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:51:22.003277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:51:22.003277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:51:22.003277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:22.003277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 555/1102] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:22.003615 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e8ebf-785a-47fd-867d-95970e2a398b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:22.011793 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:22.108093 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e8ebf-785a-47fd-867d-95970e2a398b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:22.142982 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764cfb5-ee2e-47f8-bfa1-748c16d4af0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:22.143666 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764cfb5-ee2e-47f8-bfa1-748c16d4af0c None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:22.143953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764cfb5-ee2e-47f8-bfa1-748c16d4af0c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:22.144234 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764cfb5-ee2e-47f8-bfa1-748c16d4af0c None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:22.144995 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764cfb5-ee2e-47f8-bfa1-748c16d4af0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:22.145307 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764cfb5-ee2e-47f8-bfa1-748c16d4af0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:22.146686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764cfb5-ee2e-47f8-bfa1-748c16d4af0c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:22.159748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 548/1103] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:22.165388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccdf76-f92a-4faf-b71e-dd13677c432d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:22.225708 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:22.269785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccdf76-f92a-4faf-b71e-dd13677c432d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:22.303406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0c1b0-78e0-4972-9e8f-d99c760974bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:22.304210 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0c1b0-78e0-4972-9e8f-d99c760974bd None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:22.304528 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0c1b0-78e0-4972-9e8f-d99c760974bd None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:22.304812 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0c1b0-78e0-4972-9e8f-d99c760974bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:22.313045 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0c1b0-78e0-4972-9e8f-d99c760974bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:22.314542 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0c1b0-78e0-4972-9e8f-d99c760974bd None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-f8d0c1b0-78e0-4972-9e8f-d99c760974bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:51:22.329055 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:22.330342 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:22.331886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:51:22.331886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:51:22.331886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:51:22.331886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:51:22.331886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:51:22.331886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:51:22.331886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:51:22.331886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:22.331886 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 556/1104] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:22.336261 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764cfb5-ee2e-47f8-bfa1-748c16d4af0c None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:22.402845 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:22.436627 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764cfb5-ee2e-47f8-bfa1-748c16d4af0c None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:22.472106 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbf47b-be73-45aa-9cf4-b3f6cb8f5a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:22.472796 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbf47b-be73-45aa-9cf4-b3f6cb8f5a38 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:22.473084 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbf47b-be73-45aa-9cf4-b3f6cb8f5a38 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:22.473362 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbf47b-be73-45aa-9cf4-b3f6cb8f5a38 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:22.474153 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbf47b-be73-45aa-9cf4-b3f6cb8f5a38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:22.474466 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbf47b-be73-45aa-9cf4-b3f6cb8f5a38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:22.475935 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbf47b-be73-45aa-9cf4-b3f6cb8f5a38 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:22.488444 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 549/1105] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:22.494435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0c1b0-78e0-4972-9e8f-d99c760974bd None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:22.598120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0c1b0-78e0-4972-9e8f-d99c760974bd None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:22.615085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:22.633307 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba69e5e-9e48-4930-b031-6c04b8b4d09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:22.633973 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba69e5e-9e48-4930-b031-6c04b8b4d09e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:22.634261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba69e5e-9e48-4930-b031-6c04b8b4d09e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:22.634587 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba69e5e-9e48-4930-b031-6c04b8b4d09e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:22.635420 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba69e5e-9e48-4930-b031-6c04b8b4d09e None admin] RBAC: Adding query filter params (role.id=5699d7a934b247b28e1e80867f31e61e, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=77427e1028d349b4a740efe0a985e73b) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:22.635739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba69e5e-9e48-4930-b031-6c04b8b4d09e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:22.637230 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba69e5e-9e48-4930-b031-6c04b8b4d09e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:22.643184 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 557/1106] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:51:22 2022] GET /identity/v3/role_assignments?user.id=77427e1028d349b4a740efe0a985e73b&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=5699d7a934b247b28e1e80867f31e61e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:23.342465 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e65828-2383-47b8-9b4a-2c8848f67246 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:23.343009 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e65828-2383-47b8-9b4a-2c8848f67246 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:23.343112 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e65828-2383-47b8-9b4a-2c8848f67246 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:23.345053 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 550/1107] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:23.350641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f909a-3dfe-48b9-9b6c-f09c7578c0da None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:23.351132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f909a-3dfe-48b9-9b6c-f09c7578c0da None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:23.351240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f909a-3dfe-48b9-9b6c-f09c7578c0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:23.368627 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-385f909a-3dfe-48b9-9b6c-f09c7578c0da None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:23.369833 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-385f909a-3dfe-48b9-9b6c-f09c7578c0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:23.442985 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 558/1108] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:23.469724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e65828-2383-47b8-9b4a-2c8848f67246 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:23.574912 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e65828-2383-47b8-9b4a-2c8848f67246 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:23.601384 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:23.610052 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a75f8-4af7-413e-8bcd-168bb2bd2a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:23.610858 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a75f8-4af7-413e-8bcd-168bb2bd2a67 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:23.611145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a75f8-4af7-413e-8bcd-168bb2bd2a67 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:23.611422 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a75f8-4af7-413e-8bcd-168bb2bd2a67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:23.618431 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a75f8-4af7-413e-8bcd-168bb2bd2a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:23.618431 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a75f8-4af7-413e-8bcd-168bb2bd2a67 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:23.618431 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 551/1109] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:23.621842 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f909a-3dfe-48b9-9b6c-f09c7578c0da None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:23.725383 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f909a-3dfe-48b9-9b6c-f09c7578c0da None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:23.736652 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:23.762133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64ef03-3644-468f-8256-56d0d38b8642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:23.762809 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64ef03-3644-468f-8256-56d0d38b8642 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:23.763097 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64ef03-3644-468f-8256-56d0d38b8642 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:23.763374 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64ef03-3644-468f-8256-56d0d38b8642 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:23.764088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64ef03-3644-468f-8256-56d0d38b8642 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:23.766359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64ef03-3644-468f-8256-56d0d38b8642 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:23.896635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 559/1110] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:51:23 2022] POST /identity/v3/users => generated 286 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:51:26.187614 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcc65b-c87c-481c-86a5-e14c6a2aeebe None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:26.188254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcc65b-c87c-481c-86a5-e14c6a2aeebe None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:26.188254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcc65b-c87c-481c-86a5-e14c6a2aeebe None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:26.189480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 552/1111] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:26.194829 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecfb2a-26e9-4522-a9d6-20396fb83c46 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:26.195239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecfb2a-26e9-4522-a9d6-20396fb83c46 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:26.195397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecfb2a-26e9-4522-a9d6-20396fb83c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:26.212999 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-84ecfb2a-26e9-4522-a9d6-20396fb83c46 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:26.214207 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-84ecfb2a-26e9-4522-a9d6-20396fb83c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:26.281080 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 560/1112] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:26.305395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcc65b-c87c-481c-86a5-e14c6a2aeebe None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:26.407419 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcc65b-c87c-481c-86a5-e14c6a2aeebe None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:26.443602 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d303d-d2e9-4fd3-a1b8-718fe5b661ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:26.444429 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d303d-d2e9-4fd3-a1b8-718fe5b661ed None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:26.444721 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d303d-d2e9-4fd3-a1b8-718fe5b661ed None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:26.445004 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d303d-d2e9-4fd3-a1b8-718fe5b661ed None admin] PATH_INFO: `/v3/roles/4313b1c9df5345778a249b04355b90a0` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:26.446508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d303d-d2e9-4fd3-a1b8-718fe5b661ed None admin] RBAC: Authorizing `identity:get_role(role_id=4313b1c9df5345778a249b04355b90a0)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:26.447892 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d303d-d2e9-4fd3-a1b8-718fe5b661ed None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:26.449333 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 553/1113] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:26 2022] GET /identity/v3/roles/4313b1c9df5345778a249b04355b90a0 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:26.455317 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecfb2a-26e9-4522-a9d6-20396fb83c46 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:26.546149 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:26.552674 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecfb2a-26e9-4522-a9d6-20396fb83c46 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:26.586831 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b9b16-32a4-439e-af54-421f17ed51d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:26.587788 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b9b16-32a4-439e-af54-421f17ed51d5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:26.588076 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b9b16-32a4-439e-af54-421f17ed51d5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:26.588396 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b9b16-32a4-439e-af54-421f17ed51d5 None admin] PATH_INFO: `/v3/users/3b38caad337340898f7d2136703ba66b` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:26.603099 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b9b16-32a4-439e-af54-421f17ed51d5 None admin] RBAC: Authorizing `identity:get_user(user_id=3b38caad337340898f7d2136703ba66b)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:26.605158 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b9b16-32a4-439e-af54-421f17ed51d5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:26.607016 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 561/1114] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:26 2022] GET /identity/v3/users/3b38caad337340898f7d2136703ba66b => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:26.613440 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d303d-d2e9-4fd3-a1b8-718fe5b661ed None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:26.678776 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:26.808119 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d303d-d2e9-4fd3-a1b8-718fe5b661ed None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:26.842485 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9795456-4b7e-4dff-8413-c3f9afbf0feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:26.843632 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9795456-4b7e-4dff-8413-c3f9afbf0feb None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:26.843941 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9795456-4b7e-4dff-8413-c3f9afbf0feb None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:26.844215 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9795456-4b7e-4dff-8413-c3f9afbf0feb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:26.910508 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9795456-4b7e-4dff-8413-c3f9afbf0feb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:26.911939 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9795456-4b7e-4dff-8413-c3f9afbf0feb None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:26.972499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:27.010273 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 554/1115] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:51:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:51:27.016006 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b9b16-32a4-439e-af54-421f17ed51d5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:27.119078 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b9b16-32a4-439e-af54-421f17ed51d5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:27.153644 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321cd3e-fdda-4105-bbc1-62c043a80912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:27.154503 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321cd3e-fdda-4105-bbc1-62c043a80912 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:27.154786 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321cd3e-fdda-4105-bbc1-62c043a80912 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:27.155061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321cd3e-fdda-4105-bbc1-62c043a80912 None admin] PATH_INFO: `/v3/projects/a1365787d8534a4c8d69df1fc719689d` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:27.156810 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321cd3e-fdda-4105-bbc1-62c043a80912 None admin] RBAC: Authorizing `identity:get_project(project_id=a1365787d8534a4c8d69df1fc719689d)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:27.158259 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321cd3e-fdda-4105-bbc1-62c043a80912 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:27.159979 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 562/1116] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:51:27 2022] GET /identity/v3/projects/a1365787d8534a4c8d69df1fc719689d => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:27.166140 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9795456-4b7e-4dff-8413-c3f9afbf0feb None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:27.262893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9795456-4b7e-4dff-8413-c3f9afbf0feb None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:27.296776 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27d6af-f9d7-492f-852b-a1ab22d2c681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:27.297479 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27d6af-f9d7-492f-852b-a1ab22d2c681 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:27.297764 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27d6af-f9d7-492f-852b-a1ab22d2c681 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:27.298046 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27d6af-f9d7-492f-852b-a1ab22d2c681 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:27.298873 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27d6af-f9d7-492f-852b-a1ab22d2c681 None admin] RBAC: Adding query filter params (role.id=4313b1c9df5345778a249b04355b90a0, scope.project.id=a1365787d8534a4c8d69df1fc719689d, user.id=3b38caad337340898f7d2136703ba66b) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:27.299189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27d6af-f9d7-492f-852b-a1ab22d2c681 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:27.300631 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27d6af-f9d7-492f-852b-a1ab22d2c681 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:27.305959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 555/1117] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:51:27 2022] GET /identity/v3/role_assignments?user.id=3b38caad337340898f7d2136703ba66b&scope.project.id=a1365787d8534a4c8d69df1fc719689d&role.id=4313b1c9df5345778a249b04355b90a0 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:29.514598 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33123a4-993b-4803-9683-56d2bb0c8185 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:29.515782 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33123a4-993b-4803-9683-56d2bb0c8185 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:29.515782 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33123a4-993b-4803-9683-56d2bb0c8185 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:29.517079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 563/1118] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:29.522195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591b3b-0c1e-4d73-90b9-5eabcceb7e34 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:29.522587 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591b3b-0c1e-4d73-90b9-5eabcceb7e34 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:29.522751 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db591b3b-0c1e-4d73-90b9-5eabcceb7e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:29.539767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-db591b3b-0c1e-4d73-90b9-5eabcceb7e34 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:29.541045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-db591b3b-0c1e-4d73-90b9-5eabcceb7e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:29.560859 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:29.606892 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 556/1119] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:29.628948 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33123a4-993b-4803-9683-56d2bb0c8185 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:29.653543 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:29.732620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33123a4-993b-4803-9683-56d2bb0c8185 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:29.766132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954887e-3eb1-4c9f-bf63-2c8d64133433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:29.766924 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954887e-3eb1-4c9f-bf63-2c8d64133433 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:29.767211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954887e-3eb1-4c9f-bf63-2c8d64133433 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:29.767492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954887e-3eb1-4c9f-bf63-2c8d64133433 None admin] PATH_INFO: `/v3/roles/4313b1c9df5345778a249b04355b90a0` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:29.769003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954887e-3eb1-4c9f-bf63-2c8d64133433 None admin] RBAC: Authorizing `identity:get_role(role_id=4313b1c9df5345778a249b04355b90a0)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:29.770399 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954887e-3eb1-4c9f-bf63-2c8d64133433 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:29.771777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 564/1120] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:29 2022] GET /identity/v3/roles/4313b1c9df5345778a249b04355b90a0 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:29.777637 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db591b3b-0c1e-4d73-90b9-5eabcceb7e34 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:29.873675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db591b3b-0c1e-4d73-90b9-5eabcceb7e34 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:29.908019 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048fb78-0562-4c0f-8391-5c7ba698b75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:29.908923 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048fb78-0562-4c0f-8391-5c7ba698b75d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:29.909213 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048fb78-0562-4c0f-8391-5c7ba698b75d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:29.909494 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048fb78-0562-4c0f-8391-5c7ba698b75d None admin] PATH_INFO: `/v3/users/3b38caad337340898f7d2136703ba66b` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:29.910695 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048fb78-0562-4c0f-8391-5c7ba698b75d None admin] RBAC: Authorizing `identity:get_user(user_id=3b38caad337340898f7d2136703ba66b)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:29.912138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048fb78-0562-4c0f-8391-5c7ba698b75d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:29.913817 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 557/1121] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:29 2022] GET /identity/v3/users/3b38caad337340898f7d2136703ba66b => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:29.919502 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954887e-3eb1-4c9f-bf63-2c8d64133433 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:30.009317 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:30.018571 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954887e-3eb1-4c9f-bf63-2c8d64133433 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:30.053101 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc387fa0-35c0-4fd1-a3c6-8013c64f1bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:30.053962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387fa0-35c0-4fd1-a3c6-8013c64f1bfc None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:30.054245 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387fa0-35c0-4fd1-a3c6-8013c64f1bfc None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:30.054548 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc387fa0-35c0-4fd1-a3c6-8013c64f1bfc None admin] PATH_INFO: `/v3/projects/a1365787d8534a4c8d69df1fc719689d` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:30.055728 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc387fa0-35c0-4fd1-a3c6-8013c64f1bfc None admin] RBAC: Authorizing `identity:get_project(project_id=a1365787d8534a4c8d69df1fc719689d)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:30.057215 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc387fa0-35c0-4fd1-a3c6-8013c64f1bfc None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:30.058919 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 565/1122] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:51:29 2022] GET /identity/v3/projects/a1365787d8534a4c8d69df1fc719689d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:30.065307 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048fb78-0562-4c0f-8391-5c7ba698b75d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:30.090731 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:30.165441 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048fb78-0562-4c0f-8391-5c7ba698b75d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:30.198813 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab467c-6647-4780-b919-547dfa80b067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:30.199698 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab467c-6647-4780-b919-547dfa80b067 None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:30.199981 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab467c-6647-4780-b919-547dfa80b067 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:30.200318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab467c-6647-4780-b919-547dfa80b067 None admin] PATH_INFO: `/v3/projects/a1365787d8534a4c8d69df1fc719689d/users/3b38caad337340898f7d2136703ba66b/roles/4313b1c9df5345778a249b04355b90a0` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:30.203152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab467c-6647-4780-b919-547dfa80b067 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1365787d8534a4c8d69df1fc719689d, user_id=3b38caad337340898f7d2136703ba66b, role_id=4313b1c9df5345778a249b04355b90a0)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:30.204606 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab467c-6647-4780-b919-547dfa80b067 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:30.292857 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 558/1123] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:51:30 2022] PUT /identity/v3/projects/a1365787d8534a4c8d69df1fc719689d/users/3b38caad337340898f7d2136703ba66b/roles/4313b1c9df5345778a249b04355b90a0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:51:32.489711 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db92ff9-f574-46fa-a3c7-aafc49c0810c None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:32.490296 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db92ff9-f574-46fa-a3c7-aafc49c0810c None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:32.490296 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db92ff9-f574-46fa-a3c7-aafc49c0810c None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:32.491653 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 566/1124] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:32.496944 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae561f37-fe75-4f30-a624-582643828769 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:32.497367 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae561f37-fe75-4f30-a624-582643828769 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:32.497515 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae561f37-fe75-4f30-a624-582643828769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:32.514074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-ae561f37-fe75-4f30-a624-582643828769 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:32.515787 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-ae561f37-fe75-4f30-a624-582643828769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:32.583194 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 559/1125] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:32.608717 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db92ff9-f574-46fa-a3c7-aafc49c0810c None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:32.725609 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db92ff9-f574-46fa-a3c7-aafc49c0810c None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:32.760038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d4de5-45b0-4a70-bd0e-c121aeba7e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:32.760860 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d4de5-45b0-4a70-bd0e-c121aeba7e67 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:32.761149 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d4de5-45b0-4a70-bd0e-c121aeba7e67 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:32.761430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d4de5-45b0-4a70-bd0e-c121aeba7e67 None admin] PATH_INFO: `/v3/roles/4313b1c9df5345778a249b04355b90a0` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:32.762928 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d4de5-45b0-4a70-bd0e-c121aeba7e67 None admin] RBAC: Authorizing `identity:get_role(role_id=4313b1c9df5345778a249b04355b90a0)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:32.764325 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d4de5-45b0-4a70-bd0e-c121aeba7e67 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:32.766330 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 567/1126] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:32 2022] GET /identity/v3/roles/4313b1c9df5345778a249b04355b90a0 => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:32.772120 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae561f37-fe75-4f30-a624-582643828769 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:32.787540 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:32.869697 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae561f37-fe75-4f30-a624-582643828769 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:32.903964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76ac7a-d7c2-48cc-9b5f-e8755f9fe1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:32.904879 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76ac7a-d7c2-48cc-9b5f-e8755f9fe1b7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:32.905166 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76ac7a-d7c2-48cc-9b5f-e8755f9fe1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:32.905445 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76ac7a-d7c2-48cc-9b5f-e8755f9fe1b7 None admin] PATH_INFO: `/v3/users/3b38caad337340898f7d2136703ba66b` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:32.907163 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76ac7a-d7c2-48cc-9b5f-e8755f9fe1b7 None admin] RBAC: Authorizing `identity:get_user(user_id=3b38caad337340898f7d2136703ba66b)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:32.908635 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76ac7a-d7c2-48cc-9b5f-e8755f9fe1b7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:32.910283 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 560/1127] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:32 2022] GET /identity/v3/users/3b38caad337340898f7d2136703ba66b => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:32.916191 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d4de5-45b0-4a70-bd0e-c121aeba7e67 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:32.944472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:33.022566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d4de5-45b0-4a70-bd0e-c121aeba7e67 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:33.057907 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35fa14-8dc1-407d-b24a-31e3ab7cbea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:33.058559 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35fa14-8dc1-407d-b24a-31e3ab7cbea2 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:33.058850 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35fa14-8dc1-407d-b24a-31e3ab7cbea2 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:33.059131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35fa14-8dc1-407d-b24a-31e3ab7cbea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:33.126783 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35fa14-8dc1-407d-b24a-31e3ab7cbea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:33.128307 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35fa14-8dc1-407d-b24a-31e3ab7cbea2 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:33.154198 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:33.227678 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 568/1128] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:51:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:51:33.233619 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76ac7a-d7c2-48cc-9b5f-e8755f9fe1b7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:33.313408 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:33.331300 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76ac7a-d7c2-48cc-9b5f-e8755f9fe1b7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:33.365218 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7c692-4be2-4566-8816-ae29f5e4f0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:33.366075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7c692-4be2-4566-8816-ae29f5e4f0ed None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:33.366355 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7c692-4be2-4566-8816-ae29f5e4f0ed None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:33.366628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7c692-4be2-4566-8816-ae29f5e4f0ed None admin] PATH_INFO: `/v3/projects/a1365787d8534a4c8d69df1fc719689d` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:33.367791 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7c692-4be2-4566-8816-ae29f5e4f0ed None admin] RBAC: Authorizing `identity:get_project(project_id=a1365787d8534a4c8d69df1fc719689d)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:33.369237 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7c692-4be2-4566-8816-ae29f5e4f0ed None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:33.370939 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 561/1129] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:51:33 2022] GET /identity/v3/projects/a1365787d8534a4c8d69df1fc719689d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:33.377321 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35fa14-8dc1-407d-b24a-31e3ab7cbea2 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:33.476282 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35fa14-8dc1-407d-b24a-31e3ab7cbea2 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:33.502697 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:33.511226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fdce6-a6e8-42a2-acb1-282e02ac0028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:33.517986 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fdce6-a6e8-42a2-acb1-282e02ac0028 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:33.517986 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fdce6-a6e8-42a2-acb1-282e02ac0028 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:33.517986 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fdce6-a6e8-42a2-acb1-282e02ac0028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:33.517986 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fdce6-a6e8-42a2-acb1-282e02ac0028 None admin] RBAC: Adding query filter params (role.id=4313b1c9df5345778a249b04355b90a0, scope.project.id=a1365787d8534a4c8d69df1fc719689d, user.id=3b38caad337340898f7d2136703ba66b) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:33.517986 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fdce6-a6e8-42a2-acb1-282e02ac0028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:33.518493 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fdce6-a6e8-42a2-acb1-282e02ac0028 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:33.521619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 569/1130] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:51:33 2022] GET /identity/v3/role_assignments?user.id=3b38caad337340898f7d2136703ba66b&scope.project.id=a1365787d8534a4c8d69df1fc719689d&role.id=4313b1c9df5345778a249b04355b90a0 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:35.777265 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f0c84-3a3b-46d8-a181-eed4539981fc None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:35.777861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f0c84-3a3b-46d8-a181-eed4539981fc None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:35.777861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f0c84-3a3b-46d8-a181-eed4539981fc None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:35.779191 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 562/1131] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:35.785121 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6b378-169c-406d-bd9b-00d631bdeebe None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:35.785522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6b378-169c-406d-bd9b-00d631bdeebe None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:35.785725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6b378-169c-406d-bd9b-00d631bdeebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:35.803295 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-49e6b378-169c-406d-bd9b-00d631bdeebe None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:35.804532 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-49e6b378-169c-406d-bd9b-00d631bdeebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:35.871292 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 570/1132] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:35.894758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f0c84-3a3b-46d8-a181-eed4539981fc None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:35.996465 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f0c84-3a3b-46d8-a181-eed4539981fc None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:36.029313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654a99b-9303-4964-a9ec-ef288b2025c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:36.030037 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654a99b-9303-4964-a9ec-ef288b2025c8 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:36.030319 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654a99b-9303-4964-a9ec-ef288b2025c8 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:36.031118 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654a99b-9303-4964-a9ec-ef288b2025c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:36.032305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654a99b-9303-4964-a9ec-ef288b2025c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:36.033764 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654a99b-9303-4964-a9ec-ef288b2025c8 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:36.035395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 563/1133] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:36.041392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6b378-169c-406d-bd9b-00d631bdeebe None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:36.137955 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e6b378-169c-406d-bd9b-00d631bdeebe None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:36.158391 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:36.171985 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d8119-54df-4a4a-907e-617b69161ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:36.178087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d8119-54df-4a4a-907e-617b69161ab5 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:36.178087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d8119-54df-4a4a-907e-617b69161ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:36.178087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d8119-54df-4a4a-907e-617b69161ab5 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:36.178087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d8119-54df-4a4a-907e-617b69161ab5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:36.178087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d8119-54df-4a4a-907e-617b69161ab5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:36.273563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 571/1134] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:51:36 2022] POST /identity/v3/projects => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:51:38.556075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1384e1-7b70-42ba-a2ca-b97b72a5dfb3 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:38.556686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1384e1-7b70-42ba-a2ca-b97b72a5dfb3 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:38.556686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1384e1-7b70-42ba-a2ca-b97b72a5dfb3 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:38.558049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 564/1135] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:38.563521 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fae032-644e-4332-b242-54bb8badcc30 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:38.563942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fae032-644e-4332-b242-54bb8badcc30 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:38.564081 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fae032-644e-4332-b242-54bb8badcc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:38.581316 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-12fae032-644e-4332-b242-54bb8badcc30 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:38.582518 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-12fae032-644e-4332-b242-54bb8badcc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:38.647616 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 572/1136] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:38.672605 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1384e1-7b70-42ba-a2ca-b97b72a5dfb3 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:38.692427 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:38.773893 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1384e1-7b70-42ba-a2ca-b97b72a5dfb3 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:38.807096 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b248ae41-6a17-4181-9207-1d418cbd0778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:38.807848 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248ae41-6a17-4181-9207-1d418cbd0778 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:38.808096 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248ae41-6a17-4181-9207-1d418cbd0778 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:38.808397 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248ae41-6a17-4181-9207-1d418cbd0778 None admin] PATH_INFO: `/v3/domains/default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:38.809832 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248ae41-6a17-4181-9207-1d418cbd0778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:38.811258 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b248ae41-6a17-4181-9207-1d418cbd0778 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:38.813467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 565/1137] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:51:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:38.819265 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fae032-644e-4332-b242-54bb8badcc30 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:38.922242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fae032-644e-4332-b242-54bb8badcc30 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:38.923777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:38.958496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62179d09-7030-40e7-89b3-a9b97b08c3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:38.959171 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179d09-7030-40e7-89b3-a9b97b08c3a0 None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:38.959470 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179d09-7030-40e7-89b3-a9b97b08c3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:38.959753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62179d09-7030-40e7-89b3-a9b97b08c3a0 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:38.960558 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62179d09-7030-40e7-89b3-a9b97b08c3a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:38.962031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62179d09-7030-40e7-89b3-a9b97b08c3a0 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:39.101236 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 573/1138] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:51:38 2022] POST /identity/v3/users => generated 286 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:51:41.386144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffcca6-6d20-40de-903e-ed6369c3f7f4 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:41.388189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffcca6-6d20-40de-903e-ed6369c3f7f4 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:41.388189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffcca6-6d20-40de-903e-ed6369c3f7f4 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:41.388189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 566/1139] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:41.393329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56c514-2dda-477d-8cf0-67b34bb95b05 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:41.393760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56c514-2dda-477d-8cf0-67b34bb95b05 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:41.394019 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56c514-2dda-477d-8cf0-67b34bb95b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:41.410944 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-0e56c514-2dda-477d-8cf0-67b34bb95b05 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:41.412220 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-0e56c514-2dda-477d-8cf0-67b34bb95b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:41.480468 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 574/1140] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:41.505041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffcca6-6d20-40de-903e-ed6369c3f7f4 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:41.589148 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:41.604517 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ffcca6-6d20-40de-903e-ed6369c3f7f4 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:41.639434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fc04a-722b-4ad6-b8ac-ea2b45598081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:41.640177 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fc04a-722b-4ad6-b8ac-ea2b45598081 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:41.640488 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fc04a-722b-4ad6-b8ac-ea2b45598081 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:41.640768 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fc04a-722b-4ad6-b8ac-ea2b45598081 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:41.650789 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fc04a-722b-4ad6-b8ac-ea2b45598081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-502fc04a-722b-4ad6-b8ac-ea2b45598081 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:51:41.658906 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:41.660463 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:41.661953 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 567/1141] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:51:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:41.666154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56c514-2dda-477d-8cf0-67b34bb95b05 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:41.752573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:41.764205 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56c514-2dda-477d-8cf0-67b34bb95b05 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:41.799305 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828abb9-f40c-4478-bb14-9e7150f46099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:41.799978 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828abb9-f40c-4478-bb14-9e7150f46099 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:41.800277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828abb9-f40c-4478-bb14-9e7150f46099 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:41.800584 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828abb9-f40c-4478-bb14-9e7150f46099 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:41.801337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828abb9-f40c-4478-bb14-9e7150f46099 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:41.801645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828abb9-f40c-4478-bb14-9e7150f46099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:41.803036 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8828abb9-f40c-4478-bb14-9e7150f46099 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:41.811443 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 575/1142] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:51:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:41.817475 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fc04a-722b-4ad6-b8ac-ea2b45598081 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:41.917400 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fc04a-722b-4ad6-b8ac-ea2b45598081 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:41.950622 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba14d1-f437-49f3-8b86-b3783404dba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:41.951492 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba14d1-f437-49f3-8b86-b3783404dba9 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:41.951775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba14d1-f437-49f3-8b86-b3783404dba9 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:41.952048 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba14d1-f437-49f3-8b86-b3783404dba9 None admin] PATH_INFO: `/v3/users/e37f3724160842f48531a203529566cb` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:41.964974 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba14d1-f437-49f3-8b86-b3783404dba9 None admin] RBAC: Authorizing `identity:get_user(user_id=e37f3724160842f48531a203529566cb)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:41.966415 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bba14d1-f437-49f3-8b86-b3783404dba9 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:41.968656 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 568/1143] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:41 2022] GET /identity/v3/users/e37f3724160842f48531a203529566cb => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:41.974389 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828abb9-f40c-4478-bb14-9e7150f46099 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:42.074823 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8828abb9-f40c-4478-bb14-9e7150f46099 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:42.107885 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9ad93-e5c8-4492-8bd0-fb442c1e2c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:42.108586 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9ad93-e5c8-4492-8bd0-fb442c1e2c28 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:42.108870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9ad93-e5c8-4492-8bd0-fb442c1e2c28 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:42.109147 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9ad93-e5c8-4492-8bd0-fb442c1e2c28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:42.127490 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:42.175570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9ad93-e5c8-4492-8bd0-fb442c1e2c28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:42.176992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9ad93-e5c8-4492-8bd0-fb442c1e2c28 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:42.273461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 576/1144] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:51:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:51:42.279446 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba14d1-f437-49f3-8b86-b3783404dba9 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:42.283787 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:42.377267 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bba14d1-f437-49f3-8b86-b3783404dba9 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:42.411882 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36648b86-f2dc-4adb-8c3e-342d2d72f549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:42.412780 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36648b86-f2dc-4adb-8c3e-342d2d72f549 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:42.413066 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36648b86-f2dc-4adb-8c3e-342d2d72f549 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:42.413345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36648b86-f2dc-4adb-8c3e-342d2d72f549 None admin] PATH_INFO: `/v3/projects/9ff26616333f43298c04cc20b08a1cda` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:42.414530 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36648b86-f2dc-4adb-8c3e-342d2d72f549 None admin] RBAC: Authorizing `identity:get_project(project_id=9ff26616333f43298c04cc20b08a1cda)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:42.415980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36648b86-f2dc-4adb-8c3e-342d2d72f549 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:42.417819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 569/1145] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:51:42 2022] GET /identity/v3/projects/9ff26616333f43298c04cc20b08a1cda => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:42.423957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9ad93-e5c8-4492-8bd0-fb442c1e2c28 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:42.473249 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:42.522309 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9ad93-e5c8-4492-8bd0-fb442c1e2c28 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:42.556240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aee07b-9541-4b38-9a71-3515dd568b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:42.556913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aee07b-9541-4b38-9a71-3515dd568b2e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:42.557195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aee07b-9541-4b38-9a71-3515dd568b2e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:42.557471 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aee07b-9541-4b38-9a71-3515dd568b2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:42.558301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aee07b-9541-4b38-9a71-3515dd568b2e None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=9ff26616333f43298c04cc20b08a1cda, user.id=e37f3724160842f48531a203529566cb) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:42.558612 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aee07b-9541-4b38-9a71-3515dd568b2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:42.560314 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aee07b-9541-4b38-9a71-3515dd568b2e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:42.566310 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 577/1146] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:51:42 2022] GET /identity/v3/role_assignments?user.id=e37f3724160842f48531a203529566cb&scope.project.id=9ff26616333f43298c04cc20b08a1cda&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:44.739215 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626978b8-e263-4185-89c4-02a594c19793 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:44.739834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626978b8-e263-4185-89c4-02a594c19793 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:44.739834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626978b8-e263-4185-89c4-02a594c19793 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:44.741208 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 570/1147] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:44.746681 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ac1b-01e5-4b11-927a-c821d94529ed None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:44.747124 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ac1b-01e5-4b11-927a-c821d94529ed None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:44.747238 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5ac1b-01e5-4b11-927a-c821d94529ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:44.763751 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-4db5ac1b-01e5-4b11-927a-c821d94529ed None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:44.764984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-4db5ac1b-01e5-4b11-927a-c821d94529ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:44.830216 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 578/1148] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:44.852786 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626978b8-e263-4185-89c4-02a594c19793 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:44.924248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:44.954917 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626978b8-e263-4185-89c4-02a594c19793 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:44.988912 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e8faa-047d-41db-a945-d8ee3295441b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:44.989761 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e8faa-047d-41db-a945-d8ee3295441b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:44.989999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e8faa-047d-41db-a945-d8ee3295441b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:44.990275 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0e8faa-047d-41db-a945-d8ee3295441b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:45.000261 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0e8faa-047d-41db-a945-d8ee3295441b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-ad0e8faa-047d-41db-a945-d8ee3295441b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:51:45.007875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:45.009373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:45.010744 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 571/1149] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:51:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:45.015175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5ac1b-01e5-4b11-927a-c821d94529ed None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:45.063393 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:45.117467 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5ac1b-01e5-4b11-927a-c821d94529ed None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:45.150903 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4611d-0e4b-438b-8298-4a6c0b17f8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:45.151566 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4611d-0e4b-438b-8298-4a6c0b17f8e9 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:45.151847 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4611d-0e4b-438b-8298-4a6c0b17f8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:45.152118 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4611d-0e4b-438b-8298-4a6c0b17f8e9 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:45.152887 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4611d-0e4b-438b-8298-4a6c0b17f8e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:45.153193 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4611d-0e4b-438b-8298-4a6c0b17f8e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:45.154568 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4611d-0e4b-438b-8298-4a6c0b17f8e9 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:45.163027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 579/1150] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:51:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:45.169101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e8faa-047d-41db-a945-d8ee3295441b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:45.268541 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0e8faa-047d-41db-a945-d8ee3295441b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:45.296958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:45.301801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5ae42-8a69-44e1-a581-4ecb1a6cb320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:45.303658 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5ae42-8a69-44e1-a581-4ecb1a6cb320 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:45.304124 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5ae42-8a69-44e1-a581-4ecb1a6cb320 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:45.304428 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc5ae42-8a69-44e1-a581-4ecb1a6cb320 None admin] PATH_INFO: `/v3/users/e37f3724160842f48531a203529566cb` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:45.305663 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5ae42-8a69-44e1-a581-4ecb1a6cb320 None admin] RBAC: Authorizing `identity:get_user(user_id=e37f3724160842f48531a203529566cb)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:45.307086 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc5ae42-8a69-44e1-a581-4ecb1a6cb320 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:45.308745 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 572/1151] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:45 2022] GET /identity/v3/users/e37f3724160842f48531a203529566cb => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:45.314930 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4611d-0e4b-438b-8298-4a6c0b17f8e9 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:45.416798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4611d-0e4b-438b-8298-4a6c0b17f8e9 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:45.423793 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:45.450843 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae207f43-ad78-4e27-bd94-2ffdef14068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:45.451701 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae207f43-ad78-4e27-bd94-2ffdef14068a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:45.452151 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae207f43-ad78-4e27-bd94-2ffdef14068a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:45.452925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae207f43-ad78-4e27-bd94-2ffdef14068a None admin] PATH_INFO: `/v3/projects/9ff26616333f43298c04cc20b08a1cda` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:45.454297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae207f43-ad78-4e27-bd94-2ffdef14068a None admin] RBAC: Authorizing `identity:get_project(project_id=9ff26616333f43298c04cc20b08a1cda)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:45.456090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae207f43-ad78-4e27-bd94-2ffdef14068a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:45.458229 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 580/1152] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:51:45 2022] GET /identity/v3/projects/9ff26616333f43298c04cc20b08a1cda => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:45.464112 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5ae42-8a69-44e1-a581-4ecb1a6cb320 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:45.563994 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc5ae42-8a69-44e1-a581-4ecb1a6cb320 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:45.597558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f75f08-16e4-40c6-b996-6eba5d6a45f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:45.598440 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f75f08-16e4-40c6-b996-6eba5d6a45f0 None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:45.598717 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f75f08-16e4-40c6-b996-6eba5d6a45f0 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:45.598988 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f75f08-16e4-40c6-b996-6eba5d6a45f0 None admin] PATH_INFO: `/v3/projects/9ff26616333f43298c04cc20b08a1cda/users/e37f3724160842f48531a203529566cb/roles/ed68c2195de147c0a30d9150b33e47cf` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:45.600966 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f75f08-16e4-40c6-b996-6eba5d6a45f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff26616333f43298c04cc20b08a1cda, user_id=e37f3724160842f48531a203529566cb, role_id=ed68c2195de147c0a30d9150b33e47cf)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:45.602426 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f75f08-16e4-40c6-b996-6eba5d6a45f0 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:45.681813 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 573/1153] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:51:45 2022] PUT /identity/v3/projects/9ff26616333f43298c04cc20b08a1cda/users/e37f3724160842f48531a203529566cb/roles/ed68c2195de147c0a30d9150b33e47cf => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:51:47.878924 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f2e3-091a-46a3-bec6-3bd3b8340c48 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:47.879522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f2e3-091a-46a3-bec6-3bd3b8340c48 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:47.879522 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45f2e3-091a-46a3-bec6-3bd3b8340c48 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:47.880937 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 581/1154] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:47.886329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2017fb4-c1dd-40b2-837e-93f92daf302a None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:47.886725 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2017fb4-c1dd-40b2-837e-93f92daf302a None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:47.886904 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2017fb4-c1dd-40b2-837e-93f92daf302a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:47.904620 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-b2017fb4-c1dd-40b2-837e-93f92daf302a None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:47.905848 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-b2017fb4-c1dd-40b2-837e-93f92daf302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:47.962119 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:47.974124 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 574/1155] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:47.997239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45f2e3-091a-46a3-bec6-3bd3b8340c48 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:48.102357 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45f2e3-091a-46a3-bec6-3bd3b8340c48 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:48.145266 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0244782-9ed4-42d6-993a-553aeea3bfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:48.145992 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0244782-9ed4-42d6-993a-553aeea3bfcd None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:48.146280 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0244782-9ed4-42d6-993a-553aeea3bfcd None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:48.146561 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0244782-9ed4-42d6-993a-553aeea3bfcd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:48.157130 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0244782-9ed4-42d6-993a-553aeea3bfcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-c0244782-9ed4-42d6-993a-553aeea3bfcd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:51:48.164785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:48.171508 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:51:48.172777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 582/1156] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:51:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:48.176651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2017fb4-c1dd-40b2-837e-93f92daf302a None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:48.274493 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2017fb4-c1dd-40b2-837e-93f92daf302a None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:48.310104 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e646b5-221b-4a3d-be91-b7e339ce5ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:48.310771 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e646b5-221b-4a3d-be91-b7e339ce5ccc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:48.311049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e646b5-221b-4a3d-be91-b7e339ce5ccc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:48.311338 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e646b5-221b-4a3d-be91-b7e339ce5ccc None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:48.312656 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e646b5-221b-4a3d-be91-b7e339ce5ccc None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:48.312963 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e646b5-221b-4a3d-be91-b7e339ce5ccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:48.314323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e646b5-221b-4a3d-be91-b7e339ce5ccc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:48.322320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 575/1157] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:51:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:51:48.328242 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0244782-9ed4-42d6-993a-553aeea3bfcd None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:48.362388 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:48.428641 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0244782-9ed4-42d6-993a-553aeea3bfcd None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:48.470561 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffe0fb-bc17-480c-896d-41b9dba61615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:48.471452 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffe0fb-bc17-480c-896d-41b9dba61615 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:48.471734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffe0fb-bc17-480c-896d-41b9dba61615 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:48.472013 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dffe0fb-bc17-480c-896d-41b9dba61615 None admin] PATH_INFO: `/v3/users/e37f3724160842f48531a203529566cb` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:48.473432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffe0fb-bc17-480c-896d-41b9dba61615 None admin] RBAC: Authorizing `identity:get_user(user_id=e37f3724160842f48531a203529566cb)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:48.474910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dffe0fb-bc17-480c-896d-41b9dba61615 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:48.476613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 583/1158] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:51:48 2022] GET /identity/v3/users/e37f3724160842f48531a203529566cb => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:48.482556 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e646b5-221b-4a3d-be91-b7e339ce5ccc None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:48.536755 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:48.583676 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e646b5-221b-4a3d-be91-b7e339ce5ccc None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:48.617187 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db83dbb-87ab-436c-8a2a-7b6298c8a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:48.617836 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db83dbb-87ab-436c-8a2a-7b6298c8a836 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:48.618124 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db83dbb-87ab-436c-8a2a-7b6298c8a836 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:48.618402 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db83dbb-87ab-436c-8a2a-7b6298c8a836 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:48.684511 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db83dbb-87ab-436c-8a2a-7b6298c8a836 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:48.686134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db83dbb-87ab-436c-8a2a-7b6298c8a836 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:48.735919 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:48.782281 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 576/1159] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:51:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:51:48.788486 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffe0fb-bc17-480c-896d-41b9dba61615 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:48.887683 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dffe0fb-bc17-480c-896d-41b9dba61615 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:48.889305 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:48.922488 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed4d7f-9a6c-42ba-ab1f-40b762ae272a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:48.923391 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed4d7f-9a6c-42ba-ab1f-40b762ae272a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:48.923681 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed4d7f-9a6c-42ba-ab1f-40b762ae272a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:48.923962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed4d7f-9a6c-42ba-ab1f-40b762ae272a None admin] PATH_INFO: `/v3/projects/9ff26616333f43298c04cc20b08a1cda` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:48.925191 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed4d7f-9a6c-42ba-ab1f-40b762ae272a None admin] RBAC: Authorizing `identity:get_project(project_id=9ff26616333f43298c04cc20b08a1cda)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:48.926657 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed4d7f-9a6c-42ba-ab1f-40b762ae272a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:48.928616 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 584/1160] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:51:48 2022] GET /identity/v3/projects/9ff26616333f43298c04cc20b08a1cda => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:48.934680 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db83dbb-87ab-436c-8a2a-7b6298c8a836 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:49.032476 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db83dbb-87ab-436c-8a2a-7b6298c8a836 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:49.066303 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2b9cc-ac54-47ee-8041-c23a40ff267a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:49.066958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2b9cc-ac54-47ee-8041-c23a40ff267a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:49.067250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2b9cc-ac54-47ee-8041-c23a40ff267a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:49.067557 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2b9cc-ac54-47ee-8041-c23a40ff267a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:49.068405 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2b9cc-ac54-47ee-8041-c23a40ff267a None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=9ff26616333f43298c04cc20b08a1cda, user.id=e37f3724160842f48531a203529566cb) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:49.068721 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2b9cc-ac54-47ee-8041-c23a40ff267a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:49.070189 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2b9cc-ac54-47ee-8041-c23a40ff267a None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:49.076407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 577/1161] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:51:48 2022] GET /identity/v3/role_assignments?user.id=e37f3724160842f48531a203529566cb&scope.project.id=9ff26616333f43298c04cc20b08a1cda&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:51.309657 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cacff-570c-4c6e-b57c-e26e6bd28889 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:51.310514 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cacff-570c-4c6e-b57c-e26e6bd28889 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:51.310514 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cacff-570c-4c6e-b57c-e26e6bd28889 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:51.312450 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 585/1162] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:51.317932 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758aba7-2e99-4fea-829e-030bd4f92976 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:51.318371 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758aba7-2e99-4fea-829e-030bd4f92976 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:51.318480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758aba7-2e99-4fea-829e-030bd4f92976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:51.336305 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-e758aba7-2e99-4fea-829e-030bd4f92976 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:51.337523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-e758aba7-2e99-4fea-829e-030bd4f92976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:51.348065 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:51.404496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 578/1163] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:51.426983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cacff-570c-4c6e-b57c-e26e6bd28889 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:51.528061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cacff-570c-4c6e-b57c-e26e6bd28889 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:51.562347 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ce872-82fc-4a02-9391-02f8c30fd664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:51.562943 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ce872-82fc-4a02-9391-02f8c30fd664 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:51.563226 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ce872-82fc-4a02-9391-02f8c30fd664 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:51.563500 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ce872-82fc-4a02-9391-02f8c30fd664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:51.605876 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:51.633795 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ce872-82fc-4a02-9391-02f8c30fd664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:51.635315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ce872-82fc-4a02-9391-02f8c30fd664 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:51.731760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 586/1164] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:51:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:51:51.737952 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758aba7-2e99-4fea-829e-030bd4f92976 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:51.835293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758aba7-2e99-4fea-829e-030bd4f92976 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:51.867886 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b37fce-ecc8-4fa2-ba60-272a8501cd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:51.868704 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b37fce-ecc8-4fa2-ba60-272a8501cd2d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:51.868988 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b37fce-ecc8-4fa2-ba60-272a8501cd2d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:51.869263 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b37fce-ecc8-4fa2-ba60-272a8501cd2d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:51.876308 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b37fce-ecc8-4fa2-ba60-272a8501cd2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:51.877825 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b37fce-ecc8-4fa2-ba60-272a8501cd2d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-f1b37fce-ecc8-4fa2-ba60-272a8501cd2d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:51:51.892831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:51:51.894345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:51:51.895687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 579/1165] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:51:51 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:51:51.900763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ce872-82fc-4a02-9391-02f8c30fd664 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:51.970084 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:52.000544 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ce872-82fc-4a02-9391-02f8c30fd664 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:52.033317 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ac852-741c-40fd-abee-d9fa57cb316a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:52.034005 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ac852-741c-40fd-abee-d9fa57cb316a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:52.034329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ac852-741c-40fd-abee-d9fa57cb316a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:52.034607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ac852-741c-40fd-abee-d9fa57cb316a None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:52.035322 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ac852-741c-40fd-abee-d9fa57cb316a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:51:52.035630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ac852-741c-40fd-abee-d9fa57cb316a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:52.037063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ac852-741c-40fd-abee-d9fa57cb316a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:52.044416 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 587/1166] 10.222.0.88 () {56 vars in 1115 bytes} [Thu May 19 20:51:51 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:54.273935 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a151-ab3f-44cc-84cf-a68c33c8be08 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:54.274471 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a151-ab3f-44cc-84cf-a68c33c8be08 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:54.274471 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4a151-ab3f-44cc-84cf-a68c33c8be08 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:54.275898 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 580/1167] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:54.281472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090128b-e74e-4d78-a160-fb51b14a570b None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:54.281903 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090128b-e74e-4d78-a160-fb51b14a570b None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:54.282119 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090128b-e74e-4d78-a160-fb51b14a570b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:54.299942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-c090128b-e74e-4d78-a160-fb51b14a570b None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:54.301196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-c090128b-e74e-4d78-a160-fb51b14a570b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:54.368066 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 588/1168] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:54.392073 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4a151-ab3f-44cc-84cf-a68c33c8be08 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:54.402304 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:54.494981 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4a151-ab3f-44cc-84cf-a68c33c8be08 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:54.528941 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db42a98-ce63-49f2-bead-c24b51d14d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:54.529539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db42a98-ce63-49f2-bead-c24b51d14d65 None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:54.529825 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db42a98-ce63-49f2-bead-c24b51d14d65 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:54.530101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db42a98-ce63-49f2-bead-c24b51d14d65 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:54.530847 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db42a98-ce63-49f2-bead-c24b51d14d65 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:54.532199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db42a98-ce63-49f2-bead-c24b51d14d65 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:54.649306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 581/1169] 10.222.0.88 () {60 vars in 1138 bytes} [Thu May 19 20:51:54 2022] POST /identity/v3/domains => generated 267 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:51:56.906401 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099dc7d-a820-4508-a3bc-400de988d40e None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:56.906982 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099dc7d-a820-4508-a3bc-400de988d40e None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:56.906982 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a099dc7d-a820-4508-a3bc-400de988d40e None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:56.908367 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 589/1170] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:56.913735 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83461a-672a-4d18-addf-4884106f59e0 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:56.914168 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83461a-672a-4d18-addf-4884106f59e0 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:56.914284 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83461a-672a-4d18-addf-4884106f59e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:56.930897 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-4a83461a-672a-4d18-addf-4884106f59e0 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:56.932083 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-4a83461a-672a-4d18-addf-4884106f59e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:56.992057 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:56.999312 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 582/1171] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:57.023165 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099dc7d-a820-4508-a3bc-400de988d40e None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:57.087277 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:57.125137 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a099dc7d-a820-4508-a3bc-400de988d40e None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:57.158834 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37dadf-c473-46b3-b6fc-8083ae1f86f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:57.159628 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37dadf-c473-46b3-b6fc-8083ae1f86f7 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:57.159916 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37dadf-c473-46b3-b6fc-8083ae1f86f7 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:57.160238 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37dadf-c473-46b3-b6fc-8083ae1f86f7 None admin] PATH_INFO: `/v3/domains/3bd048863ca84cde863e4e0013bd1ba7` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:57.170915 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37dadf-c473-46b3-b6fc-8083ae1f86f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3bd048863ca84cde863e4e0013bd1ba7)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:57.172429 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37dadf-c473-46b3-b6fc-8083ae1f86f7 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:57.175017 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 590/1172] 10.222.0.88 () {56 vars in 1183 bytes} [Thu May 19 20:51:57 2022] GET /identity/v3/domains/3bd048863ca84cde863e4e0013bd1ba7 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:57.181259 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83461a-672a-4d18-addf-4884106f59e0 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:57.278033 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83461a-672a-4d18-addf-4884106f59e0 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:57.311190 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84867354-d7c4-4013-9943-f99a1b724b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:57.311843 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84867354-d7c4-4013-9943-f99a1b724b1d None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:57.312151 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84867354-d7c4-4013-9943-f99a1b724b1d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:57.312431 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84867354-d7c4-4013-9943-f99a1b724b1d None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:57.313129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84867354-d7c4-4013-9943-f99a1b724b1d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:57.314547 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84867354-d7c4-4013-9943-f99a1b724b1d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:57.408109 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 583/1173] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:51:57 2022] POST /identity/v3/projects => generated 361 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:51:59.651760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f202cd-5099-4b3b-9a28-957d9638e8c7 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:59.652368 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f202cd-5099-4b3b-9a28-957d9638e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:59.652368 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f202cd-5099-4b3b-9a28-957d9638e8c7 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:59.653809 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 591/1174] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:51:59.659651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d774c0-05d5-4490-aebb-62b44c5a17e0 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:59.660101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d774c0-05d5-4490-aebb-62b44c5a17e0 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:59.660212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d774c0-05d5-4490-aebb-62b44c5a17e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:59.677408 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-84d774c0-05d5-4490-aebb-62b44c5a17e0 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:51:59.678594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-84d774c0-05d5-4490-aebb-62b44c5a17e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:51:59.722807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:59.746338 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 584/1175] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:51:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:51:59.771329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f202cd-5099-4b3b-9a28-957d9638e8c7 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:51:59.873738 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f202cd-5099-4b3b-9a28-957d9638e8c7 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:51:59.897172 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:51:59.922894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a6032-5784-4271-a3db-4cfd818bb18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:51:59.922894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a6032-5784-4271-a3db-4cfd818bb18a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:51:59.922894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a6032-5784-4271-a3db-4cfd818bb18a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:51:59.922894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a6032-5784-4271-a3db-4cfd818bb18a None admin] PATH_INFO: `/v3/domains/3bd048863ca84cde863e4e0013bd1ba7` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:51:59.923392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a6032-5784-4271-a3db-4cfd818bb18a None admin] RBAC: Authorizing `identity:get_domain(domain_id=3bd048863ca84cde863e4e0013bd1ba7)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:51:59.923392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a6032-5784-4271-a3db-4cfd818bb18a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:51:59.923392 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 592/1176] 10.222.0.88 () {56 vars in 1183 bytes} [Thu May 19 20:51:59 2022] GET /identity/v3/domains/3bd048863ca84cde863e4e0013bd1ba7 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:51:59.923660 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d774c0-05d5-4490-aebb-62b44c5a17e0 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:00.017791 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d774c0-05d5-4490-aebb-62b44c5a17e0 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:00.052316 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a246c3-97cc-4a42-93ca-9f84511b3e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:00.053023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a246c3-97cc-4a42-93ca-9f84511b3e7c None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:00.053309 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a246c3-97cc-4a42-93ca-9f84511b3e7c None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:00.053585 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a246c3-97cc-4a42-93ca-9f84511b3e7c None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:00.054293 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a246c3-97cc-4a42-93ca-9f84511b3e7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:00.055710 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a246c3-97cc-4a42-93ca-9f84511b3e7c None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:00.135055 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 585/1177] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:51:59 2022] POST /identity/v3/users => generated 311 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:52:02.378332 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28baff4-488e-4d13-babd-b385dbf3cf0d None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:02.378962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28baff4-488e-4d13-babd-b385dbf3cf0d None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:02.378962 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28baff4-488e-4d13-babd-b385dbf3cf0d None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:02.380309 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 593/1178] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:52:02.385669 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8b2fd-235e-4061-8a78-78cd9c127a13 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:02.386106 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8b2fd-235e-4061-8a78-78cd9c127a13 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:02.386210 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8b2fd-235e-4061-8a78-78cd9c127a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:02.403365 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-85a8b2fd-235e-4061-8a78-78cd9c127a13 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:52:02.404568 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-85a8b2fd-235e-4061-8a78-78cd9c127a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:52:02.431115 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:02.472324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 586/1179] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:52:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:52:02.495647 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28baff4-488e-4d13-babd-b385dbf3cf0d None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:02.599062 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28baff4-488e-4d13-babd-b385dbf3cf0d None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:02.633047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920c7dd-176b-438e-b967-16a7b5bba1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:02.633782 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920c7dd-176b-438e-b967-16a7b5bba1a9 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:02.634071 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920c7dd-176b-438e-b967-16a7b5bba1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:02.634353 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920c7dd-176b-438e-b967-16a7b5bba1a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:02.646135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920c7dd-176b-438e-b967-16a7b5bba1a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-d920c7dd-176b-438e-b967-16a7b5bba1a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:52:02.653934 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:52:02.655179 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:52:02.656408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 594/1180] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:52:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:52:02.661187 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8b2fd-235e-4061-8a78-78cd9c127a13 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:02.760313 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8b2fd-235e-4061-8a78-78cd9c127a13 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:02.794395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d258ed-b8f4-4284-862e-5ea57efcac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:02.795056 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d258ed-b8f4-4284-862e-5ea57efcac01 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:02.795339 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d258ed-b8f4-4284-862e-5ea57efcac01 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:02.795617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d258ed-b8f4-4284-862e-5ea57efcac01 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:02.796372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d258ed-b8f4-4284-862e-5ea57efcac01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:52:02.796687 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d258ed-b8f4-4284-862e-5ea57efcac01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:02.798050 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d258ed-b8f4-4284-862e-5ea57efcac01 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:02.806805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 587/1181] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:52:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:52:02.812863 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920c7dd-176b-438e-b967-16a7b5bba1a9 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:02.865633 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:02.911466 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920c7dd-176b-438e-b967-16a7b5bba1a9 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:02.945780 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0376d-f3e6-4177-8bc9-2ee04f861556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:02.946725 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0376d-f3e6-4177-8bc9-2ee04f861556 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:02.947011 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0376d-f3e6-4177-8bc9-2ee04f861556 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:02.947286 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0376d-f3e6-4177-8bc9-2ee04f861556 None admin] PATH_INFO: `/v3/users/9b26cb8e98c24683b0aaa2978745730f` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:02.960840 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0376d-f3e6-4177-8bc9-2ee04f861556 None admin] RBAC: Authorizing `identity:get_user(user_id=9b26cb8e98c24683b0aaa2978745730f)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:02.962309 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0376d-f3e6-4177-8bc9-2ee04f861556 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:02.963945 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 595/1182] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:52:02 2022] GET /identity/v3/users/9b26cb8e98c24683b0aaa2978745730f => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:52:02.969713 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d258ed-b8f4-4284-862e-5ea57efcac01 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:02.997986 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:03.069965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d258ed-b8f4-4284-862e-5ea57efcac01 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:03.104577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154b630-00a5-4048-aa8b-d811f91ca144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:03.105232 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154b630-00a5-4048-aa8b-d811f91ca144 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:03.105516 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154b630-00a5-4048-aa8b-d811f91ca144 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:03.105793 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154b630-00a5-4048-aa8b-d811f91ca144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:03.177513 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154b630-00a5-4048-aa8b-d811f91ca144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:03.178882 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154b630-00a5-4048-aa8b-d811f91ca144 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:03.204575 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:03.293239 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 588/1183] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:52:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:52:03.299096 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0376d-f3e6-4177-8bc9-2ee04f861556 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:03.402862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0376d-f3e6-4177-8bc9-2ee04f861556 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:03.413622 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:03.436786 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2aab54-d72f-45a8-91c0-707da6318676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:03.437619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2aab54-d72f-45a8-91c0-707da6318676 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:03.437905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2aab54-d72f-45a8-91c0-707da6318676 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:03.438184 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2aab54-d72f-45a8-91c0-707da6318676 None admin] PATH_INFO: `/v3/projects/e89de334bf8e4b87b6cec9e7addc11fc` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:03.439474 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2aab54-d72f-45a8-91c0-707da6318676 None admin] RBAC: Authorizing `identity:get_project(project_id=e89de334bf8e4b87b6cec9e7addc11fc)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:03.441384 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2aab54-d72f-45a8-91c0-707da6318676 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:03.443089 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 596/1184] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:52:03 2022] GET /identity/v3/projects/e89de334bf8e4b87b6cec9e7addc11fc => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:52:03.449381 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154b630-00a5-4048-aa8b-d811f91ca144 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:03.547148 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154b630-00a5-4048-aa8b-d811f91ca144 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:03.576396 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:03.586337 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724d32a-1040-4122-8d1d-6efd2f6227c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:03.592212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724d32a-1040-4122-8d1d-6efd2f6227c5 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:03.592212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724d32a-1040-4122-8d1d-6efd2f6227c5 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:03.592212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724d32a-1040-4122-8d1d-6efd2f6227c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:03.592212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724d32a-1040-4122-8d1d-6efd2f6227c5 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=e89de334bf8e4b87b6cec9e7addc11fc, user.id=9b26cb8e98c24683b0aaa2978745730f) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:52:03.592212 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724d32a-1040-4122-8d1d-6efd2f6227c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:03.592766 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724d32a-1040-4122-8d1d-6efd2f6227c5 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:03.604359 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 589/1185] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:52:03 2022] GET /identity/v3/role_assignments?user.id=9b26cb8e98c24683b0aaa2978745730f&scope.project.id=e89de334bf8e4b87b6cec9e7addc11fc&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:52:05.811333 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9e859-716b-4006-be33-f8a23d53a224 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:05.811940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9e859-716b-4006-be33-f8a23d53a224 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:05.811940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9e859-716b-4006-be33-f8a23d53a224 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:05.813305 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 597/1186] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:52:05.818812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2a270-c745-435b-a460-9dab017ea0ac None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:05.819249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2a270-c745-435b-a460-9dab017ea0ac None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:05.819372 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2a270-c745-435b-a460-9dab017ea0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:05.836213 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-fab2a270-c745-435b-a460-9dab017ea0ac None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:52:05.837420 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-fab2a270-c745-435b-a460-9dab017ea0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:52:05.903442 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 590/1187] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:52:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:52:05.927186 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9e859-716b-4006-be33-f8a23d53a224 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:06.029724 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9e859-716b-4006-be33-f8a23d53a224 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:06.064238 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebbd32-7453-4ad8-b009-bde85bed37cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:06.065062 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebbd32-7453-4ad8-b009-bde85bed37cf None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:06.065351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebbd32-7453-4ad8-b009-bde85bed37cf None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:06.065630 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebbd32-7453-4ad8-b009-bde85bed37cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:06.075783 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebbd32-7453-4ad8-b009-bde85bed37cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-b0ebbd32-7453-4ad8-b009-bde85bed37cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:52:06.084003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:52:06.085547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:52:06.086966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 598/1188] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:52:06.091381 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2a270-c745-435b-a460-9dab017ea0ac None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:06.189045 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2a270-c745-435b-a460-9dab017ea0ac None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:06.209120 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:06.223263 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5294743-7c11-4032-9686-fe4d04674060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:06.225984 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5294743-7c11-4032-9686-fe4d04674060 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:06.225984 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5294743-7c11-4032-9686-fe4d04674060 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:06.225984 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5294743-7c11-4032-9686-fe4d04674060 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:06.225984 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5294743-7c11-4032-9686-fe4d04674060 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:52:06.225984 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5294743-7c11-4032-9686-fe4d04674060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:06.226871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5294743-7c11-4032-9686-fe4d04674060 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:06.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 591/1189] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:52:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:52:06.240749 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebbd32-7453-4ad8-b009-bde85bed37cf None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:06.279875 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:06.340648 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebbd32-7453-4ad8-b009-bde85bed37cf None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:06.373004 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed70400-5c8d-44dc-9e94-e428e094d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:06.373893 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed70400-5c8d-44dc-9e94-e428e094d0fc None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:06.374175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed70400-5c8d-44dc-9e94-e428e094d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:06.374453 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed70400-5c8d-44dc-9e94-e428e094d0fc None admin] PATH_INFO: `/v3/users/9b26cb8e98c24683b0aaa2978745730f` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:06.375637 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed70400-5c8d-44dc-9e94-e428e094d0fc None admin] RBAC: Authorizing `identity:get_user(user_id=9b26cb8e98c24683b0aaa2978745730f)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:06.377110 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed70400-5c8d-44dc-9e94-e428e094d0fc None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:06.379240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 599/1190] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:52:06 2022] GET /identity/v3/users/9b26cb8e98c24683b0aaa2978745730f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:52:06.385227 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5294743-7c11-4032-9686-fe4d04674060 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:06.484683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5294743-7c11-4032-9686-fe4d04674060 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:06.517410 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22234ef5-73c5-458a-9e43-0e7b508a353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:06.518814 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22234ef5-73c5-458a-9e43-0e7b508a353d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:06.519090 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22234ef5-73c5-458a-9e43-0e7b508a353d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:06.519364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22234ef5-73c5-458a-9e43-0e7b508a353d None admin] PATH_INFO: `/v3/projects/e89de334bf8e4b87b6cec9e7addc11fc` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:06.520586 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22234ef5-73c5-458a-9e43-0e7b508a353d None admin] RBAC: Authorizing `identity:get_project(project_id=e89de334bf8e4b87b6cec9e7addc11fc)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:06.522008 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22234ef5-73c5-458a-9e43-0e7b508a353d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:06.523702 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 592/1191] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:52:06 2022] GET /identity/v3/projects/e89de334bf8e4b87b6cec9e7addc11fc => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:52:06.529576 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed70400-5c8d-44dc-9e94-e428e094d0fc None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:06.629370 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed70400-5c8d-44dc-9e94-e428e094d0fc None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:06.636069 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:06.662487 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56927573-4bd6-4927-b15a-e3df9d47eaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:06.663424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927573-4bd6-4927-b15a-e3df9d47eaf7 None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:06.663706 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927573-4bd6-4927-b15a-e3df9d47eaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:06.663986 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56927573-4bd6-4927-b15a-e3df9d47eaf7 None admin] PATH_INFO: `/v3/projects/e89de334bf8e4b87b6cec9e7addc11fc/users/9b26cb8e98c24683b0aaa2978745730f/roles/ed68c2195de147c0a30d9150b33e47cf` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:06.665982 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56927573-4bd6-4927-b15a-e3df9d47eaf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e89de334bf8e4b87b6cec9e7addc11fc, user_id=9b26cb8e98c24683b0aaa2978745730f, role_id=ed68c2195de147c0a30d9150b33e47cf)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:06.667432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56927573-4bd6-4927-b15a-e3df9d47eaf7 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:06.732569 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 600/1192] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:52:06 2022] PUT /identity/v3/projects/e89de334bf8e4b87b6cec9e7addc11fc/users/9b26cb8e98c24683b0aaa2978745730f/roles/ed68c2195de147c0a30d9150b33e47cf => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:52:08.960121 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e2d52-bca8-4dea-91c7-8853fafc4c0b None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:08.965204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e2d52-bca8-4dea-91c7-8853fafc4c0b None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:08.965204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e2d52-bca8-4dea-91c7-8853fafc4c0b None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:08.965204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 593/1193] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:52:08.967430 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a94717-a370-4ffb-bb7c-3d66fefb4be9 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:08.967866 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a94717-a370-4ffb-bb7c-3d66fefb4be9 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:08.967976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a94717-a370-4ffb-bb7c-3d66fefb4be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:08.986563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-74a94717-a370-4ffb-bb7c-3d66fefb4be9 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:52:08.987802 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-74a94717-a370-4ffb-bb7c-3d66fefb4be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:52:09.055443 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 601/1194] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:52:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:52:09.079395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e2d52-bca8-4dea-91c7-8853fafc4c0b None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:09.130411 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:09.181578 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e2d52-bca8-4dea-91c7-8853fafc4c0b None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:09.214336 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89bf58-b31b-4c7b-abf9-b9dd6e4035c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:09.215131 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89bf58-b31b-4c7b-abf9-b9dd6e4035c2 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:09.215418 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89bf58-b31b-4c7b-abf9-b9dd6e4035c2 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:09.215699 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89bf58-b31b-4c7b-abf9-b9dd6e4035c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:09.226177 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89bf58-b31b-4c7b-abf9-b9dd6e4035c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-3f89bf58-b31b-4c7b-abf9-b9dd6e4035c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:52:09.233754 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:52:09.234839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:52:09.236027 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 594/1195] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:52:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:52:09.241196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a94717-a370-4ffb-bb7c-3d66fefb4be9 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:09.333189 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:09.338840 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a94717-a370-4ffb-bb7c-3d66fefb4be9 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:09.371511 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc7ca0-b319-4888-affe-5cb4a2f335d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:09.372166 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc7ca0-b319-4888-affe-5cb4a2f335d3 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:09.372478 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc7ca0-b319-4888-affe-5cb4a2f335d3 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:09.372756 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc7ca0-b319-4888-affe-5cb4a2f335d3 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:09.373570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc7ca0-b319-4888-affe-5cb4a2f335d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:52:09.373891 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc7ca0-b319-4888-affe-5cb4a2f335d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:09.375733 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc7ca0-b319-4888-affe-5cb4a2f335d3 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:09.384564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 602/1196] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:52:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:52:09.390544 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89bf58-b31b-4c7b-abf9-b9dd6e4035c2 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:09.491821 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89bf58-b31b-4c7b-abf9-b9dd6e4035c2 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:09.501869 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:09.525940 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf48cb-5e95-43ab-9545-e3e4499560bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:09.526806 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf48cb-5e95-43ab-9545-e3e4499560bc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:09.527094 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf48cb-5e95-43ab-9545-e3e4499560bc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:09.527373 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf48cb-5e95-43ab-9545-e3e4499560bc None admin] PATH_INFO: `/v3/users/9b26cb8e98c24683b0aaa2978745730f` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:09.528578 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf48cb-5e95-43ab-9545-e3e4499560bc None admin] RBAC: Authorizing `identity:get_user(user_id=9b26cb8e98c24683b0aaa2978745730f)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:09.530016 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf48cb-5e95-43ab-9545-e3e4499560bc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:09.531682 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 595/1197] 10.222.0.88 () {56 vars in 1177 bytes} [Thu May 19 20:52:09 2022] GET /identity/v3/users/9b26cb8e98c24683b0aaa2978745730f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:52:09.537668 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc7ca0-b319-4888-affe-5cb4a2f335d3 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:09.636110 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc7ca0-b319-4888-affe-5cb4a2f335d3 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:09.669550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84414e85-130f-444a-809e-76f8047885e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:09.670196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84414e85-130f-444a-809e-76f8047885e3 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:09.670483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84414e85-130f-444a-809e-76f8047885e3 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:09.670764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84414e85-130f-444a-809e-76f8047885e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:09.697632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:09.738545 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84414e85-130f-444a-809e-76f8047885e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:09.740005 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84414e85-130f-444a-809e-76f8047885e3 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:09.837485 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 603/1198] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:52:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:52:09.843439 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf48cb-5e95-43ab-9545-e3e4499560bc None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:09.943233 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf48cb-5e95-43ab-9545-e3e4499560bc None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:09.977002 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe547b4-e97a-4095-9e0a-9b6bb05716ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:09.977860 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe547b4-e97a-4095-9e0a-9b6bb05716ed None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:09.978145 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe547b4-e97a-4095-9e0a-9b6bb05716ed None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:09.978429 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe547b4-e97a-4095-9e0a-9b6bb05716ed None admin] PATH_INFO: `/v3/projects/e89de334bf8e4b87b6cec9e7addc11fc` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:09.979613 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe547b4-e97a-4095-9e0a-9b6bb05716ed None admin] RBAC: Authorizing `identity:get_project(project_id=e89de334bf8e4b87b6cec9e7addc11fc)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:09.981075 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe547b4-e97a-4095-9e0a-9b6bb05716ed None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:09.982779 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 596/1199] 10.222.0.88 () {56 vars in 1186 bytes} [Thu May 19 20:52:09 2022] GET /identity/v3/projects/e89de334bf8e4b87b6cec9e7addc11fc => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:52:09.989063 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84414e85-130f-444a-809e-76f8047885e3 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:52:10.049527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:10.103536 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84414e85-130f-444a-809e-76f8047885e3 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:52:10.137095 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03c7c5-f34d-4c65-b5ce-a3fb0e13a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:52:10.137749 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03c7c5-f34d-4c65-b5ce-a3fb0e13a97a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:52:10.138032 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03c7c5-f34d-4c65-b5ce-a3fb0e13a97a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:52:10.138308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03c7c5-f34d-4c65-b5ce-a3fb0e13a97a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:52:10.139145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03c7c5-f34d-4c65-b5ce-a3fb0e13a97a None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=e89de334bf8e4b87b6cec9e7addc11fc, user.id=9b26cb8e98c24683b0aaa2978745730f) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:52:10.139458 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03c7c5-f34d-4c65-b5ce-a3fb0e13a97a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:52:10.140942 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03c7c5-f34d-4c65-b5ce-a3fb0e13a97a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:52:10.147259 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 604/1200] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:52:09 2022] GET /identity/v3/role_assignments?user.id=9b26cb8e98c24683b0aaa2978745730f&scope.project.id=e89de334bf8e4b87b6cec9e7addc11fc&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:52:19.972783 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=77 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:52:20.144733 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=39 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:31.390506 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d858107-8d55-4f80-b217-e52266b5960f None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:31.391123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d858107-8d55-4f80-b217-e52266b5960f None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:31.391123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d858107-8d55-4f80-b217-e52266b5960f None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:31.392537 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 597/1201] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:53:31.398407 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee63e8-80e6-470f-98a4-a007cc89ebe4 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:31.398842 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee63e8-80e6-470f-98a4-a007cc89ebe4 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:31.398969 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee63e8-80e6-470f-98a4-a007cc89ebe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:31.416249 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-f1ee63e8-80e6-470f-98a4-a007cc89ebe4 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:53:31.417461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-f1ee63e8-80e6-470f-98a4-a007cc89ebe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:53:31.485053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 605/1202] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:53:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:53:31.511888 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d858107-8d55-4f80-b217-e52266b5960f None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:31.613341 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d858107-8d55-4f80-b217-e52266b5960f None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:31.646807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc8d74-2f9f-4172-839a-f7144a59d65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:31.647538 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc8d74-2f9f-4172-839a-f7144a59d65d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:31.647819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc8d74-2f9f-4172-839a-f7144a59d65d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:31.648089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc8d74-2f9f-4172-839a-f7144a59d65d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:31.658003 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc8d74-2f9f-4172-839a-f7144a59d65d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:31.659426 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc8d74-2f9f-4172-839a-f7144a59d65d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-59fc8d74-2f9f-4172-839a-f7144a59d65d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:31.676911 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:31.678436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:31.687276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:31.687276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:31.687276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:31.687276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:31.687276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 598/1203] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:31.687777 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee63e8-80e6-470f-98a4-a007cc89ebe4 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:31.783805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee63e8-80e6-470f-98a4-a007cc89ebe4 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:31.813180 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:31.818712 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7738e560-9aff-4a83-b777-a40f042a6c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:31.819918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738e560-9aff-4a83-b777-a40f042a6c32 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:31.820201 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738e560-9aff-4a83-b777-a40f042a6c32 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:31.820512 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7738e560-9aff-4a83-b777-a40f042a6c32 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:31.821240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738e560-9aff-4a83-b777-a40f042a6c32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:31.821548 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738e560-9aff-4a83-b777-a40f042a6c32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:31.823017 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7738e560-9aff-4a83-b777-a40f042a6c32 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:31.834266 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 606/1204] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:31.840898 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc8d74-2f9f-4172-839a-f7144a59d65d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:31.893628 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:31.940674 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc8d74-2f9f-4172-839a-f7144a59d65d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:31.974692 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d2cf3-6712-49d3-b104-904f7fd4e796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:31.975550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d2cf3-6712-49d3-b104-904f7fd4e796 None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:31.976434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d2cf3-6712-49d3-b104-904f7fd4e796 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:31.976763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d2cf3-6712-49d3-b104-904f7fd4e796 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:31.977510 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d2cf3-6712-49d3-b104-904f7fd4e796 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:31.979095 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d2cf3-6712-49d3-b104-904f7fd4e796 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:32.083595 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 599/1205] 10.222.0.88 () {60 vars in 1132 bytes} [Thu May 19 20:53:31 2022] POST /identity/v3/users => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:53:34.328598 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54a5f-377a-4eb6-931f-bea963b8d144 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:34.329190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54a5f-377a-4eb6-931f-bea963b8d144 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:34.329190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54a5f-377a-4eb6-931f-bea963b8d144 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:34.330638 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 607/1206] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:53:34.336735 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a9e4a-9294-4c23-81b6-d981953f9fe0 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:34.337229 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a9e4a-9294-4c23-81b6-d981953f9fe0 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:34.337351 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a9e4a-9294-4c23-81b6-d981953f9fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:34.354339 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-499a9e4a-9294-4c23-81b6-d981953f9fe0 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:53:34.355525 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-499a9e4a-9294-4c23-81b6-d981953f9fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:53:34.431835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 600/1207] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:53:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:53:34.455852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f54a5f-377a-4eb6-931f-bea963b8d144 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:34.558948 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f54a5f-377a-4eb6-931f-bea963b8d144 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:34.592820 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157f927-a000-4ac3-8ef8-7d8002999a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:34.593704 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157f927-a000-4ac3-8ef8-7d8002999a48 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:34.593984 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157f927-a000-4ac3-8ef8-7d8002999a48 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:34.594262 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157f927-a000-4ac3-8ef8-7d8002999a48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:34.605403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157f927-a000-4ac3-8ef8-7d8002999a48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-c157f927-a000-4ac3-8ef8-7d8002999a48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:53:34.613211 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:34.614527 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:34.615808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 608/1208] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:53:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:34.622095 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a9e4a-9294-4c23-81b6-d981953f9fe0 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:34.662582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:34.721742 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a9e4a-9294-4c23-81b6-d981953f9fe0 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:34.756714 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71321c-77bc-40af-ac8d-28a75526ef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:34.757386 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71321c-77bc-40af-ac8d-28a75526ef1f None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:34.757671 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71321c-77bc-40af-ac8d-28a75526ef1f None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:34.757950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71321c-77bc-40af-ac8d-28a75526ef1f None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:34.758694 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71321c-77bc-40af-ac8d-28a75526ef1f None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:34.759069 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71321c-77bc-40af-ac8d-28a75526ef1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:34.760436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71321c-77bc-40af-ac8d-28a75526ef1f None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:34.768984 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 601/1209] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:53:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:34.774913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157f927-a000-4ac3-8ef8-7d8002999a48 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:34.824863 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:34.874307 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157f927-a000-4ac3-8ef8-7d8002999a48 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:34.909766 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0b9e-b483-4533-a42e-5cd5e3901d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:34.910567 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0b9e-b483-4533-a42e-5cd5e3901d3f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:34.910848 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0b9e-b483-4533-a42e-5cd5e3901d3f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:34.911125 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b0b9e-b483-4533-a42e-5cd5e3901d3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:34.921757 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b0b9e-b483-4533-a42e-5cd5e3901d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:34.923239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b0b9e-b483-4533-a42e-5cd5e3901d3f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-7d2b0b9e-b483-4533-a42e-5cd5e3901d3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:34.942424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:34.943665 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:34.945067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:34.945067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:34.945067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:34.945067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:34.945067 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 609/1210] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:34.949546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71321c-77bc-40af-ac8d-28a75526ef1f None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:35.050409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:35.053378 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71321c-77bc-40af-ac8d-28a75526ef1f None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:35.087997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf66ddd-8c87-4756-886f-b6bb50cdbe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:35.088695 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf66ddd-8c87-4756-886f-b6bb50cdbe93 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:35.088984 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf66ddd-8c87-4756-886f-b6bb50cdbe93 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:35.089272 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf66ddd-8c87-4756-886f-b6bb50cdbe93 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:35.089996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf66ddd-8c87-4756-886f-b6bb50cdbe93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:35.090307 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf66ddd-8c87-4756-886f-b6bb50cdbe93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:35.091677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf66ddd-8c87-4756-886f-b6bb50cdbe93 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:35.103420 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 602/1211] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:35.109298 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0b9e-b483-4533-a42e-5cd5e3901d3f None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:35.209224 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b0b9e-b483-4533-a42e-5cd5e3901d3f None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:35.211311 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:35.244922 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afecca5-318b-4839-b1d7-f5109ebae712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:35.245723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afecca5-318b-4839-b1d7-f5109ebae712 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:35.246035 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afecca5-318b-4839-b1d7-f5109ebae712 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:35.246315 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afecca5-318b-4839-b1d7-f5109ebae712 None admin] PATH_INFO: `/v3/users/placement` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:35.254133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afecca5-318b-4839-b1d7-f5109ebae712 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:35.255607 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afecca5-318b-4839-b1d7-f5109ebae712 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-6afecca5-318b-4839-b1d7-f5109ebae712 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:53:35.271010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:35.274213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:35.275596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:35.275596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:35.275596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:53:35.275596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:53:35.275596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:53:35.275596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:53:35.275596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:53:35.275596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:53:35.275596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:35.275596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:35.275596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 610/1212] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:35.278878 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf66ddd-8c87-4756-886f-b6bb50cdbe93 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:35.380180 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf66ddd-8c87-4756-886f-b6bb50cdbe93 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:35.414130 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c39e62-2b4d-46a3-8eb3-0dfc18e58ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:35.414798 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c39e62-2b4d-46a3-8eb3-0dfc18e58ffc None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:35.415081 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c39e62-2b4d-46a3-8eb3-0dfc18e58ffc None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:35.415357 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c39e62-2b4d-46a3-8eb3-0dfc18e58ffc None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:35.416188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c39e62-2b4d-46a3-8eb3-0dfc18e58ffc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:35.416527 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c39e62-2b4d-46a3-8eb3-0dfc18e58ffc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:35.418083 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c39e62-2b4d-46a3-8eb3-0dfc18e58ffc None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:35.435657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 603/1213] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:53:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:35.442058 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afecca5-318b-4839-b1d7-f5109ebae712 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:35.542563 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afecca5-318b-4839-b1d7-f5109ebae712 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:35.576811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455e00f-d853-498f-be27-66ada1ff44c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:35.577448 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455e00f-d853-498f-be27-66ada1ff44c5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:35.577735 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455e00f-d853-498f-be27-66ada1ff44c5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:35.578010 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455e00f-d853-498f-be27-66ada1ff44c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:35.608086 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:35.645139 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455e00f-d853-498f-be27-66ada1ff44c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:35.646633 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8455e00f-d853-498f-be27-66ada1ff44c5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:35.748329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 611/1214] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:53:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:53:35.754163 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c39e62-2b4d-46a3-8eb3-0dfc18e58ffc None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:35.773172 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:35.853345 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c39e62-2b4d-46a3-8eb3-0dfc18e58ffc None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:35.887453 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f90ef7-df89-419f-a7bd-6bde473283d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:35.888258 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f90ef7-df89-419f-a7bd-6bde473283d9 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:35.888577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f90ef7-df89-419f-a7bd-6bde473283d9 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:35.888858 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f90ef7-df89-419f-a7bd-6bde473283d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:35.899807 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f90ef7-df89-419f-a7bd-6bde473283d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:35.901289 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f90ef7-df89-419f-a7bd-6bde473283d9 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-d2f90ef7-df89-419f-a7bd-6bde473283d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:35.919497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:35.920799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:35.922043 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:35.922043 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:35.922043 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:35.922043 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:35.922043 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 604/1215] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:35.926501 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455e00f-d853-498f-be27-66ada1ff44c5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:35.987344 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:36.024132 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455e00f-d853-498f-be27-66ada1ff44c5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:36.073783 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdaa5c3-230d-4168-abe7-3bd2c2711738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:36.074738 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa5c3-230d-4168-abe7-3bd2c2711738 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:36.075164 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa5c3-230d-4168-abe7-3bd2c2711738 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:36.075583 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdaa5c3-230d-4168-abe7-3bd2c2711738 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:36.076633 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdaa5c3-230d-4168-abe7-3bd2c2711738 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:36.077112 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdaa5c3-230d-4168-abe7-3bd2c2711738 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:36.078871 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdaa5c3-230d-4168-abe7-3bd2c2711738 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:36.093649 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 612/1216] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:36.098993 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f90ef7-df89-419f-a7bd-6bde473283d9 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:36.169748 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:36.200592 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f90ef7-df89-419f-a7bd-6bde473283d9 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:36.234251 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22678670-1c8a-462e-8f00-8065c7b8944e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:36.235042 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22678670-1c8a-462e-8f00-8065c7b8944e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:36.235330 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22678670-1c8a-462e-8f00-8065c7b8944e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:36.235612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22678670-1c8a-462e-8f00-8065c7b8944e None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:36.243348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22678670-1c8a-462e-8f00-8065c7b8944e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:36.246771 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22678670-1c8a-462e-8f00-8065c7b8944e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-22678670-1c8a-462e-8f00-8065c7b8944e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:53:36.260890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:36.262137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:36.263343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:53:36.263343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:53:36.263343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:53:36.263343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:53:36.263343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:53:36.263343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:53:36.263343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:36.263343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:36.263343 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 605/1217] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:53:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:36.267974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdaa5c3-230d-4168-abe7-3bd2c2711738 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:36.369494 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdaa5c3-230d-4168-abe7-3bd2c2711738 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:36.393164 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:36.405869 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7c52e-d75d-4651-bc7c-54464c51f62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:36.409925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7c52e-d75d-4651-bc7c-54464c51f62e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:36.409925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7c52e-d75d-4651-bc7c-54464c51f62e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:36.409925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7c52e-d75d-4651-bc7c-54464c51f62e None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:36.409925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7c52e-d75d-4651-bc7c-54464c51f62e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:36.409925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7c52e-d75d-4651-bc7c-54464c51f62e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:36.409925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7c52e-d75d-4651-bc7c-54464c51f62e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:36.422244 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 613/1218] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:53:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:36.428897 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22678670-1c8a-462e-8f00-8065c7b8944e None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:36.530233 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22678670-1c8a-462e-8f00-8065c7b8944e None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:36.559675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:36.565991 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78583c3-743f-4b9c-8e0b-7a6a58bce352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:36.566651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78583c3-743f-4b9c-8e0b-7a6a58bce352 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:36.566937 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78583c3-743f-4b9c-8e0b-7a6a58bce352 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:36.567213 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78583c3-743f-4b9c-8e0b-7a6a58bce352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:36.568038 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78583c3-743f-4b9c-8e0b-7a6a58bce352 None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=c7f307747ddc4b2c8657ace75ce5af29) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:36.568377 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78583c3-743f-4b9c-8e0b-7a6a58bce352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:36.570356 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78583c3-743f-4b9c-8e0b-7a6a58bce352 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:36.575966 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 606/1219] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:53:36 2022] GET /identity/v3/role_assignments?user.id=c7f307747ddc4b2c8657ace75ce5af29&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:38.797617 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcff58-98e5-45fa-8bb3-63c999ae33ef None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:38.798204 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcff58-98e5-45fa-8bb3-63c999ae33ef None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:38.798204 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcff58-98e5-45fa-8bb3-63c999ae33ef None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:38.799515 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 614/1220] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:53:38.805360 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce1651-331b-40e3-ae13-f700b962062a None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:38.805793 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce1651-331b-40e3-ae13-f700b962062a None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:38.805987 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dce1651-331b-40e3-ae13-f700b962062a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:38.823770 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-1dce1651-331b-40e3-ae13-f700b962062a None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:53:38.825172 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-1dce1651-331b-40e3-ae13-f700b962062a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:53:38.892853 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 607/1221] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:53:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:53:38.916579 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcff58-98e5-45fa-8bb3-63c999ae33ef None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:39.020253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcff58-98e5-45fa-8bb3-63c999ae33ef None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:39.058841 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ff256-8540-4fcd-85a1-4378309f0283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:39.059550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ff256-8540-4fcd-85a1-4378309f0283 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:39.059830 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ff256-8540-4fcd-85a1-4378309f0283 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:39.060103 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ff256-8540-4fcd-85a1-4378309f0283 None admin] PATH_INFO: `/v3/roles/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:39.070669 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ff256-8540-4fcd-85a1-4378309f0283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-507ff256-8540-4fcd-85a1-4378309f0283 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:53:39.078288 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:39.084431 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:39.086301 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 615/1222] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:53:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:39.087348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce1651-331b-40e3-ae13-f700b962062a None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:39.188454 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dce1651-331b-40e3-ae13-f700b962062a None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:39.214111 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:39.225055 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32700f8a-a515-4c46-a8fc-291cd7385176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:39.225757 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32700f8a-a515-4c46-a8fc-291cd7385176 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:39.226043 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32700f8a-a515-4c46-a8fc-291cd7385176 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:39.226323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32700f8a-a515-4c46-a8fc-291cd7385176 None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:39.227129 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32700f8a-a515-4c46-a8fc-291cd7385176 None admin] RBAC: Adding query filter params (name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:39.227492 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32700f8a-a515-4c46-a8fc-291cd7385176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:39.228901 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32700f8a-a515-4c46-a8fc-291cd7385176 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:39.237114 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 608/1223] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:53:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:39.244507 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ff256-8540-4fcd-85a1-4378309f0283 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:39.289111 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:39.346578 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ff256-8540-4fcd-85a1-4378309f0283 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:39.380547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae67092-7bfd-40b4-a03b-573e77cae520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:39.381338 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67092-7bfd-40b4-a03b-573e77cae520 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:39.382149 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67092-7bfd-40b4-a03b-573e77cae520 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:39.382424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae67092-7bfd-40b4-a03b-573e77cae520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:39.393374 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae67092-7bfd-40b4-a03b-573e77cae520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:39.394863 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae67092-7bfd-40b4-a03b-573e77cae520 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-cae67092-7bfd-40b4-a03b-573e77cae520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:39.413079 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:39.414373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:39.415774 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:39.415774 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:39.415774 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:39.415774 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:39.416257 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 616/1224] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:39.420122 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32700f8a-a515-4c46-a8fc-291cd7385176 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:39.525310 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32700f8a-a515-4c46-a8fc-291cd7385176 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:39.559897 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3135d-f31f-48c7-85dc-67f085edb858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:39.560598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3135d-f31f-48c7-85dc-67f085edb858 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:39.560889 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3135d-f31f-48c7-85dc-67f085edb858 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:39.561435 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3135d-f31f-48c7-85dc-67f085edb858 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:39.562167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3135d-f31f-48c7-85dc-67f085edb858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:39.562479 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3135d-f31f-48c7-85dc-67f085edb858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:39.563881 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3135d-f31f-48c7-85dc-67f085edb858 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:39.576819 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 609/1225] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:39.582337 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae67092-7bfd-40b4-a03b-573e77cae520 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:39.676069 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:39.683321 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae67092-7bfd-40b4-a03b-573e77cae520 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:39.718151 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e2e8a-152f-4beb-a8af-40a5db1ccb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:39.719005 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e2e8a-152f-4beb-a8af-40a5db1ccb15 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:39.719295 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e2e8a-152f-4beb-a8af-40a5db1ccb15 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:39.719575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e2e8a-152f-4beb-a8af-40a5db1ccb15 None admin] PATH_INFO: `/v3/users/placement` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:39.727245 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e2e8a-152f-4beb-a8af-40a5db1ccb15 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:39.728749 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e2e8a-152f-4beb-a8af-40a5db1ccb15 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-d96e2e8a-152f-4beb-a8af-40a5db1ccb15 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:53:39.743718 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:39.750128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:39.751472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:39.751472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:39.751472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:53:39.751472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:53:39.751472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:53:39.751472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:53:39.751472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:53:39.751472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:53:39.751472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:39.751472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:39.751472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 617/1226] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:53:39.752218 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3135d-f31f-48c7-85dc-67f085edb858 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:39.798022 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:39.853707 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3135d-f31f-48c7-85dc-67f085edb858 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:39.890334 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facccbf0-d0fa-4590-87f2-3959dff9f510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:39.890334 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facccbf0-d0fa-4590-87f2-3959dff9f510 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:39.890334 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facccbf0-d0fa-4590-87f2-3959dff9f510 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:39.890334 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facccbf0-d0fa-4590-87f2-3959dff9f510 None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:39.890838 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facccbf0-d0fa-4590-87f2-3959dff9f510 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:39.890927 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facccbf0-d0fa-4590-87f2-3959dff9f510 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:39.892366 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facccbf0-d0fa-4590-87f2-3959dff9f510 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:39.911100 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 610/1227] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:53:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:39.917111 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e2e8a-152f-4beb-a8af-40a5db1ccb15 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:40.021929 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e2e8a-152f-4beb-a8af-40a5db1ccb15 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:40.056698 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb62135-f92e-4e6b-9d71-c4009d2e0165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:40.057513 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62135-f92e-4e6b-9d71-c4009d2e0165 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:40.057796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62135-f92e-4e6b-9d71-c4009d2e0165 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:40.058071 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb62135-f92e-4e6b-9d71-c4009d2e0165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:40.069377 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb62135-f92e-4e6b-9d71-c4009d2e0165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:40.070897 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb62135-f92e-4e6b-9d71-c4009d2e0165 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-1bb62135-f92e-4e6b-9d71-c4009d2e0165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:40.087763 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:40.093613 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:40.094779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:40.094779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:40.094779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:40.094779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:40.094779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 618/1228] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:40.095128 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facccbf0-d0fa-4590-87f2-3959dff9f510 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:40.195498 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facccbf0-d0fa-4590-87f2-3959dff9f510 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:40.197846 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:40.229526 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d87c5-6c0d-4d18-8c3c-36ce139950ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:40.230254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d87c5-6c0d-4d18-8c3c-36ce139950ec None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:40.230850 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d87c5-6c0d-4d18-8c3c-36ce139950ec None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:40.231315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d87c5-6c0d-4d18-8c3c-36ce139950ec None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:40.232252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d87c5-6c0d-4d18-8c3c-36ce139950ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:40.232951 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d87c5-6c0d-4d18-8c3c-36ce139950ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:40.234329 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d87c5-6c0d-4d18-8c3c-36ce139950ec None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:40.246796 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 611/1229] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:53:40.252282 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb62135-f92e-4e6b-9d71-c4009d2e0165 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:40.262364 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:40.357239 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb62135-f92e-4e6b-9d71-c4009d2e0165 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:40.392259 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95b7ad-1f48-4f83-ae02-1da2c1d98079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:40.393093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95b7ad-1f48-4f83-ae02-1da2c1d98079 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:40.393382 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95b7ad-1f48-4f83-ae02-1da2c1d98079 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:40.393663 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95b7ad-1f48-4f83-ae02-1da2c1d98079 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:40.400657 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95b7ad-1f48-4f83-ae02-1da2c1d98079 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:40.402138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95b7ad-1f48-4f83-ae02-1da2c1d98079 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-5e95b7ad-1f48-4f83-ae02-1da2c1d98079 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:53:40.416480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:40.417714 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:40.418882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:53:40.418882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:53:40.418882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:53:40.418882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:53:40.418882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:53:40.418882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:53:40.418882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:40.418882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:40.418882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 619/1230] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:53:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:40.423675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d87c5-6c0d-4d18-8c3c-36ce139950ec None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:40.535600 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d87c5-6c0d-4d18-8c3c-36ce139950ec None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:40.571324 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c2200-1a51-4a02-8eed-b4dd53ee4ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:40.572029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c2200-1a51-4a02-8eed-b4dd53ee4ba7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:40.572319 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c2200-1a51-4a02-8eed-b4dd53ee4ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:40.572636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c2200-1a51-4a02-8eed-b4dd53ee4ba7 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:40.573550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c2200-1a51-4a02-8eed-b4dd53ee4ba7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:40.573865 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c2200-1a51-4a02-8eed-b4dd53ee4ba7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:40.575346 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c2200-1a51-4a02-8eed-b4dd53ee4ba7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:40.589959 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 612/1231] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:53:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:40.596141 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95b7ad-1f48-4f83-ae02-1da2c1d98079 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:40.666970 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:40.700294 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95b7ad-1f48-4f83-ae02-1da2c1d98079 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:40.735355 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee22423-b2ca-4dac-9ed8-61495c2a64bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:40.736259 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee22423-b2ca-4dac-9ed8-61495c2a64bd None admin] REQUEST_METHOD: `PUT` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:40.736573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee22423-b2ca-4dac-9ed8-61495c2a64bd None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:40.736853 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee22423-b2ca-4dac-9ed8-61495c2a64bd None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/c7f307747ddc4b2c8657ace75ce5af29/roles/41728d78a94d4b7abb951f8a3cee2ce2` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:40.751658 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee22423-b2ca-4dac-9ed8-61495c2a64bd None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=c7f307747ddc4b2c8657ace75ce5af29, role_id=41728d78a94d4b7abb951f8a3cee2ce2)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:40.753163 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee22423-b2ca-4dac-9ed8-61495c2a64bd None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:40.858504 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 620/1232] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:53:40 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/c7f307747ddc4b2c8657ace75ce5af29/roles/41728d78a94d4b7abb951f8a3cee2ce2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:53:43.107822 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318855-336d-41b4-a584-c497ea52333c None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:43.108436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318855-336d-41b4-a584-c497ea52333c None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:43.108436 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318855-336d-41b4-a584-c497ea52333c None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:43.109801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 613/1233] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:53:43.115290 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89521577-cd3f-4852-8ef4-c1353c87ff84 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:43.115691 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89521577-cd3f-4852-8ef4-c1353c87ff84 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:43.116435 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89521577-cd3f-4852-8ef4-c1353c87ff84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:43.133173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-89521577-cd3f-4852-8ef4-c1353c87ff84 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:53:43.134371 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-89521577-cd3f-4852-8ef4-c1353c87ff84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:53:43.213241 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 621/1234] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:53:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:53:43.237453 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1318855-336d-41b4-a584-c497ea52333c None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:43.264167 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:43.341474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1318855-336d-41b4-a584-c497ea52333c None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:43.374201 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e09413-0469-48f2-bfbc-540c3e82b2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:43.374996 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e09413-0469-48f2-bfbc-540c3e82b2de None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:43.375277 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e09413-0469-48f2-bfbc-540c3e82b2de None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:43.375550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e09413-0469-48f2-bfbc-540c3e82b2de None admin] PATH_INFO: `/v3/roles/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:43.387174 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e09413-0469-48f2-bfbc-540c3e82b2de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-57e09413-0469-48f2-bfbc-540c3e82b2de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:53:43.394895 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:43.396199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:43.397597 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 614/1235] 10.222.0.88 () {56 vars in 1102 bytes} [Thu May 19 20:53:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:43.402477 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89521577-cd3f-4852-8ef4-c1353c87ff84 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:43.448957 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:43.501980 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89521577-cd3f-4852-8ef4-c1353c87ff84 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:43.536844 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62dad8-e7b4-4075-911c-2f457d01ec2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:43.537555 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62dad8-e7b4-4075-911c-2f457d01ec2b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:43.537845 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62dad8-e7b4-4075-911c-2f457d01ec2b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:43.538125 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62dad8-e7b4-4075-911c-2f457d01ec2b None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:43.538882 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62dad8-e7b4-4075-911c-2f457d01ec2b None admin] RBAC: Adding query filter params (name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:43.539196 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62dad8-e7b4-4075-911c-2f457d01ec2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:43.540621 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62dad8-e7b4-4075-911c-2f457d01ec2b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:43.549070 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 622/1236] 10.222.0.88 () {56 vars in 1103 bytes} [Thu May 19 20:53:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:43.555490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e09413-0469-48f2-bfbc-540c3e82b2de None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:43.640152 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:43.654907 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e09413-0469-48f2-bfbc-540c3e82b2de None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:43.688233 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fedb0-f884-4ec7-bfe3-b7b7cf78fad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:43.689020 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fedb0-f884-4ec7-bfe3-b7b7cf78fad0 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:43.689304 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fedb0-f884-4ec7-bfe3-b7b7cf78fad0 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:43.689578 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fedb0-f884-4ec7-bfe3-b7b7cf78fad0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:43.699259 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fedb0-f884-4ec7-bfe3-b7b7cf78fad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:43.700716 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457fedb0-f884-4ec7-bfe3-b7b7cf78fad0 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-457fedb0-f884-4ec7-bfe3-b7b7cf78fad0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:43.718535 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:43.719835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:43.721174 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:43.721174 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:43.721174 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:43.721174 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:43.724392 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 615/1237] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:43.726090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62dad8-e7b4-4075-911c-2f457d01ec2b None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:43.837403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62dad8-e7b4-4075-911c-2f457d01ec2b None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:43.846824 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:43.872486 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad78cf-c728-4b15-882c-308d1835e008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:43.873195 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad78cf-c728-4b15-882c-308d1835e008 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:43.873483 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad78cf-c728-4b15-882c-308d1835e008 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:43.873766 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad78cf-c728-4b15-882c-308d1835e008 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:43.874499 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad78cf-c728-4b15-882c-308d1835e008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:43.874808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad78cf-c728-4b15-882c-308d1835e008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:43.876198 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad78cf-c728-4b15-882c-308d1835e008 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:43.887329 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 623/1238] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:43.893319 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fedb0-f884-4ec7-bfe3-b7b7cf78fad0 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:43.992393 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fedb0-f884-4ec7-bfe3-b7b7cf78fad0 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:44.024682 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8705b69-c101-4762-b4e3-36482e5e22a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:44.025481 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8705b69-c101-4762-b4e3-36482e5e22a9 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:44.025759 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8705b69-c101-4762-b4e3-36482e5e22a9 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:44.026029 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8705b69-c101-4762-b4e3-36482e5e22a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:44.028805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:44.033950 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8705b69-c101-4762-b4e3-36482e5e22a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:44.034650 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8705b69-c101-4762-b4e3-36482e5e22a9 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-c8705b69-c101-4762-b4e3-36482e5e22a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:53:44.049954 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:44.051164 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:44.052648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:44.052648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:44.052648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:53:44.052648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:53:44.052648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:53:44.052648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:53:44.052648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:53:44.052648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:53:44.052648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:44.052648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:44.052648 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 616/1239] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:44.057799 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad78cf-c728-4b15-882c-308d1835e008 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:44.159408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad78cf-c728-4b15-882c-308d1835e008 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:44.193833 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fabdca-c6f3-4997-a2d4-1456c9570f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:44.194595 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fabdca-c6f3-4997-a2d4-1456c9570f4e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:44.194877 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fabdca-c6f3-4997-a2d4-1456c9570f4e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:44.195163 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fabdca-c6f3-4997-a2d4-1456c9570f4e None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:44.196003 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fabdca-c6f3-4997-a2d4-1456c9570f4e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:44.196318 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fabdca-c6f3-4997-a2d4-1456c9570f4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:44.197803 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fabdca-c6f3-4997-a2d4-1456c9570f4e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:44.215520 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 624/1240] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:53:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:44.221478 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8705b69-c101-4762-b4e3-36482e5e22a9 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:44.322758 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8705b69-c101-4762-b4e3-36482e5e22a9 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:44.357387 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ab60b-78d0-48c1-9d9d-a5f9c5960bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:44.358032 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ab60b-78d0-48c1-9d9d-a5f9c5960bda None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:44.358312 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ab60b-78d0-48c1-9d9d-a5f9c5960bda None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:44.358588 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ab60b-78d0-48c1-9d9d-a5f9c5960bda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:44.422016 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:44.428480 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ab60b-78d0-48c1-9d9d-a5f9c5960bda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:44.429930 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ab60b-78d0-48c1-9d9d-a5f9c5960bda None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:44.529941 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 617/1241] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:53:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:53:44.535781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fabdca-c6f3-4997-a2d4-1456c9570f4e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:44.573496 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:44.646059 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fabdca-c6f3-4997-a2d4-1456c9570f4e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:44.681149 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ba8ba-9b19-4432-b8b5-8bead092767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:44.681872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ba8ba-9b19-4432-b8b5-8bead092767f None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:44.682154 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ba8ba-9b19-4432-b8b5-8bead092767f None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:44.682447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ba8ba-9b19-4432-b8b5-8bead092767f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:44.692990 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ba8ba-9b19-4432-b8b5-8bead092767f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:44.695066 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ba8ba-9b19-4432-b8b5-8bead092767f None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-3d1ba8ba-9b19-4432-b8b5-8bead092767f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:44.713564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:44.714839 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:44.715976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:44.715976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:44.715976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:44.715976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:44.715976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 625/1242] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:44.720566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ab60b-78d0-48c1-9d9d-a5f9c5960bda None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:44.813427 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:44.819691 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ab60b-78d0-48c1-9d9d-a5f9c5960bda None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:44.852728 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a15d8d-85de-4579-a1a4-602157093830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:44.853378 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a15d8d-85de-4579-a1a4-602157093830 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:44.853661 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a15d8d-85de-4579-a1a4-602157093830 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:44.853936 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a15d8d-85de-4579-a1a4-602157093830 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:44.854656 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a15d8d-85de-4579-a1a4-602157093830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:44.854964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a15d8d-85de-4579-a1a4-602157093830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:44.856318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a15d8d-85de-4579-a1a4-602157093830 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:44.873481 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 618/1243] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:44.879145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ba8ba-9b19-4432-b8b5-8bead092767f None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:44.958525 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:44.978879 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ba8ba-9b19-4432-b8b5-8bead092767f None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:45.013087 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d206f84-1faa-4ba5-b177-401344e848e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:45.013905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d206f84-1faa-4ba5-b177-401344e848e2 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:45.014188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d206f84-1faa-4ba5-b177-401344e848e2 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:45.014461 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d206f84-1faa-4ba5-b177-401344e848e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:45.020974 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d206f84-1faa-4ba5-b177-401344e848e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:45.022472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d206f84-1faa-4ba5-b177-401344e848e2 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-3d206f84-1faa-4ba5-b177-401344e848e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:53:45.036398 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:45.037635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:45.039126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:53:45.039126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:53:45.039126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:53:45.039126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:53:45.039126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:53:45.039126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:53:45.039126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:45.039126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:45.039126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 626/1244] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:53:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:45.044094 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a15d8d-85de-4579-a1a4-602157093830 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:45.144089 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a15d8d-85de-4579-a1a4-602157093830 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:45.177300 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dea784-ce74-41e8-9001-7c45e1728d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:45.177956 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dea784-ce74-41e8-9001-7c45e1728d4e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:45.178816 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dea784-ce74-41e8-9001-7c45e1728d4e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:45.179147 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dea784-ce74-41e8-9001-7c45e1728d4e None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:45.179926 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dea784-ce74-41e8-9001-7c45e1728d4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:45.180236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dea784-ce74-41e8-9001-7c45e1728d4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:45.181683 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dea784-ce74-41e8-9001-7c45e1728d4e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:45.194797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 619/1245] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:53:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:45.200930 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d206f84-1faa-4ba5-b177-401344e848e2 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:45.302119 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d206f84-1faa-4ba5-b177-401344e848e2 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:45.336514 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d506e-4204-4d77-ab31-21956469cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:45.337163 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d506e-4204-4d77-ab31-21956469cc42 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:45.337447 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d506e-4204-4d77-ab31-21956469cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:45.337723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d506e-4204-4d77-ab31-21956469cc42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:45.338547 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d506e-4204-4d77-ab31-21956469cc42 None admin] RBAC: Adding query filter params (role.id=41728d78a94d4b7abb951f8a3cee2ce2, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=c7f307747ddc4b2c8657ace75ce5af29) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:45.338868 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d506e-4204-4d77-ab31-21956469cc42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:45.340303 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d506e-4204-4d77-ab31-21956469cc42 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:45.346145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 627/1246] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:53:45 2022] GET /identity/v3/role_assignments?user.id=c7f307747ddc4b2c8657ace75ce5af29&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=41728d78a94d4b7abb951f8a3cee2ce2 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:47.603377 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0c8d2-6a50-4f78-847f-f3ad14b73ff5 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:47.603955 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0c8d2-6a50-4f78-847f-f3ad14b73ff5 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:47.603955 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f0c8d2-6a50-4f78-847f-f3ad14b73ff5 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:47.605366 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 620/1247] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:53:47.610546 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31822f7f-312e-4a3c-9058-90d463194dbc None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:47.611006 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31822f7f-312e-4a3c-9058-90d463194dbc None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:47.611135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31822f7f-312e-4a3c-9058-90d463194dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:47.616378 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:47.628407 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-31822f7f-312e-4a3c-9058-90d463194dbc None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:53:47.629623 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-31822f7f-312e-4a3c-9058-90d463194dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:53:47.698737 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 628/1248] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:53:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:53:47.722254 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0c8d2-6a50-4f78-847f-f3ad14b73ff5 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:47.740249 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:47.824913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f0c8d2-6a50-4f78-847f-f3ad14b73ff5 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:47.858924 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa98c0-854b-4ba7-afdb-5e0371f09801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:47.859649 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa98c0-854b-4ba7-afdb-5e0371f09801 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:47.859933 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa98c0-854b-4ba7-afdb-5e0371f09801 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:47.860208 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa98c0-854b-4ba7-afdb-5e0371f09801 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:47.870675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa98c0-854b-4ba7-afdb-5e0371f09801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-83fa98c0-854b-4ba7-afdb-5e0371f09801 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:53:47.878323 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:47.879657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:47.881138 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 621/1249] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:53:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:47.885614 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31822f7f-312e-4a3c-9058-90d463194dbc None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:47.986213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31822f7f-312e-4a3c-9058-90d463194dbc None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:48.015528 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:48.020170 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1f670-14d7-460b-936a-1d9f25231c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:48.020860 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1f670-14d7-460b-936a-1d9f25231c8d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:48.021145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1f670-14d7-460b-936a-1d9f25231c8d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:48.021420 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1f670-14d7-460b-936a-1d9f25231c8d None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:48.022217 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1f670-14d7-460b-936a-1d9f25231c8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:48.022523 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1f670-14d7-460b-936a-1d9f25231c8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:48.023940 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1f670-14d7-460b-936a-1d9f25231c8d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:48.031790 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 629/1250] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:53:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:48.037542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa98c0-854b-4ba7-afdb-5e0371f09801 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:48.116642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:48.138770 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa98c0-854b-4ba7-afdb-5e0371f09801 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:48.171947 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9c4bc-f887-4eb2-9ad2-b99a3ec534db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:48.172774 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c4bc-f887-4eb2-9ad2-b99a3ec534db None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:48.173078 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c4bc-f887-4eb2-9ad2-b99a3ec534db None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:48.173404 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c4bc-f887-4eb2-9ad2-b99a3ec534db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:48.184752 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9c4bc-f887-4eb2-9ad2-b99a3ec534db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:48.186433 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9c4bc-f887-4eb2-9ad2-b99a3ec534db None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-c4c9c4bc-f887-4eb2-9ad2-b99a3ec534db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:48.203802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:48.208651 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:48.210088 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:48.210088 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:48.210088 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:48.210088 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:48.210088 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 622/1251] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:48.210892 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1f670-14d7-460b-936a-1d9f25231c8d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:48.313221 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1f670-14d7-460b-936a-1d9f25231c8d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:48.347246 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cdd56-4da5-4654-b02c-d6c4a6afd7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:48.347946 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cdd56-4da5-4654-b02c-d6c4a6afd7f5 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:48.348230 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cdd56-4da5-4654-b02c-d6c4a6afd7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:48.348537 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392cdd56-4da5-4654-b02c-d6c4a6afd7f5 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:48.349261 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cdd56-4da5-4654-b02c-d6c4a6afd7f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:48.349573 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cdd56-4da5-4654-b02c-d6c4a6afd7f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:48.350995 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392cdd56-4da5-4654-b02c-d6c4a6afd7f5 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:48.362619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 630/1252] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:53:48.368416 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9c4bc-f887-4eb2-9ad2-b99a3ec534db None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:48.467583 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9c4bc-f887-4eb2-9ad2-b99a3ec534db None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:48.495320 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:48.503061 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a559210-070a-415e-8cbf-480c9fc2637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:48.503861 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a559210-070a-415e-8cbf-480c9fc2637a None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:48.504148 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a559210-070a-415e-8cbf-480c9fc2637a None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:48.504453 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a559210-070a-415e-8cbf-480c9fc2637a None admin] PATH_INFO: `/v3/users/placement` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:48.512005 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a559210-070a-415e-8cbf-480c9fc2637a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:48.513475 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a559210-070a-415e-8cbf-480c9fc2637a None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-0a559210-070a-415e-8cbf-480c9fc2637a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:53:48.527701 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:48.529028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:48.530598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:48.530598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:48.530598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:53:48.530598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:53:48.530598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:53:48.530598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:53:48.530598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:53:48.530598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:53:48.530598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:48.530598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:48.530598 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 623/1253] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:48.534894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cdd56-4da5-4654-b02c-d6c4a6afd7f5 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:48.591788 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:48.639658 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392cdd56-4da5-4654-b02c-d6c4a6afd7f5 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:48.674357 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa735e6c-66ef-454f-982f-5cd8892bf004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:48.675021 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa735e6c-66ef-454f-982f-5cd8892bf004 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:48.675343 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa735e6c-66ef-454f-982f-5cd8892bf004 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:48.675620 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa735e6c-66ef-454f-982f-5cd8892bf004 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:48.676486 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa735e6c-66ef-454f-982f-5cd8892bf004 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:48.676798 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa735e6c-66ef-454f-982f-5cd8892bf004 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:48.678252 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa735e6c-66ef-454f-982f-5cd8892bf004 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:48.695400 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 631/1254] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:53:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:48.701499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a559210-070a-415e-8cbf-480c9fc2637a None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:48.802037 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a559210-070a-415e-8cbf-480c9fc2637a None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:48.835381 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fffee6-1ce3-42d5-9adf-99f1c14981c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:48.836025 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fffee6-1ce3-42d5-9adf-99f1c14981c7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:48.836312 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fffee6-1ce3-42d5-9adf-99f1c14981c7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:48.836624 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fffee6-1ce3-42d5-9adf-99f1c14981c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:48.891174 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:48.903467 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fffee6-1ce3-42d5-9adf-99f1c14981c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:48.904933 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fffee6-1ce3-42d5-9adf-99f1c14981c7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:49.003933 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 624/1255] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:53:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:53:49.010274 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa735e6c-66ef-454f-982f-5cd8892bf004 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:49.109541 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa735e6c-66ef-454f-982f-5cd8892bf004 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:49.122474 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:49.148603 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5d000-4245-4ee6-8273-027d1debcd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:49.148603 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5d000-4245-4ee6-8273-027d1debcd2a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:49.148603 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5d000-4245-4ee6-8273-027d1debcd2a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:49.148603 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e5d000-4245-4ee6-8273-027d1debcd2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:49.155791 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5d000-4245-4ee6-8273-027d1debcd2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:49.157312 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e5d000-4245-4ee6-8273-027d1debcd2a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-27e5d000-4245-4ee6-8273-027d1debcd2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:49.174693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:49.175935 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:49.177093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:49.177093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:49.177093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:49.177093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:49.177093 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 632/1256] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 19 20:53:49.181607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fffee6-1ce3-42d5-9adf-99f1c14981c7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:49.268130 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:49.280483 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fffee6-1ce3-42d5-9adf-99f1c14981c7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:49.314657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bf8bc-bf17-4595-9152-d10b311bbf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:49.343348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bf8bc-bf17-4595-9152-d10b311bbf19 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:49.343348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bf8bc-bf17-4595-9152-d10b311bbf19 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:49.343348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bf8bc-bf17-4595-9152-d10b311bbf19 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:49.343348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bf8bc-bf17-4595-9152-d10b311bbf19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:49.343348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bf8bc-bf17-4595-9152-d10b311bbf19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:49.343348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bf8bc-bf17-4595-9152-d10b311bbf19 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:49.343964 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 625/1257] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:49.344069 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5d000-4245-4ee6-8273-027d1debcd2a None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:49.435720 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e5d000-4245-4ee6-8273-027d1debcd2a None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:49.469885 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c77020-c404-475b-ac7a-b7f165466559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:49.470678 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c77020-c404-475b-ac7a-b7f165466559 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:49.470966 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c77020-c404-475b-ac7a-b7f165466559 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:49.471246 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c77020-c404-475b-ac7a-b7f165466559 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:49.478005 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c77020-c404-475b-ac7a-b7f165466559 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:49.479504 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c77020-c404-475b-ac7a-b7f165466559 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-61c77020-c404-475b-ac7a-b7f165466559 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:53:49.493835 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:49.495054 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:49.496403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:53:49.496403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:53:49.496403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:53:49.496403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:53:49.496403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:53:49.496403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:53:49.496403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:49.496403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:49.496403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 633/1258] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:53:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:49.500965 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bf8bc-bf17-4595-9152-d10b311bbf19 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:49.600802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bf8bc-bf17-4595-9152-d10b311bbf19 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:49.634300 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff0008-05b0-4950-becf-f24584204285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:49.634957 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff0008-05b0-4950-becf-f24584204285 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:49.635243 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff0008-05b0-4950-becf-f24584204285 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:49.635528 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff0008-05b0-4950-becf-f24584204285 None admin] PATH_INFO: `/v3/projects` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:49.636312 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff0008-05b0-4950-becf-f24584204285 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:49.636717 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff0008-05b0-4950-becf-f24584204285 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:49.638137 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff0008-05b0-4950-becf-f24584204285 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:49.649976 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 626/1259] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:53:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:49.656403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c77020-c404-475b-ac7a-b7f165466559 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:49.683373 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:49.758612 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c77020-c404-475b-ac7a-b7f165466559 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:49.794159 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76979f2c-2d97-459f-b684-3c624ed8400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:49.794811 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76979f2c-2d97-459f-b684-3c624ed8400b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:49.795098 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76979f2c-2d97-459f-b684-3c624ed8400b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:49.795380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76979f2c-2d97-459f-b684-3c624ed8400b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:49.796210 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76979f2c-2d97-459f-b684-3c624ed8400b None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=c7f307747ddc4b2c8657ace75ce5af29) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:49.796552 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76979f2c-2d97-459f-b684-3c624ed8400b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:49.798018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76979f2c-2d97-459f-b684-3c624ed8400b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:49.804123 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 634/1260] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:53:49 2022] GET /identity/v3/role_assignments?user.id=c7f307747ddc4b2c8657ace75ce5af29&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:52.022504 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceb7bd-aa1b-4a78-8af7-389f93a40c99 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:52.023102 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceb7bd-aa1b-4a78-8af7-389f93a40c99 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:52.023102 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ceb7bd-aa1b-4a78-8af7-389f93a40c99 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:52.024474 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 627/1261] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:53:52.029686 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d842e-6361-48e2-acaf-2a3c058cf615 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:52.030169 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d842e-6361-48e2-acaf-2a3c058cf615 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:52.030285 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d842e-6361-48e2-acaf-2a3c058cf615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:52.047098 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-232d842e-6361-48e2-acaf-2a3c058cf615 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:53:52.048359 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-232d842e-6361-48e2-acaf-2a3c058cf615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:53:52.114606 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 635/1262] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:53:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:53:52.137617 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ceb7bd-aa1b-4a78-8af7-389f93a40c99 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:52.149203 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:52.246836 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ceb7bd-aa1b-4a78-8af7-389f93a40c99 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:52.281339 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68823f-21a7-42ba-933d-916e7aece596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:52.282239 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68823f-21a7-42ba-933d-916e7aece596 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:52.282521 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68823f-21a7-42ba-933d-916e7aece596 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:52.282844 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68823f-21a7-42ba-933d-916e7aece596 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:52.293008 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68823f-21a7-42ba-933d-916e7aece596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-1e68823f-21a7-42ba-933d-916e7aece596 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:53:52.300746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:52.302101 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:52.304039 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 628/1263] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:53:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:52.308562 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d842e-6361-48e2-acaf-2a3c058cf615 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:52.328127 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:52.406815 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d842e-6361-48e2-acaf-2a3c058cf615 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:52.440806 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fb246-2758-4902-8fda-0c69c9604566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:52.441476 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fb246-2758-4902-8fda-0c69c9604566 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:52.441760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fb246-2758-4902-8fda-0c69c9604566 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:52.442040 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420fb246-2758-4902-8fda-0c69c9604566 None admin] PATH_INFO: `/v3/roles` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:52.442853 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fb246-2758-4902-8fda-0c69c9604566 None admin] RBAC: Adding query filter params (name=admin) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:52.443168 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fb246-2758-4902-8fda-0c69c9604566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:52.445212 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420fb246-2758-4902-8fda-0c69c9604566 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:52.453491 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 636/1264] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:53:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:52.459805 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68823f-21a7-42ba-933d-916e7aece596 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:52.528417 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:52.563907 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68823f-21a7-42ba-933d-916e7aece596 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:52.598874 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda0cc3-ad92-4dbc-a78b-cc5661855a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:52.599657 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda0cc3-ad92-4dbc-a78b-cc5661855a43 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:52.600600 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda0cc3-ad92-4dbc-a78b-cc5661855a43 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:52.600882 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda0cc3-ad92-4dbc-a78b-cc5661855a43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:52.611524 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda0cc3-ad92-4dbc-a78b-cc5661855a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:52.612990 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bda0cc3-ad92-4dbc-a78b-cc5661855a43 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-0bda0cc3-ad92-4dbc-a78b-cc5661855a43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:52.630642 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:52.631908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:52.633246 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:52.633246 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:52.633246 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:52.633246 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:52.633246 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 629/1265] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:52.637564 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fb246-2758-4902-8fda-0c69c9604566 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:52.730752 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:52.749787 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420fb246-2758-4902-8fda-0c69c9604566 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:52.784951 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235729f-3e31-43e4-a4dc-e2cc95f99ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:52.785737 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235729f-3e31-43e4-a4dc-e2cc95f99ec4 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:52.786034 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235729f-3e31-43e4-a4dc-e2cc95f99ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:52.786310 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2235729f-3e31-43e4-a4dc-e2cc95f99ec4 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:52.787038 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235729f-3e31-43e4-a4dc-e2cc95f99ec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:52.787351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235729f-3e31-43e4-a4dc-e2cc95f99ec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:52.789267 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2235729f-3e31-43e4-a4dc-e2cc95f99ec4 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:52.800960 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 637/1266] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:52.807217 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda0cc3-ad92-4dbc-a78b-cc5661855a43 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:52.908235 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda0cc3-ad92-4dbc-a78b-cc5661855a43 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:52.929233 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:52.943512 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa1fb2-a4c9-4309-a3f1-5eb40d382dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:52.944420 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1fb2-a4c9-4309-a3f1-5eb40d382dd7 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:52.945834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1fb2-a4c9-4309-a3f1-5eb40d382dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:52.945834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa1fb2-a4c9-4309-a3f1-5eb40d382dd7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:52.952499 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa1fb2-a4c9-4309-a3f1-5eb40d382dd7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:52.953910 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa1fb2-a4c9-4309-a3f1-5eb40d382dd7 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-cbaa1fb2-a4c9-4309-a3f1-5eb40d382dd7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:53:52.967763 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:52.969123 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:52.970677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:52.970677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:52.970677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:53:52.970677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:53:52.970677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:53:52.970677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:53:52.970677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:53:52.970677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:53:52.970677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:52.970677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:52.970677 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 630/1267] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:52.974760 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235729f-3e31-43e4-a4dc-e2cc95f99ec4 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:53.078204 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2235729f-3e31-43e4-a4dc-e2cc95f99ec4 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:53.115267 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e919ce-acf1-483b-b819-46b0378d1b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:53.115976 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e919ce-acf1-483b-b819-46b0378d1b24 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:53.116262 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e919ce-acf1-483b-b819-46b0378d1b24 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:53.116570 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e919ce-acf1-483b-b819-46b0378d1b24 None admin] PATH_INFO: `/v3/users` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:53.117424 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e919ce-acf1-483b-b819-46b0378d1b24 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:53.117734 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e919ce-acf1-483b-b819-46b0378d1b24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:53.119200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e919ce-acf1-483b-b819-46b0378d1b24 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:53.128672 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:53.137472 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 638/1268] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:53:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:53.143144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa1fb2-a4c9-4309-a3f1-5eb40d382dd7 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:53.245049 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa1fb2-a4c9-4309-a3f1-5eb40d382dd7 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:53.284978 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745f966-b6c8-45b3-aa29-5f05921f8ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:53.287192 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f966-b6c8-45b3-aa29-5f05921f8ea9 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:53.287192 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f966-b6c8-45b3-aa29-5f05921f8ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:53.287192 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745f966-b6c8-45b3-aa29-5f05921f8ea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:53.297978 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745f966-b6c8-45b3-aa29-5f05921f8ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:53.299542 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745f966-b6c8-45b3-aa29-5f05921f8ea9 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-0745f966-b6c8-45b3-aa29-5f05921f8ea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:53.317276 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:53.318520 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:53.319997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:53.319997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:53.319997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:53.319997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:53.319997 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 631/1269] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:53.324484 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e919ce-acf1-483b-b819-46b0378d1b24 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:53.424750 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e919ce-acf1-483b-b819-46b0378d1b24 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:53.459970 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7861b-637c-49a6-86d8-00d211c28d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:53.460655 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7861b-637c-49a6-86d8-00d211c28d46 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:53.460939 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7861b-637c-49a6-86d8-00d211c28d46 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:53.461213 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7861b-637c-49a6-86d8-00d211c28d46 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:53.462026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7861b-637c-49a6-86d8-00d211c28d46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:53.462338 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7861b-637c-49a6-86d8-00d211c28d46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:53.463743 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7861b-637c-49a6-86d8-00d211c28d46 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:53.474922 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 639/1270] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:53.480646 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745f966-b6c8-45b3-aa29-5f05921f8ea9 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:53.512210 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:53.580236 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745f966-b6c8-45b3-aa29-5f05921f8ea9 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:53.614357 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6867af8-fef8-4e88-bdfc-101ce3bd413e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:53.615169 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6867af8-fef8-4e88-bdfc-101ce3bd413e None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:53.615450 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6867af8-fef8-4e88-bdfc-101ce3bd413e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:53.615769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6867af8-fef8-4e88-bdfc-101ce3bd413e None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:53.622496 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6867af8-fef8-4e88-bdfc-101ce3bd413e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:53.623935 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6867af8-fef8-4e88-bdfc-101ce3bd413e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-c6867af8-fef8-4e88-bdfc-101ce3bd413e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:53:53.638490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:53.639769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:53.640998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:53:53.640998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:53:53.640998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:53:53.640998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:53:53.640998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:53:53.640998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:53:53.640998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:53.640998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:53.640998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 632/1271] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:53:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:53.646275 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7861b-637c-49a6-86d8-00d211c28d46 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:53.704709 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:53.746711 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7861b-637c-49a6-86d8-00d211c28d46 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:53.781129 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a96c3f-e31f-4638-a478-9a1f3a359e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:53.781788 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a96c3f-e31f-4638-a478-9a1f3a359e18 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:53.782071 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a96c3f-e31f-4638-a478-9a1f3a359e18 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:53.782346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a96c3f-e31f-4638-a478-9a1f3a359e18 None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:53.783214 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a96c3f-e31f-4638-a478-9a1f3a359e18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:53.783529 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a96c3f-e31f-4638-a478-9a1f3a359e18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:53.785012 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a96c3f-e31f-4638-a478-9a1f3a359e18 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:53.796856 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 640/1272] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:53:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:53.803424 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6867af8-fef8-4e88-bdfc-101ce3bd413e None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:53.895864 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:53.904420 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6867af8-fef8-4e88-bdfc-101ce3bd413e None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:53.939105 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fdb26-7d61-47fb-86b5-c8e875495c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:53.939998 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fdb26-7d61-47fb-86b5-c8e875495c9e None admin] REQUEST_METHOD: `PUT` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:53.940318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fdb26-7d61-47fb-86b5-c8e875495c9e None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:53.940620 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fdb26-7d61-47fb-86b5-c8e875495c9e None admin] PATH_INFO: `/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/c7f307747ddc4b2c8657ace75ce5af29/roles/ed68c2195de147c0a30d9150b33e47cf` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:53.942673 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fdb26-7d61-47fb-86b5-c8e875495c9e None admin] RBAC: Authorizing `identity:create_grant(project_id=eb86f5a389064e12bb38c500b225c9ac, user_id=c7f307747ddc4b2c8657ace75ce5af29, role_id=ed68c2195de147c0a30d9150b33e47cf)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:53.944831 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fdb26-7d61-47fb-86b5-c8e875495c9e None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:54.002871 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 633/1273] 10.222.0.88 () {58 vars in 1439 bytes} [Thu May 19 20:53:53 2022] PUT /identity/v3/projects/eb86f5a389064e12bb38c500b225c9ac/users/c7f307747ddc4b2c8657ace75ce5af29/roles/ed68c2195de147c0a30d9150b33e47cf => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 19 20:53:56.231768 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf94a0-449a-4e2f-a277-4b73747fcba1 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:56.232313 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf94a0-449a-4e2f-a277-4b73747fcba1 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:56.232313 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf94a0-449a-4e2f-a277-4b73747fcba1 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:56.233693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 641/1274] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:53:56.239176 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24264-6a27-425d-9e9d-d18bb1b724ea None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:56.239616 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24264-6a27-425d-9e9d-d18bb1b724ea None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:56.241055 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24264-6a27-425d-9e9d-d18bb1b724ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:56.257350 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-3da24264-6a27-425d-9e9d-d18bb1b724ea None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:53:56.258580 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-3da24264-6a27-425d-9e9d-d18bb1b724ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:53:56.327790 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 634/1275] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:53:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:53:56.352138 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf94a0-449a-4e2f-a277-4b73747fcba1 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:56.476229 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf94a0-449a-4e2f-a277-4b73747fcba1 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:56.497240 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:56.516738 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c3670-87a6-4845-a3c1-1ce1c7d0dbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:56.523113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c3670-87a6-4845-a3c1-1ce1c7d0dbf8 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:56.523113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c3670-87a6-4845-a3c1-1ce1c7d0dbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:56.523113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c3670-87a6-4845-a3c1-1ce1c7d0dbf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:56.529821 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312c3670-87a6-4845-a3c1-1ce1c7d0dbf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-312c3670-87a6-4845-a3c1-1ce1c7d0dbf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(role_id) May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 19 20:53:56.538161 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:56.539693 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:56.541175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 642/1276] 10.222.0.88 () {56 vars in 1096 bytes} [Thu May 19 20:53:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:56.545823 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24264-6a27-425d-9e9d-d18bb1b724ea None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:56.641284 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:56.662731 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24264-6a27-425d-9e9d-d18bb1b724ea None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:56.699731 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755698a0-4b6c-40e3-9cd8-954e79a30c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:56.700512 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755698a0-4b6c-40e3-9cd8-954e79a30c5d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:56.706505 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755698a0-4b6c-40e3-9cd8-954e79a30c5d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:56.706505 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755698a0-4b6c-40e3-9cd8-954e79a30c5d None admin] PATH_INFO: `/v3/roles` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:56.706505 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755698a0-4b6c-40e3-9cd8-954e79a30c5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:56.706505 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755698a0-4b6c-40e3-9cd8-954e79a30c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:56.706505 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755698a0-4b6c-40e3-9cd8-954e79a30c5d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:56.716092 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 635/1277] 10.222.0.88 () {56 vars in 1099 bytes} [Thu May 19 20:53:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:56.722228 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c3670-87a6-4845-a3c1-1ce1c7d0dbf8 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:56.832095 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c3670-87a6-4845-a3c1-1ce1c7d0dbf8 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:56.868269 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02722baa-052b-4f42-815b-e9520fe1c5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:56.869097 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02722baa-052b-4f42-815b-e9520fe1c5af None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:56.869385 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02722baa-052b-4f42-815b-e9520fe1c5af None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:56.869662 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02722baa-052b-4f42-815b-e9520fe1c5af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:56.881688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02722baa-052b-4f42-815b-e9520fe1c5af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:56.883173 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02722baa-052b-4f42-815b-e9520fe1c5af None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-02722baa-052b-4f42-815b-e9520fe1c5af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:56.901779 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:56.903133 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:56.904465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:56.904465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:56.904465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:56.904465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:56.904465 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 643/1278] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:56.909063 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755698a0-4b6c-40e3-9cd8-954e79a30c5d None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:57.039342 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755698a0-4b6c-40e3-9cd8-954e79a30c5d None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:57.075856 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d6248-c0bc-40f1-b6d3-4ac67bf3c8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:57.076566 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d6248-c0bc-40f1-b6d3-4ac67bf3c8b0 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:57.076852 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d6248-c0bc-40f1-b6d3-4ac67bf3c8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:57.077134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d6248-c0bc-40f1-b6d3-4ac67bf3c8b0 None admin] PATH_INFO: `/v3/domains` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:57.077867 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d6248-c0bc-40f1-b6d3-4ac67bf3c8b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:57.078183 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d6248-c0bc-40f1-b6d3-4ac67bf3c8b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:57.079711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d6248-c0bc-40f1-b6d3-4ac67bf3c8b0 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:57.091916 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 636/1279] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:57.097985 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02722baa-052b-4f42-815b-e9520fe1c5af None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:57.132534 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:57.207126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02722baa-052b-4f42-815b-e9520fe1c5af None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:57.243283 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd563e2f-b581-4a8c-b89d-cd817bd7b2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:57.244146 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd563e2f-b581-4a8c-b89d-cd817bd7b2fa None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:57.244501 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd563e2f-b581-4a8c-b89d-cd817bd7b2fa None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:57.244788 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd563e2f-b581-4a8c-b89d-cd817bd7b2fa None admin] PATH_INFO: `/v3/users/placement` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:57.253116 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd563e2f-b581-4a8c-b89d-cd817bd7b2fa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:57.254615 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd563e2f-b581-4a8c-b89d-cd817bd7b2fa None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-dd563e2f-b581-4a8c-b89d-cd817bd7b2fa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_user(user_id) May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 19 20:53:57.269805 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:57.271862 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:57.274027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:57.274027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:57.274027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 19 20:53:57.274027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 19 20:53:57.274027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 19 20:53:57.274027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 19 20:53:57.274027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 19 20:53:57.274027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 19 20:53:57.274027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 19 20:53:57.274027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:53:57.274027 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 644/1280] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:57.277147 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d6248-c0bc-40f1-b6d3-4ac67bf3c8b0 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:57.281162 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:57.386671 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d6248-c0bc-40f1-b6d3-4ac67bf3c8b0 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:57.422337 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ac26c-02a9-4f36-ba88-0c51e85f4a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:57.423028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ac26c-02a9-4f36-ba88-0c51e85f4a7b None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:57.423315 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ac26c-02a9-4f36-ba88-0c51e85f4a7b None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:57.423594 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59ac26c-02a9-4f36-ba88-0c51e85f4a7b None admin] PATH_INFO: `/v3/users` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:57.424516 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ac26c-02a9-4f36-ba88-0c51e85f4a7b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:57.424828 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ac26c-02a9-4f36-ba88-0c51e85f4a7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:57.426266 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59ac26c-02a9-4f36-ba88-0c51e85f4a7b None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:57.446832 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 637/1281] 10.222.0.88 () {56 vars in 1143 bytes} [Thu May 19 20:53:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:57.452828 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd563e2f-b581-4a8c-b89d-cd817bd7b2fa None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:57.547958 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:57.558090 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd563e2f-b581-4a8c-b89d-cd817bd7b2fa None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:57.592142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde933a1-55e7-4202-899f-a5774db0f10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:57.592809 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde933a1-55e7-4202-899f-a5774db0f10d None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:57.593091 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde933a1-55e7-4202-899f-a5774db0f10d None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:57.593364 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde933a1-55e7-4202-899f-a5774db0f10d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:57.667677 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde933a1-55e7-4202-899f-a5774db0f10d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:57.669201 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde933a1-55e7-4202-899f-a5774db0f10d None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:57.763715 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:57.774385 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 645/1282] 10.222.0.88 () {58 vars in 1282 bytes} [Thu May 19 20:53:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 19 20:53:57.780434 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ac26c-02a9-4f36-ba88-0c51e85f4a7b None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:57.844067 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:57.883370 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59ac26c-02a9-4f36-ba88-0c51e85f4a7b None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:57.919887 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee39cb-e3a7-41f3-b6b9-87ea46440c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:57.920630 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee39cb-e3a7-41f3-b6b9-87ea46440c91 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:57.920915 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee39cb-e3a7-41f3-b6b9-87ea46440c91 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:57.921192 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee39cb-e3a7-41f3-b6b9-87ea46440c91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:57.933335 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee39cb-e3a7-41f3-b6b9-87ea46440c91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:57.934724 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee39cb-e3a7-41f3-b6b9-87ea46440c91 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-d8ee39cb-e3a7-41f3-b6b9-87ea46440c91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 19 20:53:57.953188 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:57.954514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:57.955799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 19 20:53:57.955799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 19 20:53:57.955799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 19 20:53:57.955799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:57.955799 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 638/1283] 10.222.0.88 () {56 vars in 1108 bytes} [Thu May 19 20:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:57.960920 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde933a1-55e7-4202-899f-a5774db0f10d None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:58.068428 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde933a1-55e7-4202-899f-a5774db0f10d None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:58.104716 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d0cc3-3c00-4e34-8203-0cdc5298ae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:58.108397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d0cc3-3c00-4e34-8203-0cdc5298ae84 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:58.108397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d0cc3-3c00-4e34-8203-0cdc5298ae84 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:58.108397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d0cc3-3c00-4e34-8203-0cdc5298ae84 None admin] PATH_INFO: `/v3/domains` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:58.108397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d0cc3-3c00-4e34-8203-0cdc5298ae84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:58.108397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d0cc3-3c00-4e34-8203-0cdc5298ae84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:58.108397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d0cc3-3c00-4e34-8203-0cdc5298ae84 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:58.120124 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 646/1284] 10.222.0.88 () {56 vars in 1109 bytes} [Thu May 19 20:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:58.126031 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee39cb-e3a7-41f3-b6b9-87ea46440c91 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:58.227144 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee39cb-e3a7-41f3-b6b9-87ea46440c91 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:58.240618 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:58.260777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe7a47-ee85-45f8-824f-8debd5f1c291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:58.261573 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe7a47-ee85-45f8-824f-8debd5f1c291 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:58.261854 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe7a47-ee85-45f8-824f-8debd5f1c291 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:58.262128 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe7a47-ee85-45f8-824f-8debd5f1c291 None admin] PATH_INFO: `/v3/projects/service` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:58.268751 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe7a47-ee85-45f8-824f-8debd5f1c291 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:58.270183 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe7a47-ee85-45f8-824f-8debd5f1c291 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application [None req-64fe7a47-ee85-45f8-824f-8debd5f1c291 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(project_id) May 19 20:53:58.284801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application with Lock( May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._enter() May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.creator() May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:53:58.292179 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application created_value = creator( May 19 20:53:58.293331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 19 20:53:58.293331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 19 20:53:58.293331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 19 20:53:58.293331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 19 20:53:58.293331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 19 20:53:58.293331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 19 20:53:58.293331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 19 20:53:58.293331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: ERROR keystone.server.flask.application May 19 20:53:58.293331 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 639/1285] 10.222.0.88 () {56 vars in 1111 bytes} [Thu May 19 20:53:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:53:58.293997 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d0cc3-3c00-4e34-8203-0cdc5298ae84 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:58.342832 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:53:58.391141 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d0cc3-3c00-4e34-8203-0cdc5298ae84 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:58.424481 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdd474-c828-4829-af21-42a104de778a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:58.434776 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdd474-c828-4829-af21-42a104de778a None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:58.434776 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdd474-c828-4829-af21-42a104de778a None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:58.434776 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdd474-c828-4829-af21-42a104de778a None admin] PATH_INFO: `/v3/projects` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:58.434776 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdd474-c828-4829-af21-42a104de778a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:58.434776 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdd474-c828-4829-af21-42a104de778a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:58.434776 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdd474-c828-4829-af21-42a104de778a None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:58.440796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 647/1286] 10.222.0.88 () {56 vars in 1148 bytes} [Thu May 19 20:53:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:53:58.446935 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe7a47-ee85-45f8-824f-8debd5f1c291 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:53:58.548812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe7a47-ee85-45f8-824f-8debd5f1c291 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:53:58.583738 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd393b07-afa5-438e-bdb7-9c41b2dd8722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:53:58.584938 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd393b07-afa5-438e-bdb7-9c41b2dd8722 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:53:58.585222 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd393b07-afa5-438e-bdb7-9c41b2dd8722 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:53:58.585497 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd393b07-afa5-438e-bdb7-9c41b2dd8722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:53:58.586326 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd393b07-afa5-438e-bdb7-9c41b2dd8722 None admin] RBAC: Adding query filter params (role.id=ed68c2195de147c0a30d9150b33e47cf, scope.project.id=eb86f5a389064e12bb38c500b225c9ac, user.id=c7f307747ddc4b2c8657ace75ce5af29) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:53:58.586641 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd393b07-afa5-438e-bdb7-9c41b2dd8722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:53:58.588142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd393b07-afa5-438e-bdb7-9c41b2dd8722 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:53:58.594067 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 640/1287] 10.222.0.88 () {56 vars in 1374 bytes} [Thu May 19 20:53:58 2022] GET /identity/v3/role_assignments?user.id=c7f307747ddc4b2c8657ace75ce5af29&scope.project.id=eb86f5a389064e12bb38c500b225c9ac&role.id=ed68c2195de147c0a30d9150b33e47cf => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:54:00.916367 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff2b5-928e-443e-ad9f-bb5ff2856f73 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:00.916951 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff2b5-928e-443e-ad9f-bb5ff2856f73 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:00.916951 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff2b5-928e-443e-ad9f-bb5ff2856f73 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:00.918330 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 648/1288] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:54:00.923675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37dacee-fba7-4f23-80cc-e7f2a5b2bf76 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:00.924181 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37dacee-fba7-4f23-80cc-e7f2a5b2bf76 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:00.924299 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37dacee-fba7-4f23-80cc-e7f2a5b2bf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:00.941711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-e37dacee-fba7-4f23-80cc-e7f2a5b2bf76 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:54:00.942909 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-e37dacee-fba7-4f23-80cc-e7f2a5b2bf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:54:00.972146 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:54:01.009161 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 641/1289] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:54:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:54:01.031530 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ff2b5-928e-443e-ad9f-bb5ff2856f73 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:54:01.135605 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ff2b5-928e-443e-ad9f-bb5ff2856f73 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:54:01.146679 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:54:01.170631 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92d34e-aed9-4b5d-81a6-204e1fc08d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:54:01.171367 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d34e-aed9-4b5d-81a6-204e1fc08d4e None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:01.171654 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d34e-aed9-4b5d-81a6-204e1fc08d4e None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:01.171932 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d34e-aed9-4b5d-81a6-204e1fc08d4e None admin] PATH_INFO: `/v3/services/placement` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:01.172624 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92d34e-aed9-4b5d-81a6-204e1fc08d4e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:54:01.174031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92d34e-aed9-4b5d-81a6-204e1fc08d4e None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-6a92d34e-aed9-4b5d-81a6-204e1fc08d4e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:54:01.186918 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:54:01.188492 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:54:01.189754 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 649/1290] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:54:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:54:01.194465 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37dacee-fba7-4f23-80cc-e7f2a5b2bf76 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:54:01.293114 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37dacee-fba7-4f23-80cc-e7f2a5b2bf76 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:54:01.327113 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5aadc-f626-40a6-8549-ad3f6a1af695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:54:01.327777 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5aadc-f626-40a6-8549-ad3f6a1af695 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:01.328059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5aadc-f626-40a6-8549-ad3f6a1af695 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:01.328351 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5aadc-f626-40a6-8549-ad3f6a1af695 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:01.329059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5aadc-f626-40a6-8549-ad3f6a1af695 None admin] RBAC: Adding query filter params (name=placement) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:54:01.329364 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5aadc-f626-40a6-8549-ad3f6a1af695 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:54:01.330717 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5aadc-f626-40a6-8549-ad3f6a1af695 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:54:01.336269 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 642/1291] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:54:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:54:01.342351 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92d34e-aed9-4b5d-81a6-204e1fc08d4e None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:54:01.445362 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92d34e-aed9-4b5d-81a6-204e1fc08d4e None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:54:01.478008 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fb406-dbb7-4340-9b70-24df25f2fa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:54:01.478778 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fb406-dbb7-4340-9b70-24df25f2fa1b None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:01.479061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fb406-dbb7-4340-9b70-24df25f2fa1b None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:01.479335 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fb406-dbb7-4340-9b70-24df25f2fa1b None admin] PATH_INFO: `/v3/services` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:01.480042 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fb406-dbb7-4340-9b70-24df25f2fa1b None admin] RBAC: Adding query filter params (type=placement) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:54:01.480366 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fb406-dbb7-4340-9b70-24df25f2fa1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:54:01.481765 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fb406-dbb7-4340-9b70-24df25f2fa1b None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:54:01.486153 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 650/1292] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:54:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:54:03.678793 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35741088-acfc-4f02-ae3d-e79ca752e5f6 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:03.679407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35741088-acfc-4f02-ae3d-e79ca752e5f6 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:03.679407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35741088-acfc-4f02-ae3d-e79ca752e5f6 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:03.680785 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 643/1293] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 20:54:03.686113 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012fe0e-bc7e-4456-814a-6c894de42f4b None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:03.686796 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012fe0e-bc7e-4456-814a-6c894de42f4b None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:03.686914 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012fe0e-bc7e-4456-814a-6c894de42f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:03.704120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-9012fe0e-bc7e-4456-814a-6c894de42f4b None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:54:03.705344 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-9012fe0e-bc7e-4456-814a-6c894de42f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:54:03.734756 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:54:03.772201 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 651/1294] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:54:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:54:03.794839 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35741088-acfc-4f02-ae3d-e79ca752e5f6 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:54:03.827420 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:54:03.898729 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35741088-acfc-4f02-ae3d-e79ca752e5f6 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:54:03.932409 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa237a4-d461-4f8d-a791-94b7744f1906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:54:03.933125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa237a4-d461-4f8d-a791-94b7744f1906 None admin] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:03.933438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa237a4-d461-4f8d-a791-94b7744f1906 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:03.933716 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa237a4-d461-4f8d-a791-94b7744f1906 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:03.934444 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa237a4-d461-4f8d-a791-94b7744f1906 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:54:03.935801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa237a4-d461-4f8d-a791-94b7744f1906 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:54:04.019854 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 644/1295] 10.222.0.88 () {60 vars in 1141 bytes} [Thu May 19 20:54:03 2022] POST /identity/v3/services => generated 246 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:54:06.238529 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b2cf-da69-457d-a569-e0844bba9496 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:06.239082 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b2cf-da69-457d-a569-e0844bba9496 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:06.239082 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20b2cf-da69-457d-a569-e0844bba9496 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:06.240597 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 652/1296] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:54:06.245908 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431585b-264c-4001-bd02-8ddf6d4cc7be None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:06.251802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431585b-264c-4001-bd02-8ddf6d4cc7be None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:06.251802 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431585b-264c-4001-bd02-8ddf6d4cc7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:06.264319 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-c431585b-264c-4001-bd02-8ddf6d4cc7be None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:54:06.265539 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-c431585b-264c-4001-bd02-8ddf6d4cc7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:54:06.338708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 645/1297] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:54:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:54:06.361451 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20b2cf-da69-457d-a569-e0844bba9496 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:54:06.464206 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20b2cf-da69-457d-a569-e0844bba9496 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:54:06.497775 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb736bc-ec80-4173-b9a2-509f2527ccc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:54:06.498507 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb736bc-ec80-4173-b9a2-509f2527ccc9 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:06.498809 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb736bc-ec80-4173-b9a2-509f2527ccc9 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:06.499074 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb736bc-ec80-4173-b9a2-509f2527ccc9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:06.499731 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb736bc-ec80-4173-b9a2-509f2527ccc9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:54:06.501333 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb736bc-ec80-4173-b9a2-509f2527ccc9 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-6eb736bc-ec80-4173-b9a2-509f2527ccc9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:54:06.513684 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:54:06.515053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:54:06.516085 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 653/1298] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:54:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:54:06.521185 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431585b-264c-4001-bd02-8ddf6d4cc7be None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:54:06.540041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:54:06.618654 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431585b-264c-4001-bd02-8ddf6d4cc7be None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:54:06.653368 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91474133-c5b0-42dd-9d84-f73037b39508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:54:06.654030 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91474133-c5b0-42dd-9d84-f73037b39508 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:06.654316 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91474133-c5b0-42dd-9d84-f73037b39508 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:06.654596 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91474133-c5b0-42dd-9d84-f73037b39508 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:06.655298 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91474133-c5b0-42dd-9d84-f73037b39508 None admin] RBAC: Adding query filter params (name=placement) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:54:06.655607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91474133-c5b0-42dd-9d84-f73037b39508 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:54:06.657005 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91474133-c5b0-42dd-9d84-f73037b39508 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:54:06.662214 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 646/1299] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:54:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:54:06.668384 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb736bc-ec80-4173-b9a2-509f2527ccc9 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:54:06.701419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:54:06.775664 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb736bc-ec80-4173-b9a2-509f2527ccc9 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:54:06.799541 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab45ac-cf8f-47b9-b46a-a4ebc170fc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:54:06.800188 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab45ac-cf8f-47b9-b46a-a4ebc170fc82 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:06.800506 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab45ac-cf8f-47b9-b46a-a4ebc170fc82 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:06.800784 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab45ac-cf8f-47b9-b46a-a4ebc170fc82 None admin] PATH_INFO: `/v3/endpoints` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:06.801574 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab45ac-cf8f-47b9-b46a-a4ebc170fc82 None admin] RBAC: Adding query filter params (interface=public, service_id=e2adfdc88dbd423ab970c16dbc3c6ae0, region_id=RegionOne) {{(pid=278915) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:54:06.801890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab45ac-cf8f-47b9-b46a-a4ebc170fc82 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:54:06.803817 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab45ac-cf8f-47b9-b46a-a4ebc170fc82 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:54:06.808782 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 654/1300] 10.222.0.88 () {56 vars in 1251 bytes} [Thu May 19 20:54:06 2022] GET /identity/v3/endpoints?service_id=e2adfdc88dbd423ab970c16dbc3c6ae0&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 19 20:54:06.814708 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91474133-c5b0-42dd-9d84-f73037b39508 None admin] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:54:06.910178 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:54:06.929028 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91474133-c5b0-42dd-9d84-f73037b39508 None admin] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:54:06.963336 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482af8fb-5c7c-43b2-8ce3-87ded3699917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:54:06.963999 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482af8fb-5c7c-43b2-8ce3-87ded3699917 None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:06.964288 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482af8fb-5c7c-43b2-8ce3-87ded3699917 None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:06.964601 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482af8fb-5c7c-43b2-8ce3-87ded3699917 None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:06.965303 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482af8fb-5c7c-43b2-8ce3-87ded3699917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:54:06.966662 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482af8fb-5c7c-43b2-8ce3-87ded3699917 None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:54:06.973956 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 647/1301] 10.222.0.88 () {56 vars in 1087 bytes} [Thu May 19 20:54:06 2022] GET /identity/v3/services => generated 2191 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 19 20:54:09.222895 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84033c20-217a-4b65-bf79-71c3b2a83312 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:09.223439 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84033c20-217a-4b65-bf79-71c3b2a83312 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:09.223439 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84033c20-217a-4b65-bf79-71c3b2a83312 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:09.224894 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 655/1302] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 20:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 20:54:09.230388 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ff195-e218-452c-8ee1-551957fcf620 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:09.230820 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ff195-e218-452c-8ee1-551957fcf620 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:09.230936 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ff195-e218-452c-8ee1-551957fcf620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:09.247548 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-b98ff195-e218-452c-8ee1-551957fcf620 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 20:54:09.248771 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-b98ff195-e218-452c-8ee1-551957fcf620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 20:54:09.315717 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 648/1303] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 20:54:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 19 20:54:09.339550 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84033c20-217a-4b65-bf79-71c3b2a83312 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:54:09.433659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:54:09.440455 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84033c20-217a-4b65-bf79-71c3b2a83312 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:54:09.474099 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24683-e129-4a57-89f2-b6ce7a58bb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:54:09.474928 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24683-e129-4a57-89f2-b6ce7a58bb33 None admin] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:09.475210 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24683-e129-4a57-89f2-b6ce7a58bb33 None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:09.475502 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da24683-e129-4a57-89f2-b6ce7a58bb33 None admin] PATH_INFO: `/v3/services/placement` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:09.476150 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da24683-e129-4a57-89f2-b6ce7a58bb33 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:54:09.477579 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da24683-e129-4a57-89f2-b6ce7a58bb33 None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application [None req-3da24683-e129-4a57-89f2-b6ce7a58bb33 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application Traceback (most recent call last): May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 19 20:54:09.489297 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._get_service(service_id) May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 19 20:54:09.490852 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.get_or_create( May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application with Lock( May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self._enter() May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application return self.creator() May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application created_value = creator( May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: ERROR keystone.server.flask.application May 19 20:54:09.491971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 656/1304] 10.222.0.88 () {56 vars in 1117 bytes} [Thu May 19 20:54:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 19 20:54:09.497006 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ff195-e218-452c-8ee1-551957fcf620 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:54:09.573689 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:54:09.595685 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ff195-e218-452c-8ee1-551957fcf620 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:54:09.630106 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a260171-06aa-48e3-af3f-1e6c6cdafc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:54:09.631340 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a260171-06aa-48e3-af3f-1e6c6cdafc9d None admin] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:09.631623 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a260171-06aa-48e3-af3f-1e6c6cdafc9d None admin] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:09.631899 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a260171-06aa-48e3-af3f-1e6c6cdafc9d None admin] PATH_INFO: `/v3/services` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:09.632624 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a260171-06aa-48e3-af3f-1e6c6cdafc9d None admin] RBAC: Adding query filter params (name=placement) {{(pid=278914) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 19 20:54:09.632929 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a260171-06aa-48e3-af3f-1e6c6cdafc9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:54:09.634670 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a260171-06aa-48e3-af3f-1e6c6cdafc9d None admin] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:54:09.639800 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 649/1305] 10.222.0.88 () {56 vars in 1116 bytes} [Thu May 19 20:54:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 19 20:54:09.645663 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24683-e129-4a57-89f2-b6ce7a58bb33 None admin] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 20:54:09.746972 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da24683-e129-4a57-89f2-b6ce7a58bb33 None admin] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 20:54:09.780640 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bdadd-ffcf-4e30-a482-2870fd42b74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4f6c30de67e643ecb5582d374f07cb7c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 20:54:09.781295 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bdadd-ffcf-4e30-a482-2870fd42b74c None admin] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 20:54:09.781617 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bdadd-ffcf-4e30-a482-2870fd42b74c None admin] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 20:54:09.781900 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bdadd-ffcf-4e30-a482-2870fd42b74c None admin] PATH_INFO: `/v3/endpoints` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 20:54:09.782635 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bdadd-ffcf-4e30-a482-2870fd42b74c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 20:54:09.784031 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bdadd-ffcf-4e30-a482-2870fd42b74c None admin] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 20:54:09.877438 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 657/1306] 10.222.0.88 () {60 vars in 1144 bytes} [Thu May 19 20:54:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 19 20:54:19.657459 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=30 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 20:54:19.872631 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:33.472818 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8a7d2-6865-4eb5-a1ee-80dbc4090665 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:33.473429 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8a7d2-6865-4eb5-a1ee-80dbc4090665 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:33.473429 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8a7d2-6865-4eb5-a1ee-80dbc4090665 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:33.474796 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 650/1307] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:01:33.481475 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310497b4-9eca-44d0-9846-c33c0ad29c0f None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:33.482064 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310497b4-9eca-44d0-9846-c33c0ad29c0f None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:33.482064 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310497b4-9eca-44d0-9846-c33c0ad29c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:33.598426 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-310497b4-9eca-44d0-9846-c33c0ad29c0f None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:01:33.599925 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-310497b4-9eca-44d0-9846-c33c0ad29c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:33.725053 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 658/1308] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:01:33.776247 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c97992-1215-4b92-a4cd-42f707738c1a None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:33.777072 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c97992-1215-4b92-a4cd-42f707738c1a None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:33.777546 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c97992-1215-4b92-a4cd-42f707738c1a None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:33.779438 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 651/1309] 10.222.0.88 () {54 vars in 964 bytes} [Thu May 19 21:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:01:33.798828 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411fdc6-fa9c-40e0-a46c-ae3922e9b85e None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:33.799432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411fdc6-fa9c-40e0-a46c-ae3922e9b85e None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:33.799905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411fdc6-fa9c-40e0-a46c-ae3922e9b85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:33.966831 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-8411fdc6-fa9c-40e0-a46c-ae3922e9b85e None None] MFA Rules not processed for user `4b3f694758fa483ca497fbea18061cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:01:33.968145 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-8411fdc6-fa9c-40e0-a46c-ae3922e9b85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:34.020141 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:34.121217 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 659/1310] 10.222.0.88 () {58 vars in 1061 bytes} [Thu May 19 21:01:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:01:34.130899 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c97992-1215-4b92-a4cd-42f707738c1a None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:01:34.135079 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-49c97992-1215-4b92-a4cd-42f707738c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:34.467109 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c97992-1215-4b92-a4cd-42f707738c1a None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:01:34.645730 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd58563-000d-4d0b-8501-b8d50b508959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:01:34.646390 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd58563-000d-4d0b-8501-b8d50b508959 service neutron] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:34.650250 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd58563-000d-4d0b-8501-b8d50b508959 service neutron] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:34.651371 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd58563-000d-4d0b-8501-b8d50b508959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:34.657160 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-bcd58563-000d-4d0b-8501-b8d50b508959 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:34.752650 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:34.875660 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd58563-000d-4d0b-8501-b8d50b508959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:01:34.877181 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd58563-000d-4d0b-8501-b8d50b508959 service neutron] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:01:34.945986 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 652/1311] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 816 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:01:37.618739 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b57a9-2346-47dd-b759-8c2fc1ccf11b None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:37.619380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b57a9-2346-47dd-b759-8c2fc1ccf11b None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:37.619380 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b57a9-2346-47dd-b759-8c2fc1ccf11b None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:37.624740 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 660/1312] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:01:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:01:37.626161 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0a557-57a7-47ab-abdf-200bd1d40d1b None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:37.626636 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0a557-57a7-47ab-abdf-200bd1d40d1b None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:37.626737 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0a557-57a7-47ab-abdf-200bd1d40d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:37.712958 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-a5c0a557-57a7-47ab-abdf-200bd1d40d1b None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:01:37.714199 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-a5c0a557-57a7-47ab-abdf-200bd1d40d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:37.844675 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 653/1313] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:01:37.874863 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1012b2-430b-4854-988a-bc8481b6f124 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:37.875184 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1012b2-430b-4854-988a-bc8481b6f124 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:37.875732 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1012b2-430b-4854-988a-bc8481b6f124 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:37.877541 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 661/1314] 10.222.0.88 () {54 vars in 964 bytes} [Thu May 19 21:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:01:37.884630 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8921d-c124-4098-978e-d4a1bcdc1b43 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:37.896134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8921d-c124-4098-978e-d4a1bcdc1b43 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:37.896134 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8921d-c124-4098-978e-d4a1bcdc1b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:37.903394 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:37.989248 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-dde8921d-c124-4098-978e-d4a1bcdc1b43 None None] MFA Rules not processed for user `4b3f694758fa483ca497fbea18061cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:01:37.991471 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-dde8921d-c124-4098-978e-d4a1bcdc1b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:38.087514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 654/1315] 10.222.0.88 () {58 vars in 1061 bytes} [Thu May 19 21:01:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:01:38.096444 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1012b2-430b-4854-988a-bc8481b6f124 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:01:38.098035 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-ee1012b2-430b-4854-988a-bc8481b6f124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:38.259403 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1012b2-430b-4854-988a-bc8481b6f124 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:01:38.345028 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:38.347782 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a774239-262e-44d1-bb11-304893a7395b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:01:38.348408 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a774239-262e-44d1-bb11-304893a7395b service neutron] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:38.348695 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a774239-262e-44d1-bb11-304893a7395b service neutron] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:38.348969 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a774239-262e-44d1-bb11-304893a7395b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:38.350657 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-6a774239-262e-44d1-bb11-304893a7395b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:38.509399 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a774239-262e-44d1-bb11-304893a7395b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:01:38.510905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a774239-262e-44d1-bb11-304893a7395b service neutron] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:01:38.534198 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 662/1316] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:01:41.551416 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32e467-69a2-4c36-a719-30144a3f3fc1 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:41.552041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32e467-69a2-4c36-a719-30144a3f3fc1 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:41.552041 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32e467-69a2-4c36-a719-30144a3f3fc1 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:41.554025 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 655/1317] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:01:41.559230 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3590c-be0a-4a39-abee-a1cc0585b3c9 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:41.559659 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3590c-be0a-4a39-abee-a1cc0585b3c9 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:41.559791 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3590c-be0a-4a39-abee-a1cc0585b3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:41.624246 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-1cf3590c-be0a-4a39-abee-a1cc0585b3c9 None None] MFA Rules not processed for user `136b6f5ff4004c75b8b76ea91e0ec8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:01:41.625486 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-1cf3590c-be0a-4a39-abee-a1cc0585b3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:41.677549 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:41.721688 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 663/1318] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:01:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:01:41.761607 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32e467-69a2-4c36-a719-30144a3f3fc1 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:01:41.763161 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-0c32e467-69a2-4c36-a719-30144a3f3fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:41.867416 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c32e467-69a2-4c36-a719-30144a3f3fc1 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:01:41.870469 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:41.942989 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbec290-ef34-4f55-85f7-1837ff8c520c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:01:41.943668 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbec290-ef34-4f55-85f7-1837ff8c520c service neutron] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:41.943956 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbec290-ef34-4f55-85f7-1837ff8c520c service neutron] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:41.944268 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbec290-ef34-4f55-85f7-1837ff8c520c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:41.945991 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-ecbec290-ef34-4f55-85f7-1837ff8c520c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:42.053896 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbec290-ef34-4f55-85f7-1837ff8c520c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:01:42.055318 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbec290-ef34-4f55-85f7-1837ff8c520c service neutron] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:01:42.073586 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 656/1319] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:01:41 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:01:45.073958 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1422176-57ee-4a31-b539-4608b0504259 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:45.074516 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1422176-57ee-4a31-b539-4608b0504259 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:45.074650 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1422176-57ee-4a31-b539-4608b0504259 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:45.075959 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 664/1320] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:01:45.081430 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21f73d-8000-4141-95f7-5b7b29f7f31b None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:45.081876 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21f73d-8000-4141-95f7-5b7b29f7f31b None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:45.082009 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21f73d-8000-4141-95f7-5b7b29f7f31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:45.122892 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:45.153068 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-2f21f73d-8000-4141-95f7-5b7b29f7f31b None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:01:45.154314 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-2f21f73d-8000-4141-95f7-5b7b29f7f31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:45.250385 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 657/1321] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:01:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:01:45.289562 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1422176-57ee-4a31-b539-4608b0504259 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:01:45.291120 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-d1422176-57ee-4a31-b539-4608b0504259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:45.394175 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1422176-57ee-4a31-b539-4608b0504259 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:01:45.468980 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293a6a3-e787-4e4e-89df-53f8e632e92c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:01:45.469617 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293a6a3-e787-4e4e-89df-53f8e632e92c service neutron] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:45.469907 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293a6a3-e787-4e4e-89df-53f8e632e92c service neutron] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:45.470191 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293a6a3-e787-4e4e-89df-53f8e632e92c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:45.471877 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-0293a6a3-e787-4e4e-89df-53f8e632e92c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:45.479884 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:45.581146 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293a6a3-e787-4e4e-89df-53f8e632e92c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:01:45.582596 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293a6a3-e787-4e4e-89df-53f8e632e92c service neutron] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:01:45.600575 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 665/1322] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:01:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:01:48.812825 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3e6ba-60e6-47d4-9877-192ff44cab83 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:48.813435 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3e6ba-60e6-47d4-9877-192ff44cab83 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:48.813435 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3e6ba-60e6-47d4-9877-192ff44cab83 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:48.815035 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 658/1323] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:01:48.820498 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215387ab-86fa-4ad1-8d7a-9f83ab410ddc None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:48.820945 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215387ab-86fa-4ad1-8d7a-9f83ab410ddc None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:48.821059 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215387ab-86fa-4ad1-8d7a-9f83ab410ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:48.888151 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-215387ab-86fa-4ad1-8d7a-9f83ab410ddc None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:01:48.889454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-215387ab-86fa-4ad1-8d7a-9f83ab410ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:48.951802 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:48.991905 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 666/1324] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:01:49.028221 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3e6ba-60e6-47d4-9877-192ff44cab83 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:01:49.030000 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-e6e3e6ba-60e6-47d4-9877-192ff44cab83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:49.134672 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3e6ba-60e6-47d4-9877-192ff44cab83 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:01:49.147577 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:49.210746 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52971666-b6ea-4506-9d56-40fa2869ba45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:01:49.211407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52971666-b6ea-4506-9d56-40fa2869ba45 service neutron] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:49.211693 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52971666-b6ea-4506-9d56-40fa2869ba45 service neutron] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:49.211970 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52971666-b6ea-4506-9d56-40fa2869ba45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:49.213689 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-52971666-b6ea-4506-9d56-40fa2869ba45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:49.325246 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52971666-b6ea-4506-9d56-40fa2869ba45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:01:49.326686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52971666-b6ea-4506-9d56-40fa2869ba45 service neutron] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:01:49.345980 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 659/1325] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:01:51.652983 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db48b45-646e-44e9-889d-776905bd4cce None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:51.653493 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db48b45-646e-44e9-889d-776905bd4cce None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:51.653611 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db48b45-646e-44e9-889d-776905bd4cce None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:51.654952 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 667/1326] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:01:51.660548 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb2f36-9259-4a68-a590-1d7d36800e5b None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:51.661012 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb2f36-9259-4a68-a590-1d7d36800e5b None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:51.661124 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb2f36-9259-4a68-a590-1d7d36800e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:51.707812 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:51.727147 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-7acb2f36-9259-4a68-a590-1d7d36800e5b None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:01:51.729063 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-7acb2f36-9259-4a68-a590-1d7d36800e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:51.967877 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 660/1327] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:01:52.003510 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db48b45-646e-44e9-889d-776905bd4cce None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:01:52.005192 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-5db48b45-646e-44e9-889d-776905bd4cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:52.123124 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db48b45-646e-44e9-889d-776905bd4cce None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:01:52.221910 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150dfece-8bbb-4303-840f-b6f2bb5065ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:01:52.222965 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dfece-8bbb-4303-840f-b6f2bb5065ce service neutron] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:52.223567 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dfece-8bbb-4303-840f-b6f2bb5065ce service neutron] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:52.224061 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150dfece-8bbb-4303-840f-b6f2bb5065ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:52.227092 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-150dfece-8bbb-4303-840f-b6f2bb5065ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:52.247520 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:52.349077 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150dfece-8bbb-4303-840f-b6f2bb5065ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:01:52.351135 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150dfece-8bbb-4303-840f-b6f2bb5065ce service neutron] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:01:52.369890 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 668/1328] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:01:54.662537 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111fcd4-70fd-4698-b1fc-67a678e76dd0 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:54.663125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111fcd4-70fd-4698-b1fc-67a678e76dd0 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:54.663125 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111fcd4-70fd-4698-b1fc-67a678e76dd0 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:54.664502 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 661/1329] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:01:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:01:54.669764 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4793ed-54cd-4992-ac91-6811320fc5c5 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:54.670163 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4793ed-54cd-4992-ac91-6811320fc5c5 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:54.670321 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4793ed-54cd-4992-ac91-6811320fc5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:54.735986 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-4e4793ed-54cd-4992-ac91-6811320fc5c5 None None] MFA Rules not processed for user `136b6f5ff4004c75b8b76ea91e0ec8d4`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:01:54.737253 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-4e4793ed-54cd-4992-ac91-6811320fc5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:54.801406 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:54.834324 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 669/1330] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:01:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:01:54.872490 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111fcd4-70fd-4698-b1fc-67a678e76dd0 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:01:54.874074 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-9111fcd4-70fd-4698-b1fc-67a678e76dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:54.983197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111fcd4-70fd-4698-b1fc-67a678e76dd0 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:01:55.007247 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:55.064990 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8721c-a792-4100-98c6-2b46b96ca068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:01:55.065653 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8721c-a792-4100-98c6-2b46b96ca068 service neutron] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:55.065934 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8721c-a792-4100-98c6-2b46b96ca068 service neutron] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:55.066209 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8721c-a792-4100-98c6-2b46b96ca068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:55.067872 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-56a8721c-a792-4100-98c6-2b46b96ca068 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:55.178245 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8721c-a792-4100-98c6-2b46b96ca068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:01:55.179775 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8721c-a792-4100-98c6-2b46b96ca068 service neutron] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:01:55.199913 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 662/1331] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:01:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:01:58.028650 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d3c9c-f389-495d-9d49-969381553061 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:58.029183 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d3c9c-f389-495d-9d49-969381553061 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:58.029183 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03d3c9c-f389-495d-9d49-969381553061 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:58.031058 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 670/1332] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:01:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:01:58.036730 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c30c43-1a9f-4415-bad1-6abbc5d6afd0 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:58.037193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c30c43-1a9f-4415-bad1-6abbc5d6afd0 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:58.037310 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c30c43-1a9f-4415-bad1-6abbc5d6afd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:58.100262 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:58.104857 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-89c30c43-1a9f-4415-bad1-6abbc5d6afd0 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:01:58.107396 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-89c30c43-1a9f-4415-bad1-6abbc5d6afd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:58.203767 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 663/1333] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:01:58.240100 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d3c9c-f389-495d-9d49-969381553061 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:01:58.241696 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-c03d3c9c-f389-495d-9d49-969381553061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:58.347624 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03d3c9c-f389-495d-9d49-969381553061 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:01:58.422206 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85c1a6-a466-4a5b-9bf3-fc1432048631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:01:58.422870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85c1a6-a466-4a5b-9bf3-fc1432048631 service neutron] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:01:58.423190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85c1a6-a466-4a5b-9bf3-fc1432048631 service neutron] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:01:58.423477 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85c1a6-a466-4a5b-9bf3-fc1432048631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:01:58.425200 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-0f85c1a6-a466-4a5b-9bf3-fc1432048631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:01:58.452114 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:01:58.533837 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85c1a6-a466-4a5b-9bf3-fc1432048631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:01:58.535296 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85c1a6-a466-4a5b-9bf3-fc1432048631 service neutron] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:01:58.553554 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 671/1334] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:02:00.919317 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed8656-11c2-4df5-a421-c596a25298e9 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:00.919902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed8656-11c2-4df5-a421-c596a25298e9 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:00.919902 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed8656-11c2-4df5-a421-c596a25298e9 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:00.921287 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 664/1335] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:00.926737 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba14b6-0f60-47fc-a968-715a0d6b6187 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:00.927184 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba14b6-0f60-47fc-a968-715a0d6b6187 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:00.927310 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba14b6-0f60-47fc-a968-715a0d6b6187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:00.996645 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-74ba14b6-0f60-47fc-a968-715a0d6b6187 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:00.997870 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-74ba14b6-0f60-47fc-a968-715a0d6b6187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:01.077724 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=100 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:02:01.101072 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 672/1336] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:01.141065 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed8656-11c2-4df5-a421-c596a25298e9 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:02:01.142604 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-d7ed8656-11c2-4df5-a421-c596a25298e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:01.245021 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed8656-11c2-4df5-a421-c596a25298e9 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:02:01.273017 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:02:01.319517 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace22fa0-8410-437d-a7bb-d9f047fc3fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:02:01.320237 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22fa0-8410-437d-a7bb-d9f047fc3fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:01.320561 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22fa0-8410-437d-a7bb-d9f047fc3fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:01.321478 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22fa0-8410-437d-a7bb-d9f047fc3fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:01.323146 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-ace22fa0-8410-437d-a7bb-d9f047fc3fb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:01.429711 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace22fa0-8410-437d-a7bb-d9f047fc3fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:02:01.431685 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace22fa0-8410-437d-a7bb-d9f047fc3fb1 service neutron] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:02:01.450797 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 665/1337] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:02:04.292190 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa1310-394a-4a24-aa10-0b5ad93cf66a None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:04.292808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa1310-394a-4a24-aa10-0b5ad93cf66a None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:04.292808 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa1310-394a-4a24-aa10-0b5ad93cf66a None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:04.294174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 673/1338] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:04.299472 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f141-b20f-4827-976f-1437a9a96442 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:04.299915 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f141-b20f-4827-976f-1437a9a96442 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:04.300048 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f141-b20f-4827-976f-1437a9a96442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:04.375847 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-27e6f141-b20f-4827-976f-1437a9a96442 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:04.377111 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-27e6f141-b20f-4827-976f-1437a9a96442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:04.387306 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:02:04.492059 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 666/1339] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:02:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:04.529330 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa1310-394a-4a24-aa10-0b5ad93cf66a None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:02:04.531577 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-59aa1310-394a-4a24-aa10-0b5ad93cf66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:04.628540 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa1310-394a-4a24-aa10-0b5ad93cf66a None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:02:04.632007 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb151e7-0486-483b-af85-bb7ef066bf79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:02:04.632686 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb151e7-0486-483b-af85-bb7ef066bf79 service neutron] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:04.632969 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb151e7-0486-483b-af85-bb7ef066bf79 service neutron] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:04.633861 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb151e7-0486-483b-af85-bb7ef066bf79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:04.642958 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb151e7-0486-483b-af85-bb7ef066bf79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:02:04.644432 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb151e7-0486-483b-af85-bb7ef066bf79 service neutron] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:02:04.654011 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 674/1340] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:02:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:02:09.278891 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b38d7-0ef1-4968-b7fc-0b0633bb4592 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:09.279735 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b38d7-0ef1-4968-b7fc-0b0633bb4592 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:09.279735 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b38d7-0ef1-4968-b7fc-0b0633bb4592 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:09.284401 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 667/1341] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:02:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:09.289165 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11374d95-49dc-40c5-b411-c0d0758b1b86 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:09.290470 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11374d95-49dc-40c5-b411-c0d0758b1b86 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:09.290470 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11374d95-49dc-40c5-b411-c0d0758b1b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:09.325673 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-11374d95-49dc-40c5-b411-c0d0758b1b86 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:09.326899 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-11374d95-49dc-40c5-b411-c0d0758b1b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:09.419009 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 675/1342] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:09.459548 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b38d7-0ef1-4968-b7fc-0b0633bb4592 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:02:09.461303 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-8e4b38d7-0ef1-4968-b7fc-0b0633bb4592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:09.475584 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b38d7-0ef1-4968-b7fc-0b0633bb4592 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:02:09.479815 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c3cd6-17cc-4d37-b172-b435f506068b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:02:09.480501 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c3cd6-17cc-4d37-b172-b435f506068b service neutron] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:09.480787 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c3cd6-17cc-4d37-b172-b435f506068b service neutron] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:09.481066 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c3cd6-17cc-4d37-b172-b435f506068b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:09.490686 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c3cd6-17cc-4d37-b172-b435f506068b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:02:09.492110 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c3cd6-17cc-4d37-b172-b435f506068b service neutron] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:02:09.502411 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 668/1343] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 19 21:02:12.974917 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb070a-6200-4910-adf5-fbcc84c0d11f None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:12.975489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb070a-6200-4910-adf5-fbcc84c0d11f None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:12.975489 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb070a-6200-4910-adf5-fbcc84c0d11f None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:12.977480 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 676/1344] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:12.982900 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b29d0c-3527-45a5-b0b0-30e5d59a658a None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:12.983407 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b29d0c-3527-45a5-b0b0-30e5d59a658a None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:12.983514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b29d0c-3527-45a5-b0b0-30e5d59a658a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:13.007161 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-02b29d0c-3527-45a5-b0b0-30e5d59a658a None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:13.008423 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-02b29d0c-3527-45a5-b0b0-30e5d59a658a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:13.018600 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 669/1345] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:02:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:13.056198 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb070a-6200-4910-adf5-fbcc84c0d11f None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:02:13.067535 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb070a-6200-4910-adf5-fbcc84c0d11f None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:02:13.071128 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c2067-6641-4318-b7a3-6d1b991a7861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:02:13.071781 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c2067-6641-4318-b7a3-6d1b991a7861 service neutron] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:13.072088 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c2067-6641-4318-b7a3-6d1b991a7861 service neutron] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:13.072379 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c2067-6641-4318-b7a3-6d1b991a7861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:13.081885 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c2067-6641-4318-b7a3-6d1b991a7861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:02:13.083352 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c2067-6641-4318-b7a3-6d1b991a7861 service neutron] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:02:13.093181 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 677/1346] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:02:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:02:17.931634 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993303c0-1dbd-4d1a-87c4-92d229e5cacb None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:17.932046 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993303c0-1dbd-4d1a-87c4-92d229e5cacb None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:17.932240 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993303c0-1dbd-4d1a-87c4-92d229e5cacb None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:17.933647 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 670/1347] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:17.939809 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7412f3-d9ce-4ec8-ab64-b2c65995d5f5 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:17.940397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7412f3-d9ce-4ec8-ab64-b2c65995d5f5 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:17.940397 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7412f3-d9ce-4ec8-ab64-b2c65995d5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:17.960308 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-1b7412f3-d9ce-4ec8-ab64-b2c65995d5f5 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:17.962126 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-1b7412f3-d9ce-4ec8-ab64-b2c65995d5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:17.970842 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 678/1348] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:18.008195 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993303c0-1dbd-4d1a-87c4-92d229e5cacb None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:02:18.019154 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993303c0-1dbd-4d1a-87c4-92d229e5cacb None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:02:18.023016 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566f058-ef74-4055-a501-e26d7ce3c341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b3f694758fa483ca497fbea18061cb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:02:18.024216 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566f058-ef74-4055-a501-e26d7ce3c341 service neutron] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:18.024550 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566f058-ef74-4055-a501-e26d7ce3c341 service neutron] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:18.024834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566f058-ef74-4055-a501-e26d7ce3c341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:18.034304 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566f058-ef74-4055-a501-e26d7ce3c341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:02:18.035818 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566f058-ef74-4055-a501-e26d7ce3c341 service neutron] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:02:18.046834 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 671/1349] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:02:23.608382 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6bf0f-04d3-4eda-9286-6d6d596e6c3c None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:23.608885 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6bf0f-04d3-4eda-9286-6d6d596e6c3c None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:23.608885 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f6bf0f-04d3-4eda-9286-6d6d596e6c3c None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:23.610444 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 679/1350] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 21:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:23.615193 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e051aa-e699-4394-bb49-628b52d9db20 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:23.616001 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e051aa-e699-4394-bb49-628b52d9db20 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:23.616494 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e051aa-e699-4394-bb49-628b52d9db20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:23.703383 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-d1e051aa-e699-4394-bb49-628b52d9db20 None None] MFA Rules not processed for user `c7f307747ddc4b2c8657ace75ce5af29`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:23.704878 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-d1e051aa-e699-4394-bb49-628b52d9db20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:23.776348 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=100 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:02:23.786626 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 672/1351] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 21:02:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:02:27.954621 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=98 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:02:28.489174 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19164de1-75c9-4a13-b635-a71293364758 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:28.489703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19164de1-75c9-4a13-b635-a71293364758 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:28.490206 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19164de1-75c9-4a13-b635-a71293364758 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:28.492773 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 680/1352] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 21:02:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:28.498226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34ee3b-f342-4786-bd1c-316f042d924f None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:28.499197 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34ee3b-f342-4786-bd1c-316f042d924f None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:28.499663 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34ee3b-f342-4786-bd1c-316f042d924f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:28.521182 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-db34ee3b-f342-4786-bd1c-316f042d924f None None] MFA Rules not processed for user `c7f307747ddc4b2c8657ace75ce5af29`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:28.522695 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-db34ee3b-f342-4786-bd1c-316f042d924f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:28.532396 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 673/1353] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 21:02:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:30.060387 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e8c6e-f742-4c7f-8c81-3813401cf612 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:30.060914 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e8c6e-f742-4c7f-8c81-3813401cf612 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:30.061047 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e8c6e-f742-4c7f-8c81-3813401cf612 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:30.062346 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 681/1354] 10.222.0.88 () {54 vars in 921 bytes} [Thu May 19 21:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:30.068023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990b3ca-de51-4484-a52d-756762a7bd66 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:30.068549 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990b3ca-de51-4484-a52d-756762a7bd66 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:30.068654 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990b3ca-de51-4484-a52d-756762a7bd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:30.087875 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-1990b3ca-de51-4484-a52d-756762a7bd66 None None] MFA Rules not processed for user `c7f307747ddc4b2c8657ace75ce5af29`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:30.089128 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-1990b3ca-de51-4484-a52d-756762a7bd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:30.098395 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 674/1355] 10.222.0.88 () {58 vars in 1018 bytes} [Thu May 19 21:02:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:34.105209 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0929fa-e20a-446a-aec1-faf0d1020a10 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:34.105723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0929fa-e20a-446a-aec1-faf0d1020a10 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:34.105723 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0929fa-e20a-446a-aec1-faf0d1020a10 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:34.108026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 682/1356] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:02:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:34.115163 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd5e05-00c9-4ba3-a757-faae403486d9 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:34.115731 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd5e05-00c9-4ba3-a757-faae403486d9 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:34.115731 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd5e05-00c9-4ba3-a757-faae403486d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:34.142804 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-f1bd5e05-00c9-4ba3-a757-faae403486d9 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:34.144605 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-f1bd5e05-00c9-4ba3-a757-faae403486d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:34.163518 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 675/1357] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:34.225419 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c7fd2-4f2d-4357-9890-4adde69b3812 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:34.226119 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c7fd2-4f2d-4357-9890-4adde69b3812 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:34.226683 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c7fd2-4f2d-4357-9890-4adde69b3812 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:34.229026 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 683/1358] 10.222.0.88 () {54 vars in 961 bytes} [Thu May 19 21:02:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:34.237558 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af234fd-0484-424e-a721-029a4e9da9f9 None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:34.238204 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af234fd-0484-424e-a721-029a4e9da9f9 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:34.238735 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af234fd-0484-424e-a721-029a4e9da9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:34.302501 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-6af234fd-0484-424e-a721-029a4e9da9f9 None None] MFA Rules not processed for user `77427e1028d349b4a740efe0a985e73b`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:34.304873 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-6af234fd-0484-424e-a721-029a4e9da9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:34.407801 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 676/1359] 10.222.0.88 () {58 vars in 1058 bytes} [Thu May 19 21:02:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:02:34.415703 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c7fd2-4f2d-4357-9890-4adde69b3812 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:02:34.434454 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c7fd2-4f2d-4357-9890-4adde69b3812 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:02:34.442218 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae438137-b7f7-4240-8937-4e5b0b28c68b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77427e1028d349b4a740efe0a985e73b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:02:34.443999 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae438137-b7f7-4240-8937-4e5b0b28c68b service nova] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:34.445433 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae438137-b7f7-4240-8937-4e5b0b28c68b service nova] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:34.445959 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae438137-b7f7-4240-8937-4e5b0b28c68b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:34.456632 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae438137-b7f7-4240-8937-4e5b0b28c68b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:02:34.458713 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae438137-b7f7-4240-8937-4e5b0b28c68b service nova] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:02:34.471147 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 684/1360] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:02:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:02:36.931252 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b275d-2a03-414d-9bae-bca048e2ae4f None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:36.931741 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b275d-2a03-414d-9bae-bca048e2ae4f None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:36.931835 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b275d-2a03-414d-9bae-bca048e2ae4f None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:36.933253 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 677/1361] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:36.939311 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86227a6-b7bb-44cb-8d56-eb367da36416 None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:36.939787 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86227a6-b7bb-44cb-8d56-eb367da36416 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:36.939996 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86227a6-b7bb-44cb-8d56-eb367da36416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:36.960785 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-c86227a6-b7bb-44cb-8d56-eb367da36416 None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:36.962131 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-c86227a6-b7bb-44cb-8d56-eb367da36416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:36.970347 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 685/1362] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:37.016142 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b275d-2a03-414d-9bae-bca048e2ae4f None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:02:37.031966 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b275d-2a03-414d-9bae-bca048e2ae4f None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:02:37.037756 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3d777-a83d-41ea-8cdc-664755c54c6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77427e1028d349b4a740efe0a985e73b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:02:37.040310 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d777-a83d-41ea-8cdc-664755c54c6c service nova] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:37.040310 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d777-a83d-41ea-8cdc-664755c54c6c service nova] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:37.040310 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3d777-a83d-41ea-8cdc-664755c54c6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:37.048003 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3d777-a83d-41ea-8cdc-664755c54c6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:02:37.049665 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3d777-a83d-41ea-8cdc-664755c54c6c service nova] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:02:37.061582 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 678/1363] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:02:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:02:39.811018 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750c78c-a417-458b-9e8e-e5b569c7907e None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:39.811619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750c78c-a417-458b-9e8e-e5b569c7907e None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:39.811619 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750c78c-a417-458b-9e8e-e5b569c7907e None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:39.813020 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 686/1364] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:39.818840 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c87ff6-cfac-46f4-a195-79bd0fa797de None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:39.820269 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c87ff6-cfac-46f4-a195-79bd0fa797de None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:39.820769 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c87ff6-cfac-46f4-a195-79bd0fa797de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:39.842097 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-79c87ff6-cfac-46f4-a195-79bd0fa797de None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:39.843612 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-79c87ff6-cfac-46f4-a195-79bd0fa797de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:39.852040 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 679/1365] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 19 21:02:39.899830 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36020f-cf4b-4369-94f9-3cd19d5b1c84 None None] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:39.900529 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36020f-cf4b-4369-94f9-3cd19d5b1c84 None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:39.901043 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36020f-cf4b-4369-94f9-3cd19d5b1c84 None None] PATH_INFO: `/` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:39.902872 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 687/1366] 10.222.0.88 () {54 vars in 961 bytes} [Thu May 19 21:02:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:39.908514 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34f78e-49a2-4d84-a1ca-6574e8862ceb None None] REQUEST_METHOD: `POST` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:39.908808 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34f78e-49a2-4d84-a1ca-6574e8862ceb None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:39.909090 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34f78e-49a2-4d84-a1ca-6574e8862ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:39.928066 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.auth.core [None req-3d34f78e-49a2-4d84-a1ca-6574e8862ceb None None] MFA Rules not processed for user `77427e1028d349b4a740efe0a985e73b`. Rule list: `[]` (Enabled: `True`). {{(pid=278914) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:39.929458 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.fernet_utils [None req-3d34f78e-49a2-4d84-a1ca-6574e8862ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278914) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:39.939271 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 680/1367] 10.222.0.88 () {58 vars in 1058 bytes} [Thu May 19 21:02:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:39.948204 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36020f-cf4b-4369-94f9-3cd19d5b1c84 None None] Authenticating user token {{(pid=278915) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:02:39.963753 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36020f-cf4b-4369-94f9-3cd19d5b1c84 None None] Validating token access rules against request {{(pid=278915) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:02:39.969512 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955df97e-d96f-4d2e-af93-e9afc1581cc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77427e1028d349b4a740efe0a985e73b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278915) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:02:39.970247 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955df97e-d96f-4d2e-af93-e9afc1581cc6 service nova] REQUEST_METHOD: `GET` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:39.970608 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955df97e-d96f-4d2e-af93-e9afc1581cc6 service nova] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:39.971075 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955df97e-d96f-4d2e-af93-e9afc1581cc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:39.980913 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955df97e-d96f-4d2e-af93-e9afc1581cc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:02:39.982617 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955df97e-d96f-4d2e-af93-e9afc1581cc6 service nova] RBAC: Authorization granted {{(pid=278915) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:02:39.993558 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 688/1368] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:02:40.837226 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f407d7-1423-46c6-bc95-15ff726bde9f None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:40.838349 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f407d7-1423-46c6-bc95-15ff726bde9f None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:40.838666 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f407d7-1423-46c6-bc95-15ff726bde9f None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:40.840552 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 681/1369] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:02:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 19 21:02:40.847142 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e7a82-c652-4018-a0a6-2756f96c796b None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:40.847730 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e7a82-c652-4018-a0a6-2756f96c796b None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:40.847730 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e7a82-c652-4018-a0a6-2756f96c796b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:40.877571 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-8e0e7a82-c652-4018-a0a6-2756f96c796b None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:40.878946 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-8e0e7a82-c652-4018-a0a6-2756f96c796b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:40.895908 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 689/1370] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:42.594023 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005552d-0e50-46f2-8c68-76126dc1ec73 None None] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:42.594623 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005552d-0e50-46f2-8c68-76126dc1ec73 None None] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:42.594623 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c005552d-0e50-46f2-8c68-76126dc1ec73 None None] PATH_INFO: `/` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:42.595963 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 682/1371] 10.222.0.88 () {54 vars in 926 bytes} [Thu May 19 21:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 19 21:02:42.601426 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebacd-1aa0-4ea4-a42f-f0def760ddfe None None] REQUEST_METHOD: `POST` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:42.601853 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebacd-1aa0-4ea4-a42f-f0def760ddfe None None] SCRIPT_NAME: `/identity` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:42.601971 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebacd-1aa0-4ea4-a42f-f0def760ddfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=278915) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:42.620372 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.auth.core [None req-07cebacd-1aa0-4ea4-a42f-f0def760ddfe None None] MFA Rules not processed for user `4f6c30de67e643ecb5582d374f07cb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=278915) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 19 21:02:42.621586 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG keystone.common.fernet_utils [None req-07cebacd-1aa0-4ea4-a42f-f0def760ddfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=278915) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 19 21:02:42.629623 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: [pid: 278915|app: 0|req: 690/1372] 10.222.0.88 () {58 vars in 1023 bytes} [Thu May 19 21:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 19 21:02:42.673321 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005552d-0e50-46f2-8c68-76126dc1ec73 None None] Authenticating user token {{(pid=278914) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 19 21:02:42.688452 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c005552d-0e50-46f2-8c68-76126dc1ec73 None None] Validating token access rules against request {{(pid=278914) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 19 21:02:42.695333 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b32c50-1e45-4fb7-9b09-d5ecc5f8971e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77427e1028d349b4a740efe0a985e73b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eb86f5a389064e12bb38c500b225c9ac', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=278914) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 19 21:02:42.695986 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b32c50-1e45-4fb7-9b09-d5ecc5f8971e service nova] REQUEST_METHOD: `GET` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 19 21:02:42.696266 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b32c50-1e45-4fb7-9b09-d5ecc5f8971e service nova] SCRIPT_NAME: `/identity` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 19 21:02:42.696565 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b32c50-1e45-4fb7-9b09-d5ecc5f8971e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=278914) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 19 21:02:42.705739 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b32c50-1e45-4fb7-9b09-d5ecc5f8971e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 19 21:02:42.707352 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b32c50-1e45-4fb7-9b09-d5ecc5f8971e service nova] RBAC: Authorization granted {{(pid=278914) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 19 21:02:42.718445 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: [pid: 278914|app: 0|req: 683/1373] 10.222.0.88 () {60 vars in 1365 bytes} [Thu May 19 21:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 19 21:02:52.614535 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: DEBUG dbcounter [-] [278915] Writing DB stats keystone:SELECT=21 {{(pid=278915) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:02:52.713523 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: DEBUG dbcounter [-] [278914] Writing DB stats keystone:SELECT=82 {{(pid=278914) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 19 21:03:12.376095 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: Thu May 19 21:03:12 2022 - graceful shutdown triggered... May 19 21:03:12.375857 dv-cinder-841302-4-iscsi systemd[1]: Stopping Devstack devstack@keystone.service... May 19 21:03:12.376890 dv-cinder-841302-4-iscsi devstack@keystone.service[278914]: Gracefully killing worker 1 (pid: 278914)... May 19 21:03:12.377417 dv-cinder-841302-4-iscsi devstack@keystone.service[278915]: Gracefully killing worker 2 (pid: 278915)... May 19 21:03:13.376856 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: worker 1 buried after 1 seconds May 19 21:03:13.376856 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: worker 2 buried after 1 seconds May 19 21:03:13.377469 dv-cinder-841302-4-iscsi devstack@keystone.service[278902]: goodbye to uWSGI. May 19 21:03:13.378462 dv-cinder-841302-4-iscsi systemd[1]: devstack@keystone.service: Succeeded. May 19 21:03:13.379011 dv-cinder-841302-4-iscsi systemd[1]: Stopped Devstack devstack@keystone.service.